max_questions problem

Hi Gilles,
I don’t know I must be happy for this or no. I have 1,500 GB and 15,000 GB Transfer / Bandwidth. I sent also an email to host provider and received a reply that I need to make the coding of the site more static. I had a search on the net and find on the forum on mysql the following, but I am not familiar with coding so much and I don’t know how to fit it.
See:
http://forums.mysql.com/read.php?35,33301,34184
What do you think?
Kamran

----- Original Message ----
From: Gilles Vincent gilles.vincent@gmail.com
To: kamran Mir Hazar editor@kabulpress.org
Cc: spip-en@rezo.net
Sent: Friday, January 18, 2008 7:11:59 AM
Subject: Re: [Spip-en] max_questions problem

Your website certainly meets a great success : you then have too much
traffic for your website-provider restrictions. You should seriously
think about changing it if you can’t upgrade your service to an
unlimited bandwidth service.

.Gilles

On Jan 17, 2008 11:31 PM, kamran Mir Hazar <kamran_mirhazar@yahoo.com> wrote:

Hello,

I have face MYSQL max_questions problem for the user www.kabulpress.org/my
and it is due to the max_questions setting which is set to 50,000 per user,
per hour. I have created a new user for database and delete it, but still I
have problem after sometime, when I’m login to the SPIP CMS and log out.
Is there anybody who can help me in this issue?
Regards

Kamran


spip-en@rezo.net - http://listes.rezo.net/mailman/listinfo/spip-en


spip-en@rezo.net - http://listes.rezo.net/mailman/listinfo/spip-en

On 18/01/2008, at 3:11 PM, kamran Mir Hazar wrote:

Hi Gilles,
I don’t know I must be happy for this or no. I have 1,500 GB and 15,000 GB Transfer / Bandwidth. I sent also an email to host provider and received a reply that I need to make the coding of the site more static. I had a search on the net and find on the forum on mysql the following, but I am not familiar with coding so much and I don’t know how to fit it.
See:
http://forums.mysql.com/read.php?35,33301,34184
What do you think?
Kamran

The « solution » described on that page is:

  1. for database administrators (people who can update the mysql.users table, usually just the root
    and/or mysql users); and
  2. doesn’t fix the problem, it just turns off the restriction.

The problem is that your web host has enforced a limit of 50,000 queries per user, per hour and you are
exceeding this limit. To solve this problem you can do one or more of the following:

  1. ask your web host to increase or remove the limit (if they said « make the coding of the site more static »,
    then they probably won’t do this)

  2. add or increase the caching on your pages (add #CACHE tags to every template and make sure that the
    number is large enough – try doubling the current value)

  3. make your templates less complex – the more loops it takes to generate a page, the quicker you’ll
    reach 50,000 queries

  4. change to a host that doesn’t enforce this restriction

  5. make your site less popular – you probably don’t want to do this one.

Option 2 will probably be the quickest and easiest solution, if it is feasible. Option 3 may take considerable effort as you’ll essentially be rewriting your templates from scratch to, e.g. remove as many nested loops as possible, etc, and option 4 might be a good idea once your current hosting agreement (if they’re limiting the number of queries, what else might they be limiting?).

Options 1 and 5 are probably not possible or desirable. :slight_smile:

Hope this helps,

Thomas Sutton
Web Developer
bouncingorange
graphic + web design

On Jan 18, 2008 8:43 AM, Thomas Sutton <thomas@bouncingorange.com> wrote:

2) add or increase the caching on your pages (add #CACHE tags to every
template and make sure that the
number is large enough -- try doubling the current value)

Yes, this is something I've also forgotten a lot of time :wink:
For element included it must also be specified.

If you use <INCLURE>, the included loops are still interpreted, even
if the page is in cache (because SPIP uses a dynamic cache). So for
elements that doesn't depend on the context (except #SESSION, from
memory), you can change these inclusion instructions into #INCLURE

But if you really want to accelerate your website AND lower the number
of queries, I suggest you to use the plugin Expresso (it is inside
http://miroirspip.ventre.name/builds/plugins/plugins_dev.zip )
A short documentation is here : Expresso - SPIP-Contrib
An automatic translation is here : http://tinyurl.com/ywstts

This plugin is wonderfull !!

3) make your templates less complex -- the more loops it takes to generate
a page, the quicker you'll
reach 50,000 queries

You can use the wonderfull attribute 'var_profile' : just add it to
your url (&var_profile=1), if you are admin with the session cookie
activated : a panel appears with the complete list of the queries used
for calculating a page (with a lot of debug information).

If you see that a querie is always calculated even if it shouldn't
(for exemple if it's for a navigation menu that does'nt change really
often, you should use static include for your template -- ie. with
#INCLURE)

50'000 query in an hour. This seems a bit strange, but maybe it happens, as everyone said, it would depend on your templates and how you manage the cache.

SPIP use a cache system that will store a static copy of the page, for a limited amount of time. And only recompute it after that time.
Let's say you have 10 different main templates for your page, with ~20 queries per template.
Now, if you had only 1 instance of the page generated by each template (only one section, one article, one news item, etc...) and each template has a delay of refresh of 1h, this will only be ~200 requests per hour.

Now, you probably have more than one article in the site... probably you have thousands of them, multiplying this 200 requests an hour to get to >50'000 requests per hour.

Ok, now, the question is: DO YOU REALLY NEED TO REFRESH EVERY ARTICLE PAGE EVERY HOURS?
the fact is, the article contents won't change really often, an article could probably have a refresh time very very big. Personally, I usually use 365 days, and force a refresh when I do a correction to a published article (by pressing the "visit the article" from the private area). This way, the article pages are not recomputed too often.
This will apply to all the other templates probably.

Now, some part of the templates: forums, list of articles in a section, etc... will need to be refreshed more often. Usually, these parts are a small part of the template (most of it being navigation menus, etc. etc.), so you should split the templates in multiple part, using <INCLUDE> and #INCLUDE as needed.
The part of the templates that would need a different refresh rate (i.e. the navigation menu will probably be static, with a long refresh rate; while a list of article in a section will need to be refreshed at least once a day), then use <INCLUDE> and be sure to specify different #CACHE delays in each part included.
The part of the template that will be the same for many pages (header, etc...) and don't need to have a different delay from the main page can them be included with #INCLUDE.

Another place where a load of request can come from is the search page if you use SPIP search feature. This template is often configured to have no cache (a 0s delay) as the search results might need to be updated often.
If the visitors use the search page a lot and this template is "heavy", then it can be the source of your problem. Here again, try to split the template in different <INCLUDE> with different delay, leaving only the loop doing the search in a template with a null delay. You can also cache the inside part of the search loop with a longuer delay, this won't change too much:
<BOUCLE_articles(ARTICLES) {recherche} {0,10}>
<INCLUDE{fond=search_item}{id_article}>
</BOUCLE_articles>

You should really go through all the templates and think about their individual refresh delays.
Also check from the stats of the site which part of it gets the most traffic and concentrate on optimizing that template.

Good Luck

Pierre

PS: the percolateur plugin seems great, but I think it's limited to the 1.9.3dev version. And if I understand well, it's only gone make a difference IF you have already efficient CACHE delays

kamran Mir Hazar wrote:

Hi Gilles,
I don't know I must be happy for this or no. I have *1,500 GB* and *15,000 GB* Transfer / Bandwidth. I sent also an email to host provider and received a reply that I need to make the coding of the site more static. I had a search on the net and find on the forum on mysql the following, but I am not familiar with coding so much and I don't know how to fit it.
See:
http://forums.mysql.com/read.php?35,33301,34184
What do you think?
Kamran

----- Original Message ----
From: Gilles Vincent <gilles.vincent@gmail.com>
To: kamran Mir Hazar <editor@kabulpress.org>
Cc: spip-en@rezo.net
Sent: Friday, January 18, 2008 7:11:59 AM
Subject: Re: [Spip-en] max_questions problem

Your website certainly meets a great success : you then have too much
traffic for your website-provider restrictions. You should seriously
think about changing it if you can't upgrade your service to an
unlimited bandwidth service.

.Gilles

On Jan 17, 2008 11:31 PM, kamran Mir Hazar <kamran_mirhazar@yahoo.com <mailto:kamran_mirhazar@yahoo.com>> wrote:
>
&g t;
>
> Hello,
>
> I have face MYSQL max_questions problem for the user www.kabulpress.org/my <http://www.kabulpress.org/my&gt;
> and it is due to the max_questions setting which is set to 50,000 per user,
> per hour. I have created a new user for database and delete it, but still I
> have problem after sometime, when I'm login to the SPIP CMS and log out.
> Is there anybody who can help me in this issue?
> Regards
>
> Kamran
> _______________________________________________
> spip-en@rezo.net <mailto:spip-en@rezo.net> - http://listes.rezo.net/mailman/listinfo/spip-en
>
_______________________________________________
spip-en@rezo.net <mailto:spip-en@rezo.net> - http://listes.rezo.net/mailman/listinfo/spip-en

Hi

Based on my experience, 50,000 queries per hour is huge! even if all those who speak the language and have internet access connect to the site.
Before, looking in the templates and SPIP's cache, if I were you, I would check the server and the database log files to see what kind of queries these are.

George

Quoting Pierre Andrews <mortimer.pa@free.fr>:

50'000 query in an hour. This seems a bit strange, but maybe it happens,
as everyone said, it would depend on your templates and how you manage
the cache.

SPIP use a cache system that will store a static copy of the page, for a
  limited amount of time. And only recompute it after that time.
Let's say you have 10 different main templates for your page, with ~20
queries per template.
Now, if you had only 1 instance of the page generated by each template
(only one section, one article, one news item, etc...) and each template
has a delay of refresh of 1h, this will only be ~200 requests per hour.

Good Luck

Pierre

----- Original Message ----
From: Gilles Vincent
<gilles.vincent@gmail.com>
To: kamran Mir Hazar <editor@kabulpress.org>
Cc: spip-en@rezo.net
Sent: Friday, January 18, 2008 7:11:59 AM
Subject: Re: [Spip-en] max_questions problem

Your website certainly meets a great success : you then have too much
traffic for your website-provider restrictions. You should seriously
think about changing it if you can't upgrade your service to an
unlimited bandwidth service.

.Gilles

On Jan 17, 2008 11:31 PM, kamran Mir Hazar
<kamran_mirhazar@yahoo.com
<mailto:kamran_mirhazar@yahoo.com>> wrote:
>
&g t;
>
> Hello,
>
> I have face MYSQL max_questions problem for the user
www.kabulpress.org/my <http://www.kabulpress.org/my&gt;
> and it is due to the max_questions setting which is set to 50,000 per
user,
> per hour. I have created a new user for database and delete it, but
still I
> have problem after sometime, when I'm login to the SPIP CMS and log out.
> Is there anybody who can help me in this issue?
> Regards
>
> Kamran
> _______________________________________________