? ? Failure

User tests: Successful: Unsuccessful:

avatar brianteeman
brianteeman
17 Jul 2015

Since php 5.3.9 there is a variable max_input_vars with a default value of 1000.

Ever had a large site suddenly stopped saving changes - this is why!!

Unfortunately on sites with lots of usergroups, menu items, modules and/or categories this will create issues when saving or making changes. Sadly it is a silent error as it is only a notice in php so depending on your site and server error configuration you might never know about it. All you do know is that you press save and nothing happens.

This plugin works in the same way as the old EOSNOTIFY plugin as a quickicon plugin and is only fired on the admin home screen.

It takes a count of the usergroups, menu items, modules and categories and displays either an error message if the total is greater than the php value for max_input_vars or a warning message if the total is greater than 80% of the available variables.

This can only be an estimate of the variables in use. The only other way would be to count the ones in use on every page and this would be an unnecessary overhead.

To test - Apply the patch, discover the plugin and enable it and then all I can think of is to add echo $varcount and echo $maxinputvars to your admin template and check that $varcount is less than 80$ of $maxinputvars.

Warning Screen

warning

Error Screen

error

thanks to @zero-24 and @rdeutz who helped me to create this plugin (my first)

Votes

# of Users Experiencing Issue
1/1
Average Importance Score
5.00

6830aac 17 Jul 2015 avatar brianteeman fix
avatar brianteeman brianteeman - open - 17 Jul 2015
avatar brianteeman brianteeman - change - 17 Jul 2015
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 17 Jul 2015
Labels Added: ? ?
avatar brianteeman brianteeman - change - 17 Jul 2015
The description was changed
avatar brianteeman
brianteeman - comment - 17 Jul 2015

Another method to test would be to massively reduce the setting for max_input_vars on your site from say 1000 to 100

With the sample testing data this will easily trigger the error message and with a clean no sample data install you should see the warning.

Another option would be to use com_overload from @nikosdion to create a crazy amount of data

avatar Bakual
Bakual - comment - 18 Jul 2015

What I wonder is why you're counting the categories, menuitems, modules and usergroups all together.

Worst case I see is the module form where it sends all usergroups (permissions) and menuitems (menu assigment). The modules (module assignment) are sent in a menu item form.
I don't see a form where we send all categories. And especially not all four (or even three) groups together.

So I think the calculated number will be way to high depending on the site, giving a warning when there is no issue at all.

Or do I miss a form where we potentially send all four groups?

avatar Bakual
Bakual - comment - 18 Jul 2015

There is a codestyle issue:

FILE: ...me/travis/build/joomla/joomla-cms/plugins/quickicon/maxvars/maxvars.php
--------------------------------------------------------------------------------
FOUND 1 ERROR(S) AFFECTING 1 LINE(S)
--------------------------------------------------------------------------------
 80 | ERROR | Whitespace found at end of line
avatar Fedik
Fedik - comment - 18 Jul 2015

Additionally to @Bakual question about categories,
A lot User groups can make the problem for Super Admin, Admin and user who have access to edit Access Rules ...
I think display this message for everyone can confuse them ( eg Author, Manager and other users with limited access)

avatar brianteeman
brianteeman - comment - 18 Jul 2015

It confuses them much more when they save a module and no changes are made
On 18 Jul 2015 10:57 am, "Fedir Zinchuk" notifications@github.com wrote:

Additionally to @Bakual https://github.com/Bakual question about
categories,
A lot usergroups can make the problem for Super Admin, Admin and user
who have access to edit Access Rules ...
I think display this message for everyone can confuse them ( eg Author,
Manager and other users with limited access)


Reply to this email directly or view it on GitHub
#7456 (comment).

avatar brianteeman
brianteeman - comment - 18 Jul 2015

An author will never see message it is administration only as you will see
when you try it.
On 18 Jul 2015 11:00 am, "Brian Teeman" brian@teeman.net wrote:

It confuses them much more when they save a module and no changes are made
On 18 Jul 2015 10:57 am, "Fedir Zinchuk" notifications@github.com wrote:

Additionally to @Bakual https://github.com/Bakual question about
categories,
A lot usergroups can make the problem for Super Admin, Admin and user
who have access to edit Access Rules ...
I think display this message for everyone can confuse them ( eg Author,
Manager and other users with limited access)


Reply to this email directly or view it on GitHub
#7456 (comment).

avatar Fedik
Fedik - comment - 18 Jul 2015

I just mean that they do not have access to edit this field (Access Rule), and so they do not have this problem :wink:

ps. I have nothing against this pull, the idea is very good ... just my thoughts :wink:

avatar brianteeman
brianteeman - comment - 18 Jul 2015

And your point is what.

On 18 July 2015 at 11:02, Fedir Zinchuk notifications@github.com wrote:

I just mean that they do not have access to edit this field (Access Rule),
and so they do not have this problem [image: :wink:]


Reply to this email directly or view it on GitHub
#7456 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

avatar brianteeman
brianteeman - comment - 18 Jul 2015

Let me be really clear as people are not reading so I have to repeat

This can only be an estimate of the variables in use. The only other way would be to count the ones in use on every page and this would be an unnecessary overhead.

avatar Fedik
Fedik - comment - 18 Jul 2015

:speak_no_evil:

avatar roland-d
roland-d - comment - 18 Jul 2015

Pages with too many items, would be great to ajaxify them. This solves this problem as well.

avatar brianteeman
brianteeman - comment - 18 Jul 2015

@roland-d how on this earth would that solve the problem - but its off topic anyway

avatar roland-d
roland-d - comment - 18 Jul 2015

@brianteeman It solves it by not having to post the huge number of variables. I shouldn't have posted it here as you said, it's off-topic.

avatar Bakual
Bakual - comment - 18 Jul 2015

Let me be really clear as people are not reading so I have to repeat

This can only be an estimate of the variables in use. The only other way would be to count the ones in use on every page and this would be an unnecessary overhead.

Don't assume people don't read. That's not nice.

I wondered why you got to that estimate, as I can't think of a form where this would apply.
Let's assume we have 300 items in each of the tables in question. With your calculation it would show a big error as the count would be 1200. However in practice it would be max 600+something in the module form. Still far away from the limit. Even if there is a form with all three groups present (which I'm not aware) it would still work.
Imho, it doesn't make sense to show warnings if there is no issue at all.

It doesn't mean we have to count for each page, but we should maybe think about the cases and check if the problematic combinations yield an issue.

So instead of checking the total, check eg $menuitems + $usergroups.

avatar JoomliC
JoomliC - comment - 18 Jul 2015

@brianteeman thank you opening this PR! :+1:
This could apply to config for any extensions having a lot of settings.

There's suhosin to be checked too.
What's about something using client-side to control each form before submit like this one for WP : https://github.com/academe/wp-max-submit-protect/blob/master/wp-max-submit-protect.php ?
The script files are MIT licenced, and it seems to be easily integrated into a Joomla plugin.

avatar btoplak
btoplak - comment - 18 Jul 2015

This is good idea to inform users about possible issue.

Later on the plugin could go further to prevent submitting a form which would loose some data, like the WP plugin mentioned by @JoomliC

All should take into account that this limit is there for a reason. Originally it was made to mitigate some DoS attack vector (CVE-2011-4885). So, don't push it too high. Maybe you could include such remark in plugin's notice?

avatar btoplak
btoplak - comment - 18 Jul 2015

P.S. @JoomliC mentioned Suhosin.

suhosin.post.max_vars
suhosin.request.max_vars
suhosin.get.max_vars

Those variables have the same function as max_input_vars, so if they are set the problem would be the same. I'd suggest you check them too and take the lowest value as a threshold.

avatar brianteeman
brianteeman - comment - 18 Jul 2015

OK tried to help users with this. This is a real world problem . yiuve spoken clearly and I wontnwaste my time again

avatar brianteeman brianteeman - change - 18 Jul 2015
Status Pending Closed
Closed_Date 0000-00-00 00:00:00 2015-07-18 12:41:40
Closed_By brianteeman
avatar brianteeman brianteeman - close - 18 Jul 2015
avatar brianteeman brianteeman - close - 18 Jul 2015
avatar brianteeman brianteeman - change - 18 Jul 2015
Status New Closed
avatar btoplak
btoplak - comment - 18 Jul 2015

It was a good idea.

avatar brianteeman
brianteeman - comment - 21 Jul 2015

Here is an example of what happens on a site with many modules and only a few menu items if you hit the limit of max_vars_input. Nice how Joomla says everything is saved and working perfectly and how nothing is actually saved at all.

maxvars2

avatar brianteeman brianteeman - change - 21 Jul 2015
Status Closed New
Closed_Date 2015-07-18 12:41:39
Closed_By brianteeman
avatar brianteeman brianteeman - reopen - 21 Jul 2015
avatar brianteeman brianteeman - change - 24 Jul 2015
Category Plugins
avatar Fedik
Fedik - comment - 27 Jul 2015

just got idea how to count this thing with better precision for an each form, and without SQL involve :smile:
it can be a simple JavaScript code:

if(form.length >= maxVars){
  Joomla.renderMessages({'warning':[Joomla.JText._('PLG_QUICKICON_MAXVARS_WARN'])})
}

just not sure where the right place for add this code, maybe onContentPrepareForm?
and how safe to expose the max_input_vars value on the client side? hm, should be safe enough, if do it only for admin

avatar dgt41
dgt41 - comment - 27 Jul 2015

@Fedik why not a function on core.js and a call from submit (or something like that) ?

avatar Fedik
Fedik - comment - 27 Jul 2015

@dgt41 then this means that User will see the warning only when push "submit" (save, cancel .. so on)
not sure that it is good idea :wink:
And we still need to have maxVars (max_input_vars) value on the client side , somehow (we still do not have a simple way to add custom options #3072 :smile: )

I see two way:
1. Add this code by content plugin, in onContentPrepareForm event
2. Add this code directly to the form layout, in each component in edit.php ... by new JHtml::behavior method

but not sure, maybe there a better way

avatar dgt41
dgt41 - comment - 27 Jul 2015

@Fedik maybe just manipulating the save, the save and close and the save and new buttons to inject this script (or actually prevent the usage of those buttons, disabled) with conjunction of the plugin that will raise the warning, is easier to implement?

avatar Fedik
Fedik - comment - 27 Jul 2015

@dgt41 I thought enough just show the warning,
I not sure that we need to prevent anything, form will not be saved anyway (in most cases),
but yes, we can do this, if need :wink:

ok, so I vote for "make new behavior", name something like JHtml::_('behavior.maxvartest');, so we can add it in any form that we need, and extension developers also can use it in their own extensions.

avatar Fedik
Fedik - comment - 27 Jul 2015

hm, @dgt41 I think better warn User about possible problem before he/she fill the form, than make "surprise" after he/she spend some time to fill the form

avatar dgt41
dgt41 - comment - 27 Jul 2015

@Fedik I am not gonna support my my own idea, because there is a way better option here: ajaxify these permissions. @phproberto already showcased this: https://www.youtube.com/watch?v=vNZZhi7WB-c

avatar brianteeman
brianteeman - comment - 27 Jul 2015

thats only part of the issue with max_input_vars and until @phproberto
contributes that code it doesnt exist

On 27 July 2015 at 16:17, Dimitris Grammatiko notifications@github.com
wrote:

@Fedik https://github.com/Fedik I am not gonna support my my own idea,
because there is a way better option here: ajaxify these permissions.
@phproberto https://github.com/phproberto already showcased this:
https://www.youtube.com/watch?v=vNZZhi7WB-c


Reply to this email directly or view it on GitHub
#7456 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

avatar Fedik
Fedik - comment - 27 Jul 2015

@dgt41 well, sure that would be a very good solution :wink:
but by looking through the window of reality I see very low chance get something like that in short term :smile:

avatar Fedik
Fedik - comment - 29 Jul 2015

so there is JavaScript way #7581 to test max_input_vars limit

avatar brianteeman
brianteeman - comment - 29 Oct 2015

Closed due to lack of interest

avatar brianteeman brianteeman - change - 29 Oct 2015
Status New Closed
Closed_Date 0000-00-00 00:00:00 2015-10-29 00:29:42
Closed_By brianteeman
avatar brianteeman brianteeman - close - 29 Oct 2015
avatar brianteeman brianteeman - head_ref_deleted - 29 Oct 2015
avatar YaegerDesign
YaegerDesign - comment - 21 Mar 2016

This little plugin would have helped save us several hours of troubleshooting and a client several years of (potentially) lost data!


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/7456.

Add a Comment

Login with GitHub to post a comment