?
avatar alastairwalker
alastairwalker
23 Nov 2016

Steps to reproduce the issue

A fresh installation of Joomla 3.6.4 (i.e. latest version). Installed with sample data.

Error condition: A 'Out of sort memory' error is reported by MYSQL - Out of sort memory,
'consider increasing server sort buffer size' is reported in the heading to all activities in the Administrator interface.

Applicable sql is:

SQL=SELECT a.*, SUM(b.home) AS home,b.language,l.image,l.sef,l.title_native
FROM #__menu_types AS a
LEFT JOIN #__menu AS b ON b.menutype = a.menutype AND b.home != 0
LEFT JOIN #__languages AS l ON l.lang_code = language
WHERE (b.client_id = 0 OR b.client_id IS NULL)
GROUP BY a.id, a.menutype, a.description, a.title, b.menutype,b.language,l.image,l.sef,l.title_native

Simply paste this into the sql panel of phpMyAdmin, in the context of the tables of the freshly installed JM version, and the error will be reported.

Expected result

In line 47, the 'try' should result in line 49 being executed.

Actual result

This error is thrown in administrator/modules/mod_menu/helper.php line 54

System information (as much as possible)

Context:
Joomla 3.6.4 Full version
Apache: Server Version: Apache/2.4.6
OS: Win32 Windows 7 Professional
PHP: 5.4.19
MySQL: 5.0.10

(Note: Uniform server version 8.9.2 Coral was used as the WAMP).

Additional comments

I tried increasing the MYSQL allocated sort memory - from a default value of 139K to 32M - to no effect.
The query structure appears to be problematic.

avatar alastairwalker alastairwalker - open - 23 Nov 2016
avatar brianteeman
brianteeman - comment - 23 Nov 2016

After modifying the query to match my database prefix I could not replicate this

Mysql settings are

sort_buffer_size = 512K

Setting to 32k does give the error you describe (139k worked for me) but changing it to 512k resolves it
Did you remember to restart mysql AFTER you made the change?

Not sure where you got 139k as the default value from? Its not the mysql own default https://dev.mysql.com/doc/refman/5.6/en/server-system-variables.html#sysvar_sort_buffer_size

You stated that your version of mysql is 5.0.10
If that is correct then it is below the minimum requirement (see https://downloads.joomla.org/technical-requirements)
5.0.10 was released July 2005


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/12983.

avatar alastairwalker
alastairwalker - comment - 23 Nov 2016

I stand to be corrected - the mysql version is actually 5.5.33.
Subsequently, I have also installed the Joomla 3.6.2 under latest issue of Uniform server V 13.2.1 (Zero13). But I get the same problem.

Regarding the fresh setting of the sort value, yes - I did stop and restart the both servers (Apache and Mysql).

avatar brianteeman
brianteeman - comment - 23 Nov 2016

Not really sure what to say. With sensible mysql default values everything
seems ok to me

avatar alastairwalker
alastairwalker - comment - 23 Nov 2016

One further comment, if the reduce the extent of the group expression and leave off the last three fields i.e.
change from

$query->group('a.id, a.menutype, a.description, a.title, b.menutype,b.language,l.image,l.sef,l.title_native');
to
$query->group('a.id, a.menutype, a.description, a.title, b.menutype,b.language');

then the error condition goes away.

So - a question, is it really necessary in the sql design to include all those fields in the 'group' statement?
What will 'break' (elsewhere) if the last three fields are omitted?

avatar mbabker
mbabker - comment - 23 Nov 2016

For Joomla's claimed support of other databases, the extra fields have to be in the group clause. IIRC it was SQL Server that was more strict about all fields that are SELECTed needing to be in the GROUP clause if used.

avatar ggppdk
ggppdk - comment - 23 Nov 2016

SELECT has the aggregate function

So GROUP-BY is required to be there,

now what happens if you leave it out ?

If you leave out the GROUP-BY,
and you do have an AGGREGATE (Group) function in the SELECT

  • then SQL Server (and PostgreSQL ??) will complain
  • in MySQL , (please correct me here) there is a setting "ONLY_FULL_GROUP_BY" which by default it is OFF, which means it is not required,then it is equivalent to grouping on all columns of the SELECT

So in any case for ALL DBs the Group-by will happen ...,
just in MySQL it could have been skipped and thus be "silently" implied

[EDIT]
and if you change the group-by to make it shorter then it is a different query, than it would have been if you had not written it at all

avatar brianteeman
brianteeman - comment - 23 Nov 2016

To be honest this looks like another example where uniformserver just isnt
very good

On 23 November 2016 at 16:31, Georgios Papadakis notifications@github.com
wrote:

SELECT has the aggregate function

So GROUP-BY is required to be there,

now what happens if you leave it out ?

If you leave out the GROUP-BY,
and you do have an AGGREGATE (Group) function in the SELECT

  • then SQL Server (and PostgreSQL ??) will complain
  • in MySQL , (please correct me here) there is a setting "ONLY_FULL_GROUP_BY" which by default it is OFF, which means it is not required,then it is equivalent to grouping on all columns of the SELECT

So in any case for ALL DBs the Group-by will happen ...,
just in MySQL it could have been "silently" skipped


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#12983 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABPH8SXiXw6I55VDzqTiR15obqO_sULPks5rBGpmgaJpZM4K6R1h
.

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
https://brian.teeman.net/ http://brian.teeman.net/

avatar ggppdk
ggppdk - comment - 23 Nov 2016

One further comment, if the reduce the extent of the group expression and leave off the last three fields i.e. change from

->group('a.id,a.menutype,a.description,a.title,b.menutype,b.language,l.image,l.sef,l.title_native');
to
->group('a.id,a.menutype,a.description,a.title,b.menutype,b.language');

It can be done

  • but the LEFT Join with the languages table will need to be removed, (maybe be done and it can be done keep the LEFT Join, i am not sure without further investigation, but removing the LEFT join on language will give even more benefits, see below)

and then a 2nd separate query will be executed, to get the language data
and then add them to the menus array, via a PHP loop

this will not only reduce SQL memory usage,
but it should also give a measurable performance difference to the overall execution of the page

a lot of duplicated data (the language) are returned by the SELECT statement without real need,
but most important is not their effect on the SELECT but their participation in the GROUP-BY

avatar ggppdk
ggppdk - comment - 23 Nov 2016

I have made a PR here: #12991

avatar csthomas
csthomas - comment - 10 Feb 2017

A new PR at #14011

avatar brianteeman
brianteeman - comment - 10 Mar 2017

Can this be closed as we have a pr @zero-24

avatar Bakual Bakual - change - 10 Mar 2017
Status New Closed
Closed_Date 0000-00-00 00:00:00 2017-03-10 22:38:49
Closed_By Bakual
avatar Bakual Bakual - close - 10 Mar 2017

Add a Comment

Login with GitHub to post a comment