? Success
Pull Request for # 6246

User tests: Successful: Unsuccessful:

avatar Erftralle
Erftralle
2 Mar 2015

This is a fix for issue #6246.

After the recent changes (see #3292) the query does not deliver results as expected because the fields created, id, title were added to the group by clause. The result list actually contains all published archived articles of the last X month. Therefore the module output contains multiple entries for the same month/year.
So these fields have to be removed from the grouping again.

In standard SQL it is not allowed to have nonaggregated columns in the select list which are not in the group by columns, so that is the reason for removing the ->select('created, id, title').

But we need a created for further calculations.

Therefore I added a ->select('MIN(' . $db->quoteName('created') . ') AS created') which is an aggregate column and therefore does not need to be part of the group by. It contains the smallest date of all articles for a grouped month/year.

For test instructions see also #6246.
Make sure to have more than one archived published article for each month/year and check if you have multiple entries for the same month/year.
Before applying the patch you will have, after the patch you should not have any duplicates.

avatar Erftralle Erftralle - open - 2 Mar 2015
avatar joomla-cms-bot joomla-cms-bot - change - 2 Mar 2015
Labels Added: ?
avatar zero-24 zero-24 - change - 2 Mar 2015
Rel_Number 6246
Relation Type Pull Request for
Easy No Yes
avatar zero-24 zero-24 - change - 2 Mar 2015
Category Modules
avatar n9iels
n9iels - comment - 3 Mar 2015

@test works fine for me, thanks!


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/6263.
avatar n9iels n9iels - test_item - 3 Mar 2015 - Tested successfully
avatar waader
waader - comment - 5 Mar 2015

@test works with postgresql but not with mssql. Please look at #5959 where the respective was made.
Maybe @pmorrisarctg can help here.

avatar Erftralle
Erftralle - comment - 6 Mar 2015

Thanks for testing!
It should now work with MSSQL too.

avatar sovainfo
sovainfo - comment - 7 Mar 2015

Suggest performance improvement. No need to waste time on manipulating the date. Just order by created DESC.

avatar Erftralle
Erftralle - comment - 7 Mar 2015

@sovainfo: Unfortunately your suggestion does not work with MSSQL.

avatar sovainfo
sovainfo - comment - 7 Mar 2015

Tried to fix that, but solving one issue returns another. Guess the only proper solution would be to have an attribute archive or a function that returns YYYYMM.

avatar alikon
alikon - comment - 8 Mar 2015

with this PR the query issued is:

SELECT EXTRACT (MONTH 
  FROM "created") AS created_month,MIN("created") AS created,EXTRACT (YEAR 
  FROM "created") AS created_year
  FROM scaku_content
  WHERE state = 2 
  AND checked_out = 0
  GROUP BY EXTRACT (YEAR 
  FROM "created"), EXTRACT (MONTH 
  FROM "created")
  ORDER BY EXTRACT (YEAR 
  FROM "created") DESC, EXTRACT (MONTH 
  FROM "created") DESC 
  LIMIT 10

could be simplyfied like this

SELECT EXTRACT (MONTH FROM "created") AS created_month,
EXTRACT (YEAR FROM "created") AS created_year
  FROM scaku_content
  WHERE state = 2 
  AND checked_out = 0
  group by created_month,created_year
  ORDER BY created_month DESC, created_year DESC 
  LIMIT 10
avatar Erftralle
Erftralle - comment - 8 Mar 2015

@alikon:
No, your suggestion does not work with MSSQL (regarding to the group by and order by part of the clause).

And even more, since the created column is missing in your suggestion it would not work with any database engine at all.

avatar alikon
alikon - comment - 9 Mar 2015

i'm not able to test on MSSQL so i trust you
,but, to be precise here is how that query works fine both on postgresql
image

and on Mysql

localhost 127 0 0 1 m340a phpmyadmin 4 1 6

avatar Erftralle
Erftralle - comment - 9 Mar 2015

Thanks a lot, that you trust me :-) .

Did you have a look at the second commit I have pushed.
I don't think so, otherwise you would have seen the changes I made to make the query work with MSSQL.

What you are showing me with the pictures just proves that the queries do not throw a fatal error which means that they are syntactical correct.

But they have to work also in the content of the module. The column created is still missing in your query which is used in the module's code for further calculations. So trust me again, your query does not work.

avatar alikon
alikon - comment - 9 Mar 2015

The created column can be not necessary be extracted from the query as you can in an alternative way extract the month name from month number with some php code

avatar Erftralle
Erftralle - comment - 9 Mar 2015

The created column can be not necessary be extracted from the query as you can in an alternative way extract the month name from month number with some php code

Sorry, not sure if I understand you correctly. Does it mean, that you want me to rewrite the module to adapt to your query suggestion? Does it than also work with MSSQL? Did you check that? Please provide some complete code, e.g. with a pull request against mine, which can be checked by me an others. Thank you.

avatar sovainfo
sovainfo - comment - 9 Mar 2015

@alikon MSSQL doesn't allow the created_year and created_month on the group, it requires the functions as mentioned in the select.

avatar alikon
alikon - comment - 9 Mar 2015

@Erftralle Don't get me wrong , I really appreciate your work, it's not so common when someone take care of all supported DB's , I'm just wondering if we can explore an alternative solution to avoid possible performance issue , not more

avatar alikon
alikon - comment - 9 Mar 2015

i've got some tests

image

  • so it seems that we can not execute at least the year() and month() functions on ORDER BY clause

  • for eliminate the created column

foreach ($rows as $row)
{
 $monthName  = date("F", mktime(0, 0, 0, $row->created_month, 10));
 $lists[$i] = new stdClass;
 $lists[$i]->link = JRoute::_('index.php?option=com_content&view=archive&year=' . $row->created_year . '&month=' . $monthName . $itemid);
 $lists[$i]->text = JText::sprintf('MOD_ARTICLES_ARCHIVE_DATE', $monthName,  $row->created_year);
 $i++;
}
avatar sovainfo
sovainfo - comment - 9 Mar 2015

@alikon Combine that with:
SELECT DISTINCT(YEAR([created]) * 100 + MONTH([created])) AS [YM]
FROM [j336_content]
ORDER BY [YM] DESC
Extract the month from $row->YM, last 2 positions.

avatar Erftralle
Erftralle - comment - 10 Mar 2015

First of all, thanks for your testing and your proposals.

@alikon:

so it seems that we can not execute at least the year() and month() functions on ORDER BY clause

Can't confirm. When using the following query

        $query->select($query->month($db->quoteName('created')) . ' AS created_month')
            ->select($query->year($db->quoteName('created')) . ' AS created_year')
            ->from('#__content')
            ->where('state = 2 AND checked_out = 0')
            ->group($query->year($db->quoteName('created')) . ', ' . $query->month($db->quoteName('created')))
            ->order('created_year DESC, created_month DESC');

I get the following error message with MSSQL

[Microsoft][SQL Server Native Client 11.0][SQL Server]Invalid column name 'created_year'.
SQL=SELECT MONTH([created]) AS created_month,YEAR([created]) AS created_year , ROW_NUMBER() OVER (ORDER BY created_year DESC, created_month DESC) AS RowNumber FROM l72ei_content WHERE state = 2 AND checked_out = 0 GROUP BY YEAR([created]), MONTH([created])

for eliminate the created column

The link text must be a translated month name, not a number.
You are neither considering time offsets nor localization, which JDate does.

@sovainfo:

Unfortunately your query also does not work with MSSQL.
When using the following query

        $query->select('DISTINCT('. $query->year($db->quoteName('created')) . ' * 100 + ' . $query->month($db->quoteName('created')) . ') AS created_ym')
            ->from('#__content')
            ->where('state = 2 AND checked_out = 0')
            ->order('created_ym DESC');

I get the following error message with MSSQL

[Microsoft][SQL Server Native Client 11.0][SQL Server]Invalid column name 'created_ym'.
SQL=SELECT DISTINCT(YEAR([created]) * 100 + MONTH([created])) AS created_ym , ROW_NUMBER() OVER (ORDER BY created_ym DESC) AS RowNumber FROM l72ei_content WHERE state = 2 AND checked_out = 0 

That both of the queries are throwing errors might be caused by the processLimit() function of JDatabaseQuerySqlsrv.

avatar sovainfo
sovainfo - comment - 10 Mar 2015

Indeed, it requires those order and group statements. So, if we can't get rid of them it leaves only the removal of the year and month from the select. Checked this against all three in SQL, not in PHP.
It might be worth to remove them for better performance on large sites.

avatar sovainfo sovainfo - test_item - 10 Mar 2015 - Tested successfully
avatar alikon
alikon - comment - 10 Mar 2015

We are hitting the real point
processLimit() function of JDatabaseQuerySqlsrv.

If we can have a requirement for mssql >= 2012
ie SQL:2008 compliant we can rewrite the limit function in a more modern way

Select* from table Offset 1 rows fetch next 10 rows only

But this is outside the scope of this pr

avatar wilsonge
wilsonge - comment - 25 Mar 2015

This has conflicts. If this is still a valid issue can you update the PR please :)

avatar wilsonge
wilsonge - comment - 25 Mar 2015

This breaks the URLs btw for the articles (i'm guessing removing the id from the select does that)

avatar sovainfo
sovainfo - comment - 25 Mar 2015

@wilsonge Can you elaborate?
This is not about article urls, it is about archive urls. The row->id was removed because it wasn't used anywhere.

avatar Erftralle
Erftralle - comment - 25 Mar 2015

This has conflicts. If this is still a valid issue can you update the PR please :)

Yes, it is. Rebased the pull request.

This breaks the URLs btw for the articles (i'm guessing removing the id from the select does that)

Agree to @sovainfo. The article id has never been used in the module code. If something is breaking the URLs it has for sure nothing to do with the changes suggested in this pull request.

avatar sovainfo
sovainfo - comment - 4 Apr 2015

Verified correct working of this PR on all platforms (MySQL, PGSQL and MSSQL). It works!
The archive view of com_content has some issues. With SEF on it doesn't get the Year right. Also it uses published_up instead of created (as in the module).

At least this PR is fixing multiple rows where only 1 row per month is expected.
Guess that fixing determining the archive (created vs published_up) should be in a separate PR.

avatar zero-24
zero-24 - comment - 11 Apr 2015

@test successful moving to RTC. The other issues can be handeld by a new PR or disable SEF thanks @Erftralle

avatar zero-24 zero-24 - change - 11 Apr 2015
Labels Added: ?
avatar zero-24 zero-24 - change - 11 Apr 2015
Status Pending Ready to Commit
avatar zero-24 zero-24 - test_item - 11 Apr 2015 - Tested successfully
avatar roland-d roland-d - reference | - 20 Apr 15
avatar roland-d roland-d - merge - 20 Apr 2015
avatar roland-d roland-d - change - 20 Apr 2015
Status Ready to Commit Closed
Closed_Date 0000-00-00 00:00:00 2015-04-20 13:26:03
Closed_By roland-d
Labels Removed: ? ?
avatar roland-d roland-d - close - 20 Apr 2015
avatar roland-d roland-d - close - 20 Apr 2015
avatar zero-24 zero-24 - close - 20 Apr 2015
avatar zero-24 zero-24 - change - 14 Oct 2015
Labels Removed: ?

Add a Comment

Login with GitHub to post a comment