? ? Failure

User tests: Successful: Unsuccessful:

avatar csthomas
csthomas
18 Dec 2016

Competition for #11938

Summary of Changes

Fix various problems with sqlsrv queries:

  • Add alias to column (or expression) which does not have it, but it is required if query will use OFFSET, which means use wrapped query to emulate offset. Every column in SELECT statement required alias if it is placed in subquery.
  • If SELECT contains a wild column x.* then load and put all columns from x table to GROUP BY
  • If GROUP BY contains alias to (column or expression) then replace it with expression.
    Ex:
    SELECT LOWER(name) AS name, catid AS cc, count(*) ... GROUP BY name, cc
    will be replaced to
    SELECT LOWER(name) AS name, count(*) ... GROUP BY LOWER(name), catid
  • Add column from ORDER BY to GROUP BY if not yet exists. If this is an alias then replace it with expression as mentioned earlier.
  • Add columns from SELECT to GROUP BY if missing (related to mysql ONLY_FULL_GROUP_BY). Probably it would be better to fix joomla queries than create such workaround.
    Example of fixed query:
    SELECT id, name, COUNT(*) FROM #__x GROUP BY id
    will be replaced by:
    SELECT id, name, COUNT(*) AS columnAlias0 FROM #__x GROUP BY id, name

Fixed issue: #11279

Removes notices, warnings and errors from:

  • /administrator/index.php?option=com_banners
  • /administrator/index.php?option=com_categories&extension=com_banners
  • /administrator/index.php?option=com_banners&view=clients
  • /administrator/index.php?option=com_finder&view=maps
  • /administrator/index.php?option=com_categories&extension=com_newsfeeds
  • /administrator/index.php?option=com_categories&extension=com_contact
  • /administrator/index.php?option=com_postinstall

Fix issue with Popular Tags module on the front end.

Testing Instructions

Be sure that you can see PHP Notices.
Test above links and module Popular Tags before and after patch.

  • Install a new staging version of joomla (without sample data because currently joomla on mssql is broken - menu assets)
  • Create a category and two articles with a few tags.
  • Be sure that you have module Popular Tags enabled (ex. position-7)
  • Go to the front page
  • I should see some similar sql error as on image but with less details

joomla_win_3 6 6

Documentation Changes Required

No

avatar csthomas csthomas - open - 18 Dec 2016
avatar csthomas csthomas - change - 18 Dec 2016
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 18 Dec 2016
Category MS SQL Libraries
avatar csthomas csthomas - change - 18 Dec 2016
Labels Added: ?
avatar csthomas csthomas - change - 19 Dec 2016
The description was changed
avatar csthomas csthomas - edited - 19 Dec 2016
avatar joomla-cms-bot joomla-cms-bot - change - 19 Dec 2016
Category MS SQL Libraries MS SQL Libraries Unit Tests
avatar csthomas csthomas - change - 19 Dec 2016
Labels Added: ?
avatar csthomas csthomas - change - 19 Dec 2016
The description was changed
avatar csthomas csthomas - edited - 19 Dec 2016
avatar csthomas csthomas - change - 19 Dec 2016
The description was changed
avatar csthomas csthomas - edited - 19 Dec 2016
avatar csthomas csthomas - edited - 19 Dec 2016
avatar csthomas csthomas - change - 20 Dec 2016
Title
[WIP] More advanced MSSQL query workaround for limit, offset and for sql with aggregate functions
Advanced MSSQL query workaround for limit, offset and for sql with aggregate functions
avatar csthomas csthomas - change - 20 Dec 2016
The description was changed
avatar csthomas csthomas - edited - 20 Dec 2016
avatar csthomas csthomas - change - 4 Jan 2017
The description was changed
avatar csthomas csthomas - edited - 4 Jan 2017
avatar csthomas
csthomas - comment - 7 Jan 2017

@waader Which URL page or sql query did not work?

avatar waader
waader - comment - 7 Jan 2017

With current staging, testing data and your patch applied just go to the frontend. There are a lot of notices (something in the module helper). I have already shutdown my test pc, so I could give you the exact messages tomorrow.

avatar csthomas
csthomas - comment - 8 Jan 2017

I have found a few errors and I fixed it.
Please inform me if you find some errors.

avatar alikon
alikon - comment - 8 Jan 2017

i'm testing on backend and seems ok, unfortunately even if is not caused by this pr i see issues on frontend too as reported by @waader
can you have a look at frontend with current staging and testing data
thanks in advance

avatar waader
waader - comment - 8 Jan 2017

With testing data installed frontend gives an "404 Component not found" error. Starting with index.php?option=com_content the page loads. The module helper notices are also present and no modul content is displayed.

In backend I get an error when deleting Banner Tracks:
An expression of non-boolean type specified in a context where a condition is expected, near ')'.

avatar waader
waader - comment - 8 Jan 2017

Can you have a look at Smart Search > Content Maps:

[Microsoft][SQL Server Native Client 11.0][SQL Server]Column "d.branch_title" is invalid in the ORDER BY clause because it is not contained in either an aggregate function or the GROUP BY clause.

avatar csthomas
csthomas - comment - 8 Jan 2017

I will take a look. Are you sure do you use the last commit in this PR, I added it today morning (night)?

avatar waader
waader - comment - 8 Jan 2017

I have several patches for mssql installed now and also this patch and now the problem with content map has gone away.

Frontend is broken nonetheless.

avatar csthomas
csthomas - comment - 8 Jan 2017

On test database I have found two 404 links because mssql table does not have 2 articles for:

  • Popular Tags (Alias: popular-tags)
  • Similar Tags (Alias: similar-tags)
  • one in news-feed - ??
  • and search component has some error. ??

but home page work for me ok.

avatar csthomas csthomas - edited - 12 Jan 2017
avatar wilsonge
wilsonge - comment - 5 Feb 2017

@waader and @alikon is this patch in a mergable state?

avatar csthomas
csthomas - comment - 5 Feb 2017

This PR have to wait for #13585
I have to remove escaping chars from this PR which was done in #13585
After #13585 will be merged then it can go to tests and merge.

avatar csthomas
csthomas - comment - 5 Feb 2017

There is also PR #13895 which has part of code from this PR and could be merged probably by code review (unit tests pass).

avatar csthomas csthomas - change - 6 Feb 2017
The description was changed
avatar csthomas csthomas - edited - 6 Feb 2017
avatar csthomas csthomas - change - 6 Feb 2017
The description was changed
avatar csthomas csthomas - edited - 6 Feb 2017
avatar csthomas
csthomas - comment - 6 Feb 2017

This PR is ready to test:)

avatar csthomas csthomas - change - 10 Feb 2017
The description was changed
avatar csthomas csthomas - edited - 10 Feb 2017
avatar csthomas csthomas - edited - 11 Feb 2017
avatar waader
waader - comment - 14 Feb 2017

I tested with current staging with no samples data installed and clicked through the cms. I did not work out if it has anything to do with this patch:

Frontend - Home:
Serialization of 'Closure' is not allowed

Backend:
Multilingual Associations
Warning: ksort() expects parameter 2 to be long, string given in C:\Program Files (x86)\Ampps\www\joomla37\administrator\components\com_associations\models\fields\itemtype.php on line 57

Smart Search Indexer:
[Microsoft][SQL Server Native Client 11.0][SQL Server]Unclosed quotation mark after the character string 'O:19:"FinderIndexerResult":19:{s:11:"'.

Manage > Install Languages
[Microsoft][SQL Server Native Client 11.0][SQL Server]Unclosed quotation mark after the character string 'O:19:"FinderIndexerResult":19:{s:11:"'.

avatar wilsonge
wilsonge - comment - 14 Feb 2017

Frontend issue is a separate issue with a fix here: #14057

avatar csthomas
csthomas - comment - 14 Feb 2017

Warning: ksort() expects parameter 2 to be long
I assume this is php 5.3 issue, I have created an issue.

avatar waader
waader - comment - 14 Feb 2017

@wilsong #14057 resolves the problem - thanks
@csthomas You are right. My mssql-test system run´s on 5.3.29

avatar wilsonge
wilsonge - comment - 14 Feb 2017

@csthomas that redirect plugin error might be related to this pr :)

avatar csthomas
csthomas - comment - 15 Feb 2017

By the way: with the redirect plugin enabled I get 500 - PLG_SYSTEM_REDIRECT_ERROR_UPDATING_DATABASE

[Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Cannot insert the value NULL into column 'comment', table 'joomla.dbo.j37_redirect_links'; column does not allow nulls. INSERT fails.
This is unrelated.

I have found a big problem with sqlsrv updates in joomla (administrator/components/com_admin/sql/updates/sqlazure/*.sql).
A lots of that ALTER TABLE ALTER COLUMN has invalid syntax.

https://msdn.microsoft.com/en-us/library/ms190273.aspx

They are not executed to database at all. At least for my environment.
I need a little time to create an issue.

avatar csthomas csthomas - change - 21 Feb 2017
The description was changed
avatar csthomas csthomas - edited - 21 Feb 2017
avatar waader
waader - comment - 22 Feb 2017

@csthomas: Can you sychronize this patch with the changes made in other yet commited sqlserver-patches.

avatar csthomas
csthomas - comment - 22 Feb 2017

OK. Please wait a moment

avatar csthomas
csthomas - comment - 22 Feb 2017

I have rebased it. It should work but you can not use sample data, because it is broken, I'm working on that now (#13981).

But you can get sample data from #7680

avatar csthomas
csthomas - comment - 22 Feb 2017

Please ignore errors like:

Cannot insert the value NULL into column 'term_id', table 'joomla.dbo.#__finder_tokens_aggregate'; column does not allow nulls. INSERT fails.

It is unrelated, and it is waiting for new PR.

avatar csthomas
csthomas - comment - 23 Feb 2017

There is not need to combine this PR for tests with #14102 in the same time.

But if someone want to test 2 PRs at once then:

  • first apply #14102 and upgrade joomla database to latest version,
  • remove above patch from Patch Tester,
  • apply this PR (this PR has common, identical file database/driver/sqlsrv.php with PR #14102)
  • test this PR.
avatar waader waader - test_item - 24 Feb 2017 - Tested successfully
avatar waader
waader - comment - 24 Feb 2017

I have tested this item successfully on 00fd4a9

Fixes these problems:
/administrator/index.php?option=com_banners
/administrator/index.php?option=com_categories&extension=com_banners
/administrator/index.php?option=com_banners&view=clients
/administrator/index.php?option=com_finder&view=maps
/administrator/index.php?option=com_categories&extension=com_newsfeeds
/administrator/index.php?option=com_categories&extension=com_contact
/administrator/index.php?option=com_postinstall

Thanks!


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/13262.

avatar csthomas
csthomas - comment - 24 Feb 2017

Last commit fix error on "sample testing" for menu items:

Compact tagged: should show one record
Tagged items: should show one record

avatar csthomas
csthomas - comment - 24 Feb 2017

For All tags the is other error which has to be resolved by separate PR.

DB function failed with error number 169
[Microsoft][ODBC Driver 13 for SQL Server][SQL Server]A column has been specified more than once in the order by list. Columns in the order by list must be unique.
SQL =

SELECT TOP 20 a.*
FROM [#__tags] AS a
WHERE [a].[access] IN (1,5,1) AND [a].[parent_id] <> 0 AND [a].[title] LIKE N'%[gl]%' AND . [a].[published] = 1
ORDER BY [title] ASC, a.title ASC

[Microsoft][ODBC Driver 13 for SQL Server][SQL Server]A column has been specified more than once in the order by list. Columns in the order by list must be unique.

avatar wilsonge wilsonge - change - 25 Feb 2017
Status Pending Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2017-02-25 12:28:42
Closed_By wilsonge
avatar wilsonge wilsonge - close - 25 Feb 2017
avatar wilsonge wilsonge - merge - 25 Feb 2017
avatar wilsonge
wilsonge - comment - 25 Feb 2017

Merged with waaders test given this is only affecting mssql

avatar csthomas
csthomas - comment - 25 Feb 2017

Thanks

avatar photodude
photodude - comment - 25 Feb 2017

@csthomas Will you consider making these changes to the Framework also?

Add a Comment

Login with GitHub to post a comment