? Success

User tests: Successful: Unsuccessful:

avatar williamsandy
williamsandy
8 Jul 2016

Pull Request for Issue # .

Summary of Changes

Testing Instructions

Default values are not being assigned correctly for data types such as
Integer, Date and Datetime. So as a result it is causing the insert
query to fail. I have tested this on SQL SERVER 2008-2016.

The root cause of this issue lies with the getTableColumns in the SQL
server driver class, which relies on using the information_schema views
to get the information about the columns. The default values it
gets for the table columns are from the underlying system comments table
and as a result it is not in a format that can be injected as is into a
INSERT Query.

Below is a partial list of how the syscomments table stores default
values for the following data types:

Datatype : int
Actual default Value : 0
Value in syscomments table: ((0))

Datatype : int
Actual default Value : 10
Value in syscomments table: ((10))

Datatype : Datetime
Actual default Value : ‘1900-01-01T00:00:00.000’
Value in syscomments table: (‘1900-01-01T00:00:00.000’)

Datatype : Datetime
Actual default Value : getdate()
Value in syscomments table: (getdate())

Datatype : Date
Actual default Value : ‘1900-01-01’
Value in syscomments table: (‘1900-01-01’)

Datatype : Varchar
Actual default Value : ''
Value in syscomments table: ('')

Datatype : Varchar
Actual default Value : '1234'
Value in syscomments table: ('1234')

Datatype : NVarchar
Actual default Value : N''
Value in syscomments table: (N'')

Datatype : NVarchar
Actual default Value : N'1234'
Value in syscomments table: (N'1234')

Datatype : Varchar
Actual default Value : NULL
Value in syscomments table: (NULL)

You see the general pattern that the default value stored in the
syscomments table is wrapped around in curly brackets, whether that be
single or double depends on the data type.

Now the current implementation of the getTableColumns function only
deals with nvarchar data types by assuming every default string value
will be an empty string so defaults to that value ignoring the actual database value. This works but IMHO it is a bit too simplistic in that it does not allow for default string
values that is not an empty string value. Also it does not deal with
other data types such as datetime, date etc... So as a result the INSERT
query will still blow up when you try for example to pass the value ‘((0))’
for an integer data type.

My change just parses the value by removing the superfluous brackets and
returns the actual value for the data type in the correct format.

So for example the value ‘((0))’ will be returned as 0, (N‘1’) will
be returned as ‘1’ and value (getdate()) will be returned as getdate() and so on ....

avatar williamsandy williamsandy - open - 8 Jul 2016
avatar brianteeman brianteeman - change - 8 Jul 2016
Category MS SQL
avatar brianteeman
brianteeman - comment - 2 Aug 2016

Hi

Does #11396 resolve this issue?


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/11056.

avatar williamsandy
williamsandy - comment - 2 Aug 2016

Unfortunately no, this is because the fault lies with the getTableColums method of the JDatabaseDriverSqlsrv that tries to get the column details for a given database table.

The issue here is it uses this query below to get the default values for the table rows say for example '#__user_notes'--which to be honest you don't really have another way of getting that information.

SELECT column_name as Field, data_type as Type, is_nullable as 'Null', column_default as 'Default'
FROM information_schema.columns WHERE table_name = 'nqu7n_user_notes'

[Note 'nqu7n' is the prefix used for my database please change this with the one you are using]

If you run this query in any version of MS SQL Server and look at what it is given you for the default values, you will see straight away that this is going to cause issues. Please see my description above which goes in more depth.

The SQL Issue arises (Cannot convert value as Int or Cannot convert blah as datetime2 etc..) when you are using JTable to create a new row in a database table, and you don't explicitly supply the default values for every column that has a default value.

If you apply my patch, it will filter the default values and return them in a correct format.

avatar brianteeman
brianteeman - comment - 2 Aug 2016

Sorry I don't have mssql to test and to be honest with you it looks like
there are very few <100 users of joomla on mssql

avatar williamsandy
williamsandy - comment - 2 Aug 2016

Thank you for your reply. I do understand that this is difficult to test if you don’t have MSSQL but the Joomla team clearly have put a lot of effort in making Joomla work on non-MySQL servers, and it would be a shame to see that work go to waste with such a simple fix.

On our side, we have thoroughly tested it as we have been working on a live project
that uses SQL Server so it is important to us that this fix can be applied so we can continue to upgrade Joomla for that project for security patches. The reason I am pushing this change is due to this issue #11280
(#11280 (comment)) I have committed to fixing this issue involving MSSQL. However, I cannot apply the fix for that one because it is being held up by this PR.

I also believe that there is a duty of care issue as Joomla states it is MSSQL compliant. We would be more than happy to fix these issues; we just need someone to test our patches so we will be able to fix issues
like this and the other one mentioned in this PR

avatar brianteeman
brianteeman - comment - 2 Aug 2016

Unfortunately the people who did the work to support mssql disappeared as
soon as they committed the code.

Right now my understanding is that the PLT are re-evaluating if Joomla
should continue to support mssql as we don't have the resources to test and
maintain it and the number of users is so small.

Maybe @wilsonge or @rdeutz caan comment

avatar rdeutz
rdeutz - comment - 2 Aug 2016

We would be more than happy to support more database types. As far as I know then besides PostgreSQL all others are broken. I don't think it is realistic and we don't have the people and knowledge to support more.

I would merge it when it has 2 successful tests as we usually do

avatar williamsandy
williamsandy - comment - 2 Aug 2016

@sovainfo and @roland-d would you be so kind to test this PR. Thanks.

avatar roland-d
roland-d - comment - 2 Aug 2016

I agree with @rdeutz here. I don't have any MSSQL server available, hopefully others have and can check. Although community support for MSSQL is nearly non-existent as we found out.

avatar sovainfo
sovainfo - comment - 2 Aug 2016

Tried to submit a positive test with issues.joomla.org, but that failed.

Used this code to test:

<?php
$datatypes = array("((0))", "((10))", "('1900-01-01T00:00:00.000')", "(getdate())", "('1900-01-01')", "('')", "('1234')", "(N'1234')", "(NULL)");
foreach ($datatypes as $datatype) {
    echo $datatype;echo " : ";
    echo preg_replace("/(^(\(\(|\('|\(N'|\()|(('\)|(?<!\()\)\)|\))$))/i", '', $datatype);
    echo "\n";
}
?>

Output:
((0)) : 0
((10)) : 10
('1900-01-01T00:00:00.000') : 1900-01-01T00:00:00.000
(getdate()) : getdate()
('1900-01-01') : 1900-01-01
('') :
('1234') : 1234
(N'1234') : 1234
(NULL) : NULL

avatar sovainfo sovainfo - test_item - 2 Aug 2016 - Tested successfully
avatar sovainfo
sovainfo - comment - 2 Aug 2016

I have tested this item successfully on 0fba84c


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/11056.

avatar williamsandy
williamsandy - comment - 3 Aug 2016

@waader and @lmcculley Hi gents I am trying to get a fix in for those of us that have MSSQL and I am wondering if you could test this PR so that we can get this fix in

avatar marksmeed
marksmeed - comment - 5 Aug 2016

@Bakual would you be so kind to test the fix?

avatar waader waader - test_item - 6 Aug 2016 - Tested successfully
avatar waader
waader - comment - 6 Aug 2016

I have tested this item successfully on 0fba84c

Thanks @williamsandy! Just ping me if you have other mssql patches to test. But it may take some days as I am busy right now.


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/11056.

avatar rdeutz rdeutz - change - 6 Aug 2016
Milestone Added:
Status New Ready to Commit
avatar rdeutz
rdeutz - comment - 6 Aug 2016

Thanks for testing this, RTC


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/11056.

avatar zero-24
zero-24 - comment - 8 Aug 2016

@joomla-cms-bot please ?

avatar brianteeman
brianteeman - comment - 8 Aug 2016

@zero-24 I am changing ! my name

avatar zero-24
zero-24 - comment - 8 Aug 2016

hehe ? Thanks.

Add a Comment

Login with GitHub to post a comment