? ? Success

User tests: Successful: Unsuccessful:

avatar mbabker
mbabker
17 Aug 2014

Summary

An idea that was brought up during the Joomla! Developer Conference was adding support for PDO based connections for our MySQL database support. As this support already exists in the Framework (having refactored the legacy MySQL driver which was using the deprecated mysql PHP extension), this PR backports the PDO based MySQL driver and supporting classes into the CMS architecture and adds to our internal checks support for the new PDO driver. As this is a MySQL based driver, it required no database related changes to existing code based on initial testing.

Testing Instructions

First, you'll need to determine if your environment supports PDO based connections, and if so, if the PDO based MySQL driver is supported. You can use the System Information view in your site's backend to review the PHP configuration to check for this support. If your environment supports PDO, apply the patch via your preferred method. If you want to test an existing site, you should be able to switch to the MySQL (PDO) driver via the site's Global Configuration (or set $dbtype = 'pdomysql'; in configuration.php) BUT ENSURE YOU BACKUP THE SITE BEFORE TESTING. You can also test new installs selecting the MySQL (PDO) option on the database step. You should be able to use your existing database credentials in both instances as this option merely changes how Joomla is connecting to your MySQL database.

Please test both core features and your favorite extensions to ensure there aren't any unexpected errors with the new driver, and if there are any errors, please report them so we can determine if the error is related to the driver or the third party code. Developers who are already familiar with PDO are also invited to test PDO related functionality with this patch and report their results.

avatar mbabker mbabker - open - 17 Aug 2014
avatar jissues-bot jissues-bot - change - 17 Aug 2014
Status Pending New
Labels Added: ? ?
avatar brianteeman
brianteeman - comment - 17 Aug 2014

Tested installing the pdo version using this zip https://github.com/mbabker/joomla-cms/archive/pdo_mysql.zip

I was able to successfully install all the different sample data sets on mamp using the mysql pdo driver and an existing database

However while MAMP is able to create a brand new database with mysql and mysqli it would not create a new database with the pdo driver and resulted in the following error
Could not connect to the database. Connector returned number: Could not connect to PDO: SQLSTATE[42000] [1049] Unknown database 'pdo1'

avatar brianteeman
brianteeman - comment - 17 Aug 2014

@test
SQLSTATE[42000]: Syntax error or access violation: 1065 Query was empty

Attempting to enable debug in global configuration produces this error.
Attmepting to save a new article produces this error

avatar mbabker
mbabker - comment - 17 Aug 2014

@brianteeman I've been trying to work around the automatically creating the database issue, it doesn't look like it's possible the way our code is currently architected. PostgreSQL experiences the same issue too. I'm about to commit code to make the error message a bit more clear about that but for now that's the best I can do.

avatar mbabker
mbabker - comment - 17 Aug 2014

The empty query error is fixed and I've addressed another issue found with creating new items. Should be ready for more testing now.

avatar brianteeman
brianteeman - comment - 17 Aug 2014

@test I can confirm the empty query error is fixed and that the error message when pdo cant create a new database appears and is good

avatar brianteeman brianteeman - change - 20 Aug 2014
Labels Added: ?
avatar nicksavov nicksavov - change - 21 Aug 2014
Labels Removed: ?
avatar mbabker
mbabker - comment - 21 Aug 2014

So I got an idea out of the blue today about how the PDO MySQL driver could be forced into creating a database and I managed to hack something together. @brianteeman you should now be able to create your database if it doesn't exist when installing with the PDO driver.

avatar brianteeman
brianteeman - comment - 21 Aug 2014

Thanks Michael I'll check it out in the morning
On 21 Aug 2014 22:30, "Michael Babker" notifications@github.com wrote:

So I got an idea out of the blue today about how the PDO MySQL driver
could be forced into creating a database and I managed to hack something
together. @brianteeman https://github.com/brianteeman you should now be
able to create your database if it doesn't exist when installing with the
PDO driver.


Reply to this email directly or view it on GitHub
#4123 (comment).

avatar brianteeman
brianteeman - comment - 22 Aug 2014

@mbabker I can confirm that the change does allow the PDO MySQL driver to create a new database (clap)

avatar dgt41
dgt41 - comment - 23 Aug 2014

So far no probs, haven’t try the installation tho
screenshot 2014-08-23 17 38 16

avatar gwsdesk
gwsdesk - comment - 24 Aug 2014

@test
Took me a while to implement these 'minor' patches :-)
Zipped the stuff and uploaded to one of our own servers (online)

Environment:

  • Centos 6.5
  • Apache 2.4.10
  • MySQL 5.5.37
  • PHP 5.4.31
  • cgi/fcgi

Test results:
Upon installation I could choose the PDO MySQL driver and used this one. --> Installed like a charm!
I tested changing in super admin from PDO to MySQLi etc and vise versa --> works like a charm
It still works like a charm with Mod_rewrite/Memcache in progressive caching.

I did the same installation on my local box (Uniformserver) which runs Apache2Handler and PHP5.5.x and run the same tests --> Like a charm

@ Michael: A whole lot of claps!

Note: Site runs on gwsdev3.info

avatar gwsdesk
gwsdesk - comment - 24 Aug 2014

@test
Issues with Akeeba products. Extensions installed via normal installer with PDO Database selected
AdminTools: The layout is blown away and an error shows: "table does not exist" See image attached.... I go to global config, changed PDO database to MySQLi and Admin Tools shows normal
Weird experience: Change back after that again to PDO and AT works fine

Akeebabackup: Same story: Table missing " Table 'gwsdev3_pdo.nprlk_ak_profiles' doesn't exist SQL=SELECT configuration FROM nprlk_ak_profiles WHERE id = '1' ". I change to MySQLi and Akeeba works flawless . Same weird experience...Change back after that again to PDO and Akeebabackup works flawless

Ok installations: JCE + Image Manager XTD/Acymailing Enterprise/Widgetkit/Breezingforms/Mijoshop/SPupgrade/Nonumber/Perfect AJAX Popup Contact Form/NicePaypal/JSN Framework and Template/

Here is AT output in backend:
admintools

I assume Nicholas might have ideas why the 2 are throwing these errors with PDO and not with MySQLi? Both give in PDO tables missing but is not the case (!) and working under MySQLi

avatar Bakual
Bakual - comment - 24 Aug 2014

@nikosdion It may be because it is built with FoF which maybe also needs to add some support for PDO?

avatar nikosdion
nikosdion - comment - 24 Aug 2014

There is no PDO support in my software yet. It's one of those changes I had no idea was going to be included in Joomla! 3 until I got the notification by Thomas. I'll try to code something later today. If I'm right, supporting pdomysql only requires one line in one XML file per product and half a dozen if-blocks in Admin Tools. If you can hold on for a couple of hours I'll know for sure... :)

avatar Bakual
Bakual - comment - 24 Aug 2014

@nikosdion Michael wrote the PR 8 days ago. I didn't know before as well :smile:
If it works, then it's great of course and makes sense :+1:

avatar mbabker
mbabker - comment - 24 Aug 2014

Like I said, it was one of those ideas that came up on the road, and I finally found a late night to port the code :-D

avatar nikosdion
nikosdion - comment - 24 Aug 2014

Good news, there is no need to change anything in FOF. If a component is using FOFDatabaseInstaller you just need to change <meta>
<!-- Supported driver types -->
<drivers>
<driver>mysql</driver>
<driver>mysqli</driver>
</drivers>
</meta>

to <meta>
<!-- Supported driver types -->
<drivers>
<driver>mysql</driver>
<driver>mysqli</driver>
<driver>pdomysql</driver>
</drivers>
</meta>

in the XML schema file and the installation works.

Having done that, it seems that the pdomysql driver works just fine except with international characters.

How to replicate the issue

Before applying the patch create a new article with:
Title: Δοκιμή
Alias: δοκιμή
Content: Μια δοκιμή
(This is Greek for "Test", "test" and "A test" respectively)

Open the database with phpMyAdmin or similar. You can read the fields just fine.

After applying the patch do the same. The fields are now all messed up! The title fields reads "Δοκιμή" which is what happens when you try to store UTF-8 values in ASCII encoding. Even though Joomla! still works, trying to dump and restore the database of a site using the MySQL/MySQLi driver to a site using the pdomysql driver or vice versa (with phpMyAdmin, Sequel Pro, Akeeba Backup etc) will result in broken text.

Therefore I have to give a :-1: to this patch as it is right now. Sorry :(

avatar nikosdion
nikosdion - comment - 24 Aug 2014

Apparently you need to pass $options['driverOptions'] = array(PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES utf8') to the parent JDatabaseDriverPdo in line 71 of JDatabaseDriverPdomysql, i.e. change line 71 to:

$this->charset = $options['charset'];
if (strtolower($this->charset) == 'utf8')
{
   $options['driverOptions'] = array(PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES utf8');
}

I have no idea how I can possibly make a PR to a pending PR, so I'll leave the code fix here for @mbabker to integrate.

PS: I have no idea why GitHub is squashing my line breaks in the code snippet above.

avatar mbabker
mbabker - comment - 24 Aug 2014

I actually just went about it using the charset declaration in the connection DSN in mbabker@baa7211 (since that wasn't being set either).

If that doesn't work, we can try your suggestion.

avatar nikosdion
nikosdion - comment - 24 Aug 2014

That totally worked and I can happily report that you've got a successful test from me!

avatar mbabker
mbabker - comment - 24 Aug 2014

Great!

avatar Bakual
Bakual - comment - 24 Aug 2014

Looks RTC then to me.
Only thing left is to fix Travis, which Michael loaded on his shoulders by enabling the codestyle checks :smiley:

avatar Bakual
Bakual - comment - 24 Aug 2014

@mbabker It should be fixed if you rebase your branch.

avatar mbabker
mbabker - comment - 24 Aug 2014

Merged it, let's see what happens.

avatar Bakual Bakual - change - 24 Aug 2014
Labels Added: ?
avatar Bakual
Bakual - comment - 24 Aug 2014

Travis fine. Setting RTC.

avatar gwsdesk
gwsdesk - comment - 25 Aug 2014

I am testing this once again through com_patchtester and I get with applying the patch on com_patchtester:

Error
The file marked for modification does not exist: phpunit.xml.dist

The file clearly exists in this repro here so is this an error on com_patchtester? If so it is not that fun since that means patching again manually 37 files? Suggestions?

avatar gwsdesk
gwsdesk - comment - 25 Aug 2014

Note that @brianteeman made a language change Issue #4163 which breaks the installation in com_patchtester. I assume we need to upgrade language strings here as well to avoid this from happening?

avatar Bakual
Bakual - comment - 25 Aug 2014

I assume we need to upgrade language strings here as well to avoid this from happening?

Git will most likely be able to handle that itself. That's the good thing with Git :smile:

As for the error you see: The patch applied fine for me using Patchtester. So it's maybe a hickup in your system somehow. Can you try with a fresh installation if it works?

avatar gwsdesk
gwsdesk - comment - 25 Aug 2014

No not working. Throws same error. I uninstalled, checked database all
gone and reinstalled --> same error message on this.
On 8/25/2014 1:28 PM, Thomas Hunziker wrote:

I assume we need to upgrade language strings here as well to avoid
this from happening?

Git will most likely be able to handle that itself. That's the good
thing with Git :smile:

As for the error you see: The patch applied fine for me using
Patchtester. So it's maybe a hickup in your system somehow. Can you
try with a fresh installation if it works?


Reply to this email directly or view it on GitHub
#4123 (comment).

Leo Lammerink
MD GWS - Enterprise Ltd
Skype: gwsgroup
www.gws-desk.com http://www.gws-desk.com | www.gws-host.com
http://www.gws-host.com | www.gws-deals.today http://gws-deals.today

avatar Bakual
Bakual - comment - 25 Aug 2014

Strange. Maybe @mbabker knows more. Doesn't sound like an issue with this PR but more like some bug with the Patchtester. I'm not sure.

avatar gwsdesk
gwsdesk - comment - 25 Aug 2014

Again: Full fresh install on a Fresh Joomla 3.3.3 (!) See my experience
http://screencast.com/t/hWLj4DB5
On 8/25/2014 1:36 PM, Thomas Hunziker wrote:

Strange. Maybe @mbabker https://github.com/mbabker knows more.
Doesn't sound like an issue with this PR but more like some bug with
the Patchtester. I'm not sure.


Reply to this email directly or view it on GitHub
#4123 (comment).

Leo Lammerink
MD GWS - Enterprise Ltd
Skype: gwsgroup
www.gws-desk.com http://www.gws-desk.com | www.gws-host.com
http://www.gws-host.com | www.gws-deals.today http://gws-deals.today

avatar mbabker
mbabker - comment - 25 Aug 2014

Log a bug report at https://github.com/joomla-extensions/patchtester and we'll go from there.

avatar Bakual
Bakual - comment - 31 Aug 2014

Merged into 3.4-dev branch. Thanks!

avatar Bakual Bakual - change - 31 Aug 2014
Status New Closed
Closed_Date 0000-00-00 00:00:00 2014-08-31 18:33:14
avatar Bakual Bakual - close - 31 Aug 2014

Add a Comment

Login with GitHub to post a comment