? ? Failure

User tests: Successful: Unsuccessful:

avatar alikon
alikon
7 Mar 2019

Pull Request for Issue #24095 .

Summary of Changes

use lower() to check user email

Testing Instructions

see #24095

Expected result

test@test.fr and Test@test.fr must be detected as the same email address even on db's like Postgresql

Actual result

detected as the different email address even on db's like Postgresql

Note

mysql < 5.7 don't support function based index ... iirc

Votes

# of Users Experiencing Issue
1/1
Average Importance Score
5.00

avatar alikon alikon - open - 7 Mar 2019
avatar alikon alikon - change - 7 Mar 2019
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 7 Mar 2019
Category Libraries
avatar mdaoudi123
mdaoudi123 - comment - 8 Mar 2019

Thanks a lot for your help.

But There is also another file where exists a email comparaison:

  • components/com_users/models/reset.php => function processResetConfirm
  • components/com_users/models/remind.php => function processRemindRequest
  • ...
    and into other files..... (search for "$db->quoteName('email')")

Could you please add the correction to thoses files @alikon

avatar alikon alikon - change - 8 Mar 2019
Labels Added: ?
avatar joomla-cms-bot joomla-cms-bot - change - 8 Mar 2019
Category Libraries Front End com_users Libraries
avatar joomla-cms-bot joomla-cms-bot - change - 9 Mar 2019
Category Libraries Front End com_users Administration Front End com_users Libraries
avatar joomla-cms-bot joomla-cms-bot - change - 9 Mar 2019
Category Libraries Front End com_users Administration Administration Front End com_users Installation Libraries
avatar joomla-cms-bot joomla-cms-bot - change - 9 Mar 2019
Category Libraries Front End com_users Administration Installation Administration Front End com_users Installation Libraries Plugins
e28daa1 9 Mar 2019 avatar alikon cs
avatar alikon alikon - change - 9 Mar 2019
Title
[com_user] - case insesitive check for email address
case insensitive management for email address
avatar alikon alikon - change - 9 Mar 2019
Title
[com_user] - case insesitive check for email address
case insensitive management for email address
avatar alikon alikon - edited - 9 Mar 2019
avatar joomla-cms-bot joomla-cms-bot - change - 10 Mar 2019
Category Libraries Front End com_users Administration Installation Plugins Administration Front End com_users Libraries Plugins
avatar mdaoudi123
mdaoudi123 - comment - 22 Mar 2019

Can i download the new patch some where?
@alikon
Thanks

avatar alikon
alikon - comment - 22 Mar 2019
avatar mdaoudi123
mdaoudi123 - comment - 25 Mar 2019

Hello,
Thanks for the patchTester,
I tested the patch and it's OK.
Could you merge this branch and build a new patch?

avatar joomla-cms-bot joomla-cms-bot - change - 25 Mar 2019
Category Libraries Front End com_users Administration Plugins Administration Front End com_users Postgresql SQL Installation Libraries Plugins
avatar joomla-cms-bot joomla-cms-bot - change - 25 Mar 2019
Category Libraries Front End com_users Administration Plugins Postgresql SQL Installation Postgresql SQL Administration com_admin Front End com_users Installation Libraries Plugins
d1dbb9f 25 Mar 2019 avatar alikon cs
avatar alikon alikon - change - 25 Mar 2019
The description was changed
avatar alikon alikon - edited - 25 Mar 2019
avatar alikon
alikon - comment - 25 Mar 2019

@mdaoudi123 please mark as tested on https://issues.joomla.org/tracker/joomla-cms/24117
this should be carefully tested on mysql side too....and mssql also i'm afraid

avatar mdaoudi123
mdaoudi123 - comment - 26 Mar 2019

Hello,
im going to test it into POSTGRES too,
can you tell me how to mark this post as tested?
Thanks

avatar mdaoudi123
mdaoudi123 - comment - 26 Mar 2019

I have tested this item successfully on a7c980e

Tested Successfully on Mysql and Postgres databases.

Thanks


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/24117.

@alikon

avatar mdaoudi123 mdaoudi123 - test_item - 26 Mar 2019 - Tested successfully
avatar mdaoudi123
mdaoudi123 - comment - 1 Apr 2019

Hello,
I have already tested this feature.
Could you please creat a new patch @alikon


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/24117.

avatar mdaoudi123
mdaoudi123 - comment - 1 Apr 2019

Hello,
I have already tested this feature.
Could you please creat a new patch @alikon


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/24117.

avatar alikon
alikon - comment - 1 Apr 2019

one test is not enough, we need at least two, before this pr can be considered by the maintaners

avatar mdaoudi123
mdaoudi123 - comment - 1 Apr 2019

There is an error into the continuous-integration/drone/p

The problem is the phpcs confirmation

@alikon

avatar mdaoudi123
mdaoudi123 - comment - 30 Apr 2019

There is an error into the continuous-integration/drone/p

The problem is the phpcs confirmation

Could do you check whats happening please

@alikon

avatar alikon alikon - change - 30 Apr 2019
Labels Removed: J3 Issue
avatar alikon
alikon - comment - 1 May 2019

sorry but really don't know how to fix CS for an inline function

avatar HLeithner
HLeithner - comment - 1 May 2019

I just wanted to say that the local part of an email is case sensitive, it's ok for me to ignore this because it's common practice to handle email addresses case insensitive.

avatar mdaoudi123
mdaoudi123 - comment - 19 Jun 2019

@HLeithner @alikon
please there is news for this fix?
When this patch will be available?

Please Help

avatar mdaoudi123
mdaoudi123 - comment - 19 Jun 2019

please could you fix the phpcsfixer warning

avatar mdaoudi123
mdaoudi123 - comment - 19 Jun 2019

Any one can help me please

avatar franz-wohlkoenig
franz-wohlkoenig - comment - 19 Jun 2019

one test is not enough, we need at least two, before this pr can be considered by the maintaners

@mdaoudi123

avatar mdaoudi123
mdaoudi123 - comment - 19 Jun 2019

I tedted the fix. but who can test it another time to deblock the situation
Anyone can test please

avatar mdaoudi123
mdaoudi123 - comment - 19 Jun 2019

Im waiting since 3 months

avatar franz-wohlkoenig
franz-wohlkoenig - comment - 20 Jul 2019

@richard67 you use Postgresql? Can you please look at this PR?

avatar richard67
richard67 - comment - 20 Jul 2019

@franz-wohlkoenig Today I have a full schedule. I will try to find a time slot, but it might be tomorrow until I can do something. I do not really use postgresql, I just have it for testing.

avatar richard67
richard67 - comment - 21 Jul 2019

@alikon @franz-wohlkoenig This PR will not work as it is. In the schema update, the index is dropped and then added back with same name. This is valid SQL, but the database schema checker can't handle that due to its bad design, so it will end in a toggle of reported database error "index missing but should exist" and "index existing but it should not". The only thing you can do is to add it back with other name, but the for consistency it might be necessary to change index name for MySQL, too.

avatar alikon
alikon - comment - 21 Jul 2019

thanks @richard67
btw,i don't think we need to do something on mysql side

avatar richard67
richard67 - comment - 21 Jul 2019

@alikon Did you test if update notification emails are still working with your change (which I don't fully understand and so I have same question as JM had).

avatar richard67
richard67 - comment - 21 Jul 2019

@alikon I've made a PR adding the missing function comment which has to be completed by you, and which fixes a few of the code style issues. But I still do not understand why it needs that call back function stuff. Why don't you just make a local copy of the emails array, lowercase it, and use that for the query? It would not need this inline function and not need this new function and so no need for new function comments and all codestyle could be fine. Can you explain me why it needs to be done in that way with the callback?

avatar alikon
alikon - comment - 21 Jul 2019

ouch i've to admit that i don't remember why i've done that, in that way maybe in that period i worked at the office "complicate simple thing" 😄 .... hope i'll find free time to rethink soon

avatar richard67
richard67 - comment - 21 Jul 2019

@alikon As long as you rethink that, ignore my PR against your branch of this PR. I think about it, too, and maybe come with new PR for your branch.

avatar richard67
richard67 - comment - 21 Jul 2019

@alikon I see another problem with the new unique index on postgresql (and later in 4.0-dev where we have MySQL >= 5.7 on MySQL, too, if we add the same index then):

When users do an update of their Joomla and have duplicate lowercase emails in their users table database, like "gaga@blabla.de" and "GaGa@blabla.de", the unique index adding will fail.

It would require to clean existing data by duplicate records. We once had the same problem when we wanted to change collations of com_finder tables in MySQL from general to unicode, because that change would have caused duplicate values, and so we gave that up and remained com_finder tables with general collations.

If you want to keep that new index for performance reasons, then make it non-unique, i.e. just an index.

Or you don't add that new index.

Of course this would not be a problem if we could be sure that the PHP code of the CMS never allowed to add users having same lowercase email address. But I am pretty sure even if that was the case, we could not 100% rely on it over all the history of Joomla version in the update history of a web site.

So I suggest to make the new index non-unique or leave it away.

avatar richard67
richard67 - comment - 21 Jul 2019

@alikon P.S. to my previous post: I think @mdaoudi123 did not have that problem during his test because either he did not have duplicate records, or he did not run the schema update sql (which patchtester will also not do) and so he tested only the changes in PHP but not the database change.

avatar richard67
richard67 - comment - 21 Jul 2019

@alikon My PR alikon#55 replaces the callback stuff by easier solution and so fixes all cs issues. What remains to be thought about is not to add the new index as unique because it may break update of sites with existing duplicate lowercase email addresses in their data.

avatar richard67
richard67 - comment - 21 Jul 2019

@alikon My PR alikon#56 handles the index problems and renames the schema update according to latest releases.

So if you accept my 2 PRs and update yours here with latest staging we can test yours.

avatar alikon
alikon - comment - 21 Jul 2019

thanks @richard67

avatar richard67
richard67 - comment - 21 Jul 2019

Drone failure is not related to this PR, is javascript stuff. PHPCS is ok now.

avatar richard67
richard67 - comment - 21 Jul 2019

@mdaoudi123 Sorry for the inconvenience and the long waiting time. Could you test this PR again with its latest changes?

avatar richard67
richard67 - comment - 21 Jul 2019

@alikon Could you synch this PR with staging by using that button in GitHub at the bottom of your PR?

avatar brianteeman
brianteeman - comment - 21 Jul 2019

Unless there are conflicts there is no benefit

avatar richard67
richard67 - comment - 21 Jul 2019

Yes but looks less scary for beginner testers.

avatar richard67
richard67 - comment - 21 Jul 2019

I have tested this item successfully on a8c025b

Tested on MySQL and PostgreSQL.

On MySQL it does not break anything.

On PostgreSQL it checks now for present email address in case-insensitive way without touching existing user data.

Tested as follows:

Pre-conditions: In Global configuration, set error reporting to maximum or development and watch the PHP error logs during the tests. Check that there are no errors reported.

  1. Without this PR applied, have super admin email address be like e.g. "test@test.local".
  2. Create another user with email address be like e.g. "Test@test.local", i.e. same as 1st one but different case.
    Result: User is created.
  3. Apply this PR.
  4. Go to "Extensions -> Manage -> Database" and fix the database error to apply the schema update sql of this PR.
    Result: Database upt to date.
  5. Create another user with email address be like e.g. "tEst@test.local", i.e. same as 1st and 2nd one but different case.
    Result: User is not created because email address already exists.
  6. Edit the 2nd user but do not change email address and try to save.
    Result: User can't be saved because email address already exists.
  7. Exit user edit without saving.
  8. Work a bit in backend with user-related functions like privacy requests or other stuff related to files changed by this PR.
    Result: You can't add new users or change users so more than one user has same email address, case-independent, but the existing 2 users with same case-independent email addresses do not make problems.

@alikon Could you update your testing instructions to what I wrote here above so other users can test more easily?


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/24117.

avatar richard67 richard67 - test_item - 21 Jul 2019 - Tested successfully
avatar alikon alikon - change - 3 Aug 2019
Status Pending Closed
Closed_Date 0000-00-00 00:00:00 2019-08-03 10:11:51
Closed_By alikon
avatar alikon alikon - close - 3 Aug 2019
avatar richard67
richard67 - comment - 3 Aug 2019

@alikon why closing this?

avatar richard67
richard67 - comment - 3 Aug 2019

@alikon Did you maybe accidently hit the wrong button, close instead of rebase?

avatar alikon alikon - change - 4 Aug 2019
Status Closed New
Closed_Date 2019-08-03 10:11:51
Closed_By alikon
avatar alikon alikon - change - 4 Aug 2019
Status New Pending
avatar alikon alikon - reopen - 4 Aug 2019
avatar richard67
richard67 - comment - 4 Aug 2019

I have tested this item successfully on 27d3519


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/24117.

avatar richard67 richard67 - test_item - 4 Aug 2019 - Tested successfully
avatar richard67
richard67 - comment - 4 Aug 2019

@mdaoudi123 Sorry for the inconvenience.

Could you test this PR again with its latest changes?

And then mark your test result at the issue tracker here https://issues.joomla.org/tracker/joomla-cms/24117, using the "Test this" button at the top left corner? Then just select the appropriate result and submitt.

avatar jwaisner jwaisner - test_item - 7 Jan 2020 - Tested successfully
avatar jwaisner
jwaisner - comment - 7 Jan 2020

I have tested this item successfully on 27d3519


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/24117.

avatar jwaisner
jwaisner - comment - 7 Jan 2020

I have tested this item successfully on 27d3519


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/24117.

avatar SharkyKZ SharkyKZ - change - 7 Jan 2020
Status Pending Ready to Commit
avatar SharkyKZ
SharkyKZ - comment - 7 Jan 2020

RTC.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/24117.

avatar mdaoudi123
mdaoudi123 - comment - 7 Jan 2020

Thank's a lot.

avatar mdaoudi123
mdaoudi123 - comment - 7 Jan 2020

@alikon This correction will be available into the new joomla release?.

regards

avatar alikon
alikon - comment - 7 Jan 2020

it's up to the maintainers/release lead to decide if and when

avatar HLeithner
HLeithner - comment - 8 Jan 2020

Can you please rename the sql file to 3.9.15 then I will merge this.

avatar alikon alikon - change - 8 Jan 2020
Labels Added: ?
avatar alikon
alikon - comment - 8 Jan 2020

renamed

avatar HLeithner
HLeithner - comment - 8 Jan 2020

I'm not the biggest fan this PR but think it solves more problems then it creates.

Thanks

avatar HLeithner HLeithner - change - 8 Jan 2020
Status Ready to Commit Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2020-01-08 13:13:58
Closed_By HLeithner
avatar HLeithner HLeithner - close - 8 Jan 2020
avatar HLeithner HLeithner - merge - 8 Jan 2020

Add a Comment

Login with GitHub to post a comment