Implementation looks very good

just some comments about UI and placement of code (please see pictures)

• there is already a joomla user plugin that can host the event method onUserBeforeSave this is: plugins/user/joomla/joomla.php

thus adding a new plugin just to add this 1 method can be avoided

• Furthermore there are several parameters for password limitations in the parameters of USERs component (com_users) . So adding the username limitations parameters inside the plugin, is somewhat an inconsistency for the UI

Plus it is always more difficult for users to find parameters in so many plugins ... that a web-site has ... instead the user would expect to find the parameters in the "Users manager" where the password parameters already exist.

Please see the pictures that are the result of modifying the config.xml of the users component

For spacers with "label" class (example without language string ...)

<field type="spacer" name="pass_lims" label="Password limitations" class="label" />
...
<field type="spacer" name="uname_lims" label="Username limitations" class="label" />

OR for tabs

</fieldset><fieldset name="pass_lims" label="Password" >
...
</fieldset><fieldset name="user_lims" label="Username" >

I think this might be a good opportunity to incorporate Unicode abuse checking too. For example, see

http://www.m3aawg.org/sites/default/files/m3aawg-unicode-tutorial-2016-02.pdf
http://www.m3aawg.org/sites/default/files/m3aawg-unicode-best-practices-2016-02.pdf

OK @ggppdk, all you said looks good to me, and have all the sense. If no one have more objections, I'll place the code just inside the USERs component to achieve the UI you proposes in the picture.
I'm going to read the "Unicode abuse" too, trying to find out what's going on there.
I think, I'm going to publish the plugin 'usernamecheck' as a third party extension, so people (@baijianpeng) can apply usernamecheck plugin right now in 3.5 versions. Is this right way?

Thank you @framontb , I prefer to see this feature as a Joomla core
com_users feature. In this way, it will help more users like me.

2016-04-16 20:35 GMT+08:00 framontb notifications@github.com:

OK @ggppdk https://github.com/ggppdk, all you said looks good to me,
and have all the sense. If no one have more objections, I'll place the code
just inside the USERs component to achieve the UI you proposes in the
picture.
I'm going to read the "Unicode abuse" too, trying to find out what's going
on there.
I think, I'm going to publish the plugin 'usernamecheck' as a third party
extension, so people (@baijianpeng https://github.com/baijianpeng) can

apply usernamecheck plugin right now in 3.5 versions. Is this right way?

This comment was created with the J!Tracker Application
https://github.com/joomla/jissues at issues.joomla.org/joomla-cms/9943
https://issues.joomla.org/tracker/joomla-cms/9943.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#9943 (comment)

Anyway: https://github.com/framontb/plg_user_usernamecheck

For me to test this plugin was necessary to discover and install the usernamecheck plugin prior to test. I suppose, this is not the right way, because core plugins should be installed itself. I don't know how to do this.

• You need to add then to the extension table for the 3 database types supported: mysql, postgresql, sqlazure
  Example for mysql (plg_system_stats): https://github.com/joomla/joomla-cms/blob/staging/installation/sql/mysql/joomla.sql#L613

• Also you need to add on update sql (for the 3 database types too)
  Example for mysql (plg_system_stats): https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_admin/sql/updates/mysql/3.5.0-2015-11-05.sql#L1-L2

• And also need to add it to the update manifest list on update
  Example for plg_system_stats: https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_admin/script.php#L343

• Also i think you need to add the language files to the install.xml file
  Example for plg_system_stats: https://github.com/joomla/joomla-cms/blob/staging/administrator/language/en-GB/install.xml#L204-L205

Don't know if anything else is needed

@framontb

Actually to move the code inside users component is rather little work. The place to add the code is already there, the username validation code is supposed to be inside file:

joomla/form/rule/username.php

1. Place usernane parameters (as described above) inside XML file:
   administrator/components/com_users/config.xml

2. inside the test() function of joomla/form/rule/username.php add:
   $params = JComponentHelper::getParams('com_users');
   plus your code

3. return false / true, if the username is acceptable
   currently the existing code only check for duplicate username

About limitations of password, you will see that they are being checked inside
joomla/cms/rule/username.php, also calling: $params = JComponentHelper::getParams('com_users');

But 2 notes the registration form:
Currently the config.xml is using type="text" for type of the username

• it would be nice if someone added an AJAX validation type, i say AJAX validation because e.g. the duplicate username check can not be handled from the form:

name="username" type="text" validate="ajax" validate_url="..." 
// or
name="username" type="ajax_text" validate_url="..." 

• Registration form needs some inline message or something about the limitations of the username maybe adding a new form element "username" ?? exactly like this exists: libraries\joomla\form\fields\password.php there should be a: libraries\joomla\form\fields\username.php

I guess the relevant strings (such as JLIB_DATABASE_ERROR_VALID_AZ09) should also be modified...

Don't merge !
I'm still placing code.

@ggppdk

I think what you suggested in your post is done (not the AJAX validation). You were right, it was straightforward putting the code there (joomla/form/rule/username.php).
I have two more questions:

• Should I add the password tab here in this issue?
• How could I set a value for the fields in username? (I try with "value", but doesn't work.) Thanks in advance for you invaluable help.

If "Username" (limitations) gets a different TAB
then it makes sense to have seperate TAB for "Password" too
or maybe both should be in the 1 new TAB calling it "Validation",

• anyway this NOT my call, someone else should say about this, everything i wrote here are just opinions, you can just do it and then see if maintainers agree with it, i am just a user here

Now about your question of how to "set a value",

• i think you mean how to use "default" values, you need to do 2 things

1. almost all "form elements" (parameters) used in XML file respect / use the attribute "default"
   e.g. <field name="maximum_length_username" ... default="99" ...

2. then inside your code use the same default value:
   $maxNumChars = $params->get('maximum_length_username', 99);

Some notes / concerns:

• Default values of new features should be wide enough

not to change / break existing behaviour in existing installations, e.g. you added a default value of max characters 99 for username, this is good should not cause problems for 99,9% of cases
But you need to remove default value for allowed characters

• existing sites that do not have latin-only limitation for username, will start having this limitation, even if you do not add to PHP code, adding default value to XML file will cause this as soon as user resave com_users configuration
• plus it will cause problems with entering emails as username which is common case, but there is a way out of adding this see last note

so remove default value from XML file and inside your code check if parameter is empty because now you do not check if it is empty, if it is empty do not try to enforce any character limitation

• Terminology,

  you have a parameter "charset" which has a name that is not appopriate: charset which usually implies the name of a character set aka values 'utf-8' , 'utf-16' etc, so better call parameter: "allowed_chars" and because it concerns the username, the parameter name should be:
  allowed_chars_username

• Easy configuration of common cases for allowed characters without typing / avoiding problematic default values

since you can not have default value as explained above, but we don't want users to type all allowed characters and even make mistakes,

• the user should be to offered common cases via parameter, "allowed characters" parameter, for start only add these cases:

<field name="allowed_chars_username_preset" type="list"  defaut="0" ... >
  <option value="0">No limit</option>
  <option value="1">- Custom - </option>
  <option value="2">Latin Only</option>
  <option value="3">Email valid characters</option>
</field>

• for case 1 "- Custom -" you will use the characters from the textarea
• for latin only set allowed characters directly in your code
• to validate email case please use this code (someone please correct me)

[EDIT] i have corrected the email check code according to smz 's comment

$allowed_preset = $params->get('allowed_chars_username_preset', 99);

// Check for valid email
if ( $allowed_preset==3 )
{
   jimport('joomla.mail.helper');
   if ( ! JMailHelper::isEmailAddress($value) )
   {
      // Enqueue error message and return false
   }
}

Changed the title of the PR to reflect that it's no longer a plugin.

@ggppdk

to validate email case please use this code (someone please correct me)

We already have JMailHelper::isEmailAddress() (amongst others...) to validate email addresses. Please do not reinvent the wheel!

Why the status is failure in Issue Tracker while the status is "All checks have passed" in github?

Very useful feature framontb! I was looking for this feature and found your PR right on time :)
I think it is correct to insert this as a core functionality. Username with spaces and multiple spaces are correctly considered different usernames, but this leads to problems most of the times in the real life and I think an admin should have the possibility to limit this behaviour as a core option, since usernames registration is a sensible and primordial part in a CMS.

I have tested your PR with Standard and Custom character set and it seems good to me.

Insight request:
Added to "username check" two new features:
First, give chance to administrator for supplying his own regexp, for restricting username field. So the admin could do whatever he wants with username field, in an absolutely custom way. Too far?
Second, implement "M3AAWG highly restrictive definitions for East Asian languages" from docs supplied by @chrisdavenport. I write somo regexps, but I think it will be worth if someone check the job. The M3AAWG doc is intended to support this combinations (if I understood well):
● Latin + Han + Hiragana +Katakana;
● Latin + Han + Bopomofo;
● Latin + Han + Hangul;
Please if someone could have a look...

Should this be tested starting from the "staging" or the "3.6.x" branch? I see things which I don't understand, like an "Integration" tab with untranslated strings and this seems having to do with @Hackwar new router...

@framontb

If I understand it correctly the M3AAWG document you cited in the code comment (https://www.m3aawg.org/sites/default/files/m3aawg-unicode-tutorial-2016-02.pdf ) says that for each identifier (the "user name" in our case) all characters should come from a single script (e.g. Latin or Greek or Cyrillic) or a selected combination of scripts (e.g.: the three ones you cited).

I haven't tested yet, but TBH and at first sight it doesn't seems to me that your case 6: is doing that, but I may be wrong.

@smz
The PR I sent is for 3.6.x.
Do you know a way to detect if all characters come from a single script? A simple way seems to check against each one of them... but how many of them are there? You quoted three (e.g. Latin or Greek or Cyrillic), are these enough for the purposes? Sorry, about so many questions.

@framontb

Do you know a way to detect if all characters come from a single script?

No, I don't. It is quite complicated stuff if you think that Unicode is formed by 17 "planes" each of 2^16 code points, of those 17 planes 3 are allocated to "Basic", "Supplemental" and "Ideographic" multilingual characters and just the "Basic Multilingual Plane" is divided in 160 blocks (of different sizes, IIRC), each roughly corresponding to a "script". And I don't know about the other 2 multilingual planes...

Edit: Look, I found this: https://en.wikipedia.org/wiki/Unicode_block and apparently there are 93 blocks in the SMP and 2 more in the SIP. That adds up to 255 blocks, which means, give or take, about 250 "scripts"

Edit 2: Well, not really: there are also blocks for things like "Playing cards", "Mahjong Tiles" and "Meroitic Hieroglyphs" which we'll probably don't want as user names. But why in the world shouldn't we accept a Tamil, Kannada, Armenian or, heck, even a Cherokee user name?

Difficult issue for me, posted a question in stackoverflow :
PHP regex for checking if a string comes from single script whatever it was
The trivial thing is check against each one. We will see if someone gives better idea.

What about create an associative array of Block ranges in Unicode block (en.wikipedia.org/wiki/Unicode_block) and check the characters of username against it? Could this work?
Just done... http://php.net/manual/en/intlchar.getblockcode.php

But only available for PHP 7 :-(

After some research, I get an implementation of the Unicode security mechanism: Highly Restrictive level (this level will satisfy the vast majority of users), first introduced in the M3AAWG docs above.
I think it deserve some explanation here because of the regex.
This implementation, follows guidelines in UNICODE SECURITY MECHANISMS. The restrictions applied for username field are:

1. Whenever scripts are tested for in the following definitions, characters with Script_Extension=Common and Script_Extension=Inherited are ignored.

2. Only some scripts are recommended for Highly Restrictive level. These recommended scripts are specified here.. I left the 'aspirational scripts' out. If someone think, it should be in, say.

3. All characters in each identifier must be from a single script, or from the combinations.

4. No characters in the identifier (username) can be outside of the Identifier Profile. Because of this restriction, there is a big regex in the script ($regexAllowedChars). Although long, the structure of this regex is very simple, a list of characters (ranges) allowed. I find no other way to access this Identifier Profile that a container regex.

5. Finally there is a commented implementation of the Default Identifier Syntax.. I commented this because it seems to me very restrictive for the field username, disallowing all the unicode characters that no fit /[^\p{Lu}\p{Ll}\p{Lt}\p{Lm}\p{Lo}\p{Nl}\p{Mn}\p{Mc}\p{Nd}\p{Pc}]/u

Comments are welcome.

@framontb
As you know I'm essentially against this PR, but I find this last work you did for implementing the Unicode security mechanism really commendable and possibly useful in many other circumstances.

So much that I propose to move that code in an ad-hoc method for general usage. I don't know which library would be the best for hosting it, but I really hope the PLT could find a place for it.

Bravo!

... actually this, after implementing the necessary options for selecting the various possible profiles, could even be proposed as a PHP function

It would be fine such a function (in PHP or Joomla!), because this check would be arranged with only one line of code, like in the email case.
I found this interesting tool in inet: Homoglyph Attack Generator. Simply, you put your username and choose the most similar chars.
Could be Joomla! the first CMS that implements this security check?

Can we get a New Feature label here too? @brianteeman @wilsonge

Please also take a look at a previous issue #7158 that the rules for usernames in the installation of joomla (Super Users) is not the same as that for users created in Joomla. Would be great of we could gt both fixed at the same time

Fixed Issue #7158, # 10391 in previous commit.

Dunno if this is an issue but "username rule" depend of the "com_users language ini", so if we use this rule outside of the com_users component, there is not translation... noticed the same for "password rule"

First I need to say this is a great improvement idea.
Maybe it's good (if possible) to add a email address blacklist feature for users registering, so you could easily block some domain names with wildcard options.

@manojLondhe, @Devportobello, @conconnl
Thanks for the comments ! However it is unclear whether this PR is appropriate for Joomla. It seems that is under discussion (don't know where or who). It makes no sense to continue working on it until it becomes clear.

@framontb Thank you for working on this. I am in favor of including this into the core. Especially if the wildcard option is included, this way we can have a list of forbidden usernames as well.

Pinging @chrisdavenport to see if he is interested in including this for 3.7. What do you say Chris?

@roland-d @framontb I think this will be a valuable addition to Joomla and I'd be keen to include it in 3.7. From a quick scan of the code I think it still needs a bit of work, but I like where it's going.

As its mentioned above we now have anRTC pr to move passwords to their own tab #10941

With new RTC #10941 , seems this PR is over. If no one is against, I close it.

The other PR is only visual changes, your is about new functionalities.
I would hope you go further with it and extend it a little more so it can ultimately go in 3.7

@framontb Issue #10941 is only a cosmetic change moving the password options to it's own tab. So it is unrelated to this PR in regards of functionality.

Better strategy is to chop this PR in pieces to test it properly like Brian did because the code still needs a bit of work, perhaps from people with more experience.