Forgot to say that a mail like
joomlatest@джумла-тест.рф
gives
joomlatest@xn----7sblgc4ag8bhcd.xn--p1ai
once punyencoded

maybe i'm still sleeping but i do not see any difference before or after apply the pacth
before

after

p.s.
surely i need more coffee

@alikon
Have you saved the article after patch?

ops.... no
after save seems better

hmm, that was working yesterday...
Looking what's wrong now

Thats what I saw

On 2 February 2016 at 07:38, infograf768 notifications@github.com wrote:

hmm, that was working yesterday...
Looking what's wrong now

—
Reply to this email directly or view it on GitHub
#9041 (comment).

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

@alikon @brianteeman

Please test again.

This would be a good candidate for some good unit tests.

Great idea, but not sure if the cleaning function is the right place to do this, specially at the end of the function (because of separation of concerns). Probably a separate, new, function doing just that and called before this function is called would be cleaner and probably safer.

@beat
How would you that?

this helper is called in the xmls with
filter="JComponentHelper::filterText"

Changing all xmls is out of this scope imho

Shall we create a new method in JComponentHelper or JFilterInput and run it as first action in filterText()?

We could do this (if no pb with security) which would let create tests.

diff --git a/libraries/cms/component/helper.php b/libraries/cms/component/helper.php
index 32216d5..925ace1 100644
--- a/libraries/cms/component/helper.php
+++ b/libraries/cms/component/helper.php
@@ -134,4 +134,7 @@
    public static function filterText($text)
    {
+       // Punyencoding email addresses
+       $text = JFilterInput::getInstance()->emailToPunycode($text);
+
        // Filter settings
        $config     = static::getParams('com_config');
diff --git a/libraries/joomla/filter/input.php b/libraries/joomla/filter/input.php
index 8259bb1..b2c69cd 100644
--- a/libraries/joomla/filter/input.php
+++ b/libraries/joomla/filter/input.php
@@ -348,4 +348,30 @@

    /**
+    * Function to punyencode utf8 mail when saving content
+    *
+    * @param   string  $text  The string to filter
+    *
+    * @return  string  The punyencoded mail
+    *
+    * @since   3.5
+    */
+   public function emailToPunycode($text)
+   {
+       $pattern = '/(("mailto:)+[\w\.\-\+]+\@[^"?]+\.+[^."?]+("|\?))/';
+
+       if (preg_match_all($pattern, $text, $matches))
+       {
+           foreach ($matches[0] as $match)
+           {
+               $match  = (string) str_replace('"', '', $match);
+               $match  = (string) str_replace('?', '', $match);
+               $text   = (string) str_replace($match, JStringPunycode::emailToPunycode($match), $text);
+           }
+       }
+
+       return $text;
+   }
+
+   /**
     * Function to determine if contents of an attribute are safe
     *

What do think?

@infograf768 yes that's a better approach, security-wise, as the final filter filters the final result and also better for separation of concern

Ideally, to complete the separation of concerns, you would also move the rest of function filterText() into a new function and just call the 2 new functions in filterText. But that last part is just a suggestion. ;-)

A different question, non-security related, just functionality-wise: Is it ok to do the emailToPunycode on an already punycode-encoded email (e.g. if a content is edited a second time). Means if you re-apply the transform on an already punycoded-email address, does it leave it as is ?

@beat
yes, applying the method to an already pure ascii mail has no effect. wil update this as prposed tomorow.

@infograf768 great! and many thanks for your work on Joomla

@beat @brianteeman @alikon @Hoffi1

! modified the PR by creating a new method in JFilterInput. This will let test specialists create tests.

What you should get:
Source before saving the article (no editor or toggle editor):

Source after saving the article (same when cloaking is off in frontend)

When cloaking is ON, source in frontend will show:

<p>UTF8 mail<span id="cloak58767">This email address is being protected from spambots. You need JavaScript enabled to view it.</span><script type='text/javascript'>
 //<!--
 document.getElementById('cloak58767').innerHTML = '';
 var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
 var path = 'hr' + 'ef' + '=';
 var addy58767 = 'j&#111;&#111;ml&#97;t&#101;st' + '&#64;';
 addy58767 = addy58767 + 'xn----7sblgc4&#97;g8bhcd' + '&#46;' + 'xn--p1&#97;&#105;';
 var addy_text58767 = ' j&#111;&#111;ml&#97;t&#101;st@джумла-тест.рф';
 document.getElementById('cloak58767').innerHTML += '<a ' + path + '\'' + prefix + ':' + addy58767 + '\'>'+addy_text58767+'<\/a>';
 //-->
 </script></p>
<p><span id="cloak53996">This email address is being protected from spambots. You need JavaScript enabled to view it.</span><script type='text/javascript'>
 //<!--
 document.getElementById('cloak53996').innerHTML = '';
 var prefix = '&#109;a' + 'i&#108;' + '&#116;o';
 var path = 'hr' + 'ef' + '=';
 var addy53996 = '&#101;m&#97;&#105;l' + '&#64;';
 addy53996 = addy53996 + '&#101;x&#97;mpl&#101;' + '&#46;' + '&#111;rg?s&#117;bj&#101;ct=джумла';
 var addy_text53996 = 'джумлаs&#111;m&#101;th&#105;ng';
 document.getElementById('cloak53996').innerHTML += '<a ' + path + '\'' + prefix + ':' + addy53996 + '\'>'+addy_text53996+'<\/a>';
 //-->
 </script></p>

This is totally B/C as any ascii mail is not touched and it solves using UT8 mails when placed in a mailto:

Thank you for testing. It would be great to not wait 3.5.+ to get this in.

I have tested this item successfully on 3e4ccc0

Test OK. When the sample piece of html is pasted in the editor and saved, it is cloaked.

I have tested this item successfully on 3e4ccc0

Tested successfully with the provided html snippet plus pure unicode email address without mailto somewhere within the body.
Result:

• Email address with unicode domain in a mailto is punyencoded, anything else is untouched.
• Any email address (unicode or not) in a mailto is cloaked, pure email (i.e. without mailto) is not cloaked.

=> All as desired.

rtc thanks for testing!

Thank you @infograf768 and testers!