Do we have more details on it?

@dgt41 I've just tested and everything is good. See the prove:
PR jtester/tests#50
Tracker http://issues.joomla.org/tracker/jtests/50

Seems to be ok now. I will close this one

Ok I came across to this problem once again!
I realize that the problem isn’t the bot but maybe the editor of jissues?
Anyway these are two links and two images that show the problem
on github joomla/joomla-cms#5815

And on jissues http://issues.joomla.org/tracker/joomla-cms/5815

As you can see on Jissues nothing gets parsed after the emoticon!

Confirmed. The strange ist the following image. If we follow the diff on the Tracker the bot added the lines after the emoticon but it is not there:

I can't fully understand at what moment it happens.

@dgt41 can you describe step by step what you did and I will try to reproduce. Thank you.

BTW this emotion looks quite strange on GitHub.

My two hunches right now are that our controller isn't handling filtering it very well (which may mean an issue in Joomla/Filter) or the GitHub API isn't handling it very well. To debug, I'd suggest dumping the data just before we send a request to the API to render the markdown data and see what the description field contains. If it's good before that point, then that seems to point to either a GitHub API issue or an issue with Joomla/GitHub and Joomla/HTTP and handling the request for that; whereas if it is missing content before that point then it would seem to point to a filtering issue.

@zero-24 The strange character should be unicode U+1F603 the third one in this list
Any idea where is the mapping for emoticons?

@b2z Steps to reproduce:
Create a new PR on github, some title etc but in description add an emoticon with its Unicode (not as :smilr:)
Then let the bot fetch it
and then try to access it in Jissues

but in description add an emoticon with its Unicode

May be this is the problem. I will try to reproduce.

@dgt41 I am trying to reproduce, but do not understand how to add the emotion in Unicode. I tried to use U+1F603 but it is only text. Please explain how to use it :)

@b2z can you copy paste

Ah ok got it now. ????

So I have tested and I do not know how to fix it :( I traced the data till the updateObject() method and here is how the fields looks like just before the query statement execution:

As you can see description and description_raw are good. But after saving in the DB they looks like this:

:(

Can the database not handle the characters that are getting printed after the closing tag? The table is defined with a utf8 charset, the columns in question are utf8_general_ci collation and utf8 encoded.

Seems so. I trying to Google for it, but nothing found at the moment.

Isn't this a filtering problem? My guess is tha <g-emoji is not a valid, known tag in the filters and thus rejected.

It wouldn't make it to the database if there were a filtering problem.

@mbabker you re right, of course...
Found this:
create table MyEmoji (emojicolumn VARCHAR(255) CHARACTER SET utf8mb4) default character set utf8mb4;

Emoji is represented as 4 byte characters. That’s mean your MySQL table charset should be set as utf8mb4.

If we are running MySQL 5.5.3+ then we could switch to utf8mb4.

We're on 5.6 on the server.

@mbabker does it make sense to switch to utf8mb4 ?

I don't see an issue with it.

Then close it

How about a PR to update the default schema to use the utf8mb4 character set

@mbabker you are so kind

So with any luck, ????will work now...