There is no screenshot

i have the same problem.
(php-version 7.0)

Screenshot unnecessary. You'll basically just get an error page anyway. Maybe a stack trace if debugging is enabled.

The problem based on what I've seen in the forum is that an invalid JSON string is being passed into the Registry API's JSON formatter and json_decode($string) is failing miserably. The couple of code paths I've seen indicate it's some module configuration from the database.

Sorry - see below

Also, Here is the debug info I get:
Error decoding JSON data: Syntax error
Call stack

Function Location

1 JApplicationCms->execute() /var/www/vhosts/harwoodlandscaping.ca/httpdocs/index.php:49
2 JApplicationSite->doExecute() /var/www/vhosts/harwoodlandscaping.ca/httpdocs/libraries/cms/application/cms.php:261

3 JApplicationSite->initialiseApp() /var/www/vhosts/harwoodlandscaping.ca/httpdocs/libraries/cms/application/site.php:209

4 JApplicationCms->initialiseApp() /var/www/vhosts/harwoodlandscaping.ca/httpdocs/libraries/cms/application/site.php:681

5 JPluginHelper::importPlugin() /var/www/vhosts/harwoodlandscaping.ca/httpdocs/libraries/cms/application/cms.php:669

6 JPluginHelper::import() /var/www/vhosts/harwoodlandscaping.ca/httpdocs/libraries/cms/plugin/helper.php:170

7 plgSystemJblibrary->__construct() /var/www/vhosts/harwoodlandscaping.ca/httpdocs/libraries/cms/plugin/helper.php:255

8 JPlugin->__construct() /var/www/vhosts/harwoodlandscaping.ca/httpdocs/plugins/system/jblibrary/jblibrary.php:45

9 Joomla\Registry\Registry->loadString() /var/www/vhosts/harwoodlandscaping.ca/httpdocs/libraries/cms/plugin/plugin.php:75

10 Joomla\Registry\Format\Json->stringToObject() /var/www/vhosts/harwoodlandscaping.ca/httpdocs/libraries/vendor/joomla/registry/src/Registry.php:366

try disabling the JB library plugin

try disabling the JB library plugin

That might fix this one particular case but it's not just this plugin. See https://forum.joomla.org/viewtopic.php?f=706&t=937036 as an example where the stack trace seems to be for a module on the admin login page.

Brilliant - disabling the JB Library Plugin fixed the issue. Many thanks.

That might fix this one particular case but it's not just this plugin.

i know, just to help troubleshoot

I just wanted to make sure it's clear we aren't dealing solely with an issue of a wild plugin here is all. It does seem like we've got more issues with invalid JSON strings being processed for one reason or another than we should have after an update.

ok I debugged this on several sites - the quick easy fix was to change the

$obj = $handler->stringToObject($data, $options);

to be

$obj = json_decode($data);

Then login, logout, and change the code back... that fixes the "immediate WTF moment" while the root issue is debugged and fixed for real.

to be clear all the stack traces I have seen so far have been during Joomla Admin login page rendering, and all debugging leads to this line in the Registry.php

All of the traces lead to Json::stringToObject() which is json_decode() with error checking. So instead of throwing an error the API's just going to handle a null or false return which does nothing to the Registry's datastore. It works as a quick fix to get things functional again but you're missing the configuration for the affected JSON data payload(s) (luckily our code has enough fallbacks onto default values it at least keeps things functioning).

install the JB library plugin and you will have the error and the callstack on frontend

Im guessing that a string of "" will raise the exception then ? whereas it never used to?

Does json_decode() set an error state with that string?

And actually, "" wouldn't even hit the JSON decoder. If you get back into that function I linked before, it explicitly checks for the first and last characters being brackets and if those aren't found it tries to decode with our INI formatter, which doesn't have an error check like that.

I dont know if it sets an error state - the customer was looking for a quick win this time, and did not want me "playing" on their live site...when I get another ill play more with it, the very early reports were always core login module though - not custom plugins/modules

Does the JB plugin have zero parameters? I think I found a code path when installing extensions that can cause an invalid value to be stored for the parameters field.

If my hunch is right on the parameters thing, #12463 would fix the possibility of the column receiving an invalid JSON string.

We probably need a separate PR to reset the column's values if the value is empty or NULL so it has a correct default value of {}.

The screenshot of System Information shows: Joomla! 3.5.1
(?)
regards, christine

It says 3.5.1 because that is the only back up I had to restore it with.

so the fix for this is PR #12463 so this should be closed @mbabker ?

No. The code fix right now would basically be wrapping certain places trying to create Registry objects with data that is supposed to be in JSON form but is invalid in try/catch blocks and just create null registries instead similar to https://github.com/joomla/joomla-cms/blob/3.6.3/libraries/cms/menu/menu.php#L83-L98

That PR, while discovered as part of it, only addresses a small issue of a core code path being able to create an invalid data record.

Got to love the doc block in that code :( Technical debt, everywhere I look... I fear one day Joomla might be bankrupt... (Just reading about JObject and JError, not leaving us for 10+years to come...)

ok in this case of this particular extension (JB library plugin) the problem seems to be unproper regex replace that this extension uses on install (see install.mysql.utf8.sql inside the package).
From what i checked....

-- This extension injects the params on install with
{ ... "jqregex":"([\\\/a-zA-Z0-9_:\\.-]*)jquery([0-9\\.-]|min|pack)*?.js" ...}

-- which after the query executed is stored in the database as:
{ ... "jqregex":"([\/a-zA-Z0-9_:\.-]*)jquery([0-9\.-]|min|pack)*?.js" ...}

-- which in param json_decode turns to:
error! is not valid json!

-- -------------------------------------

-- It should be inject sql with:
{ ... "jqregex":"([\\\\/a-zA-Z0-9_:\\\\.-]*)jquery([0-9\\\\.-]|min|pack)*?.js" ...}

-- which after the query executed is stored in the database as:
{ ... "jqregex":"([\\/a-zA-Z0-9_:\\.-]*)jquery([0-9\\.-]|min|pack)*?.js" ...}

-- which in param json_decode turns into escaped regex (the extension in this param uses regex):
jqregex = ([\/a-zA-Z0-9_:\.-]*)jquery([0-9\.-]|min|pack)*?.js

So, imho this is a problem with the extension. joomla is working fine

Hello, I got this problem on my side.

This time the $data received was {""}, which is of course invalid, but I think throwing is a bit violent in this case (I could'nt do anything on my website).

I var_dump a traceback and I found the problem was at in administrator/components/com_login/views/login/tmpl/default.php:20 with $loginmodule["params"] being '{""}' which effectively reflect the state of my database.

I think the throw introduced in Json.php is a bit risky !

Maybe we could at least test that we are note in the {""} case !

The API that's throwing the error is a very high level API. It's not the only class at that level possibly throwing errors, it's just that with Joomla's heavy use of JSON it's the one that will be most visible (the XML and YAML formatters could throw errors too but those are unused by core).

So the thing with this (and other changes in the last few releases which have caused more errors to be thrown) is that they are tightening up conditions where bad configurations or data were being processed and known error states blissfully ignored if even detected at all, but they can all easily be caught and handled by lower level implementations. With the cache API the new errors people have been groaning about relate to misconfigured servers, this error state relates to invalid data syntax (which the core API doesn't produce, so it's coming from some tool issuing a bad query somehow).

We found one place in the core code that was generating invalid JSON syntax. We patched that section of code to catch this specific error condition because we knew because of a core screwup this error could be hit in that code path. See https://github.com/joomla/joomla-cms/blob/3.6.3/libraries/cms/menu/menu.php#L83-L98

We can't account for the fact that others may have inserted invalid data without either removing all error checks and blissfully discarding bad data (AKA the 3.6.2 way) or by working through every new Registry(source); or $registry->load*() call to catch the thrown exception and let the local code blissfully ignore the error state.

Just in case, this SQL request removed the problem for me.

 update `xxxx_modules` set params="" WHERE params = "{\"\"}";

I agree that it's better to catch error than to ignore them, however Joomla may, maybe, provide a "known bug" for this issue, or help migration for this very trivial case ? I won't be surprised that the {""} was set by joomla itself at some point in the past (this is a 6 year old joomla site). I can't see why a module would set params for login module ! So maybe the problem will be wide spread on old instances ?

One of the reports I saw on the forum this morning noted that jUpgrade used that value for sites it was used for. So that would be one source of a third party tool causing issues if that's indeed the case (I can't confirm it so I'm taking that at face value).

Are you aware of anything that could make me know if jUpgrade was used ? I inherit from this site so I don't know.
The only reference to JUpgrade I find is in a file of josetta, but it doesn't seems to mess with params (maybe in the past ?).

I can confirm that running the SQL query fixed the issue for me.

updatexxxx_modulesset params="" WHERE params = "{\"\"}";

One of the reports I saw on the forum this morning noted that jUpgrade used that value for sites it was used for. So that would be one source of a third party tool causing issues if that's indeed the case (I can't confirm it so I'm taking that at face value).

I have updated about 20 websites so far and had four of them having this problem. All 4 websites did run on Joomla! 2.5 and 1.5 earlier. I'm not 100% sure if JUpgrade has been used to migrate them, because sometimes I used JUpgrade and sometimes SPUpgrade was used.

None of my websites build with J!3 from beginning has any problem with the 3.6.3 update.

update xxxx_modules set params="" WHERE params = "{\"\"}";

Solves the problem.

I think it may be a good idea to add above update sql to Joomla core as there were really only 2 commonly used tools to update your Joomla 1.5 site.

Wow, this sure caused some issues... I updated 3 different sites and only one of them got affected by this error. The SQL command didn't do anything for me, it didn't affect any rows since none of the params where set to "{\"\"}", so I have no idea where that null JSON data comes from. I also couldn't even find "JB library plugin" in the database.

What worked for me was to go to /libraries/vendor/joomla/registry/src/Format and edit Json.php:
Comment the line 72:
throw new \RuntimeException(sprintf('Error decoding JSON data: %s', json_last_error_msg()));
After that I can login and navigate the admin panel. If the decoded data coming back as null isn't that big of a deal, throwing an exception like this seems quite dramatic.

After that I can login and navigate the admin panel. If the decoded data coming back as null isn't that big of a deal, throwing an exception like this seems quite dramatic.

At that high level of an API, it is appropriate. The issue though is that the lower level code doesn't expect error conditions and therefore doesn't catch them.

For anyone having this issue specifically with jblibrary, a new version has just been released with a fix.

The issue was as andrepereiradasilva pointed out above. The latest version will update the regex without touching other parameters.

In my case, it was due to a content plugin which added extra fields to articles. It was pulling extra fields from xml, which had all fields wrapped under <fields name="attribs">. Even though data was saved in a separate table, as unwanted (and unnoticed) side effect they were also saved in the attribs column of the content table. Some of them had large entries, so since attribs column was limited to 5120 characters, data was truncated. So then opening any article with truncated attribs spit this error. Maybe someone else has such plugins, so at least this can serve as a pointer where to look and what to fix (not a Joomla problem imho)

I got the same problem when clicking an article in K2. Then I get this error. It's because of a bug in Tiny MCE.

It's because of a bug in Tiny MCE.

What kind of bug how did you fix this and who told you this is a bug in TinyMCE?

The guys at Gavick did some tests because I had used their template in combination with K2. Eventually this tiny MCE editor was being replaced with NO editor and eventually with JCE. Now it works.

Eventually this tiny MCE editor was being replaced with NO editor and eventually with JCE. Now it works.

I'm confused does it work or not with tinyMCE? I guess you use the core TinyMCE? (just to be sure). If it not work because of a real bug in TinyMCE this needs fixing. If it is a problem with K2 & TinyMCE or Gavick & TinyMCE the developers of there compontens / templates need to fix it ;)

I wonder which solution is the best.
Comment the line 72:
throw new \RuntimeException(sprintf('Error decoding JSON data: %s', json_last_error_msg()));
worked for me
or
update xxxx_modules set params="" WHERE params = "{\"\"}";

The latter.

Commenting out the former will mask the error but doesn't address the issue that you have invalid data.

Thanks .. :-)

update xxxx_modules set params="" WHERE params = "{\"\"}"; didn't work for me, it returned 0 affected rows.

I'm using the comment line 72 fix so that I can have the site online but I know it's not the correct solution.

Any ideas on how to find the offender? I don't use JB Library nor K2 with Gavick which are the other problems found on this thread.

Thank you.

update xxxx_modules set params='' WHERE params = '{""}';

That worked for me today. It could also be a difference of MySQL clients.

That worked for me today. It could also be a difference of MySQL clients.

No, didn't work either.

The problem is with a module having exactly {" "} in the params column? Because I don't have any of those, it goes straight from blank params to params with information. I attach a screenshot.

Thank you.

It could be one of many JSON related columns in the database. The extensions and modules tables are going to be the two "worst" culprits of this. It could also be other tables depending on where in the site you're hitting issues, i.e. the content table has 4 JSON fields.

I just updated 10 sites and two of them that were upgrade from older versions of Joomla in the past got this JSON error which in my case was solved by running:

update xxxx_modules set params='' WHERE params = '{""}';

The query returned 4 instances in each case.

Someone pointed out to me they were having issues with the joomla.org backend and I found a bunch of modules with that same issue. It's unfortunate the issue exists, but at least we can point people to the "problem" records.

I totally agree I'm just happy to find a quick and easy fix.

I had the issue show up on the administrator login with the front-end working fine. I do not have JB Library Installed. I do have Ark Media Library and PHP Excel Library Installed. I ran the command: update xxxx_modules set params="" WHERE params = "{\"\"}"; in my SQL query and I can now login. It says 4 tables were updated.

I updated 80 sites today. This is the only one with the issue. It is a site that has migrated from 1.5 to 3. All other sites have Ark Media installed.

Thanks for digging into this folks!

Using "What worked for me was to go to /libraries/vendor/joomla/registry/src/Format and edit Json.php:
Comment the line 72:
throw new \RuntimeException(sprintf('Error decoding JSON data: %s', json_last_error_msg()));" worked for me but the problem returned after this last update.

Well no kidding.

1) You core hacked to "fix" the issue by masking it.
2) You didn't actually fix the underlying issue of invalid data.

I hacked it to "fix" the issue waiting for Joomla to release the fix. Since there was a new release, I thought it would be fixed. I just notified others who are having the same issue that it didn't "fix" mine. But thanks for the critical thinking and helpful commentary.

Joomla isn't going to release a fix. There isn't a code error here. Unless you're saying that adding additional checks to ensure valid data is being processed is an error. Yes, it sucks majorly when these error checks bring down sites, but it exposes invalid configurations or data records and those root causes are beyond the scope of what Joomla can fix.

Joomla isn't going to release a fix.

This should move to the FAQ and then close this issue.

My issue isn't a front end issue. It's within the standard admin. The only time it happens is when I edit an article. It didn't happen before the Joomla 3.6.3 update. It happened after. Out of all of the websites I manage, this is the only one to have the issue. I cannot find an issue with MySQL. So tell me what configuration would cause an issue with editing an article?

Problem announces itself also with Joomla sites updated from v1.5. The articles created in the past have attribs column filled (XXXX_content) with html entities and json_decode can not handle them. As proposed earlier, do not disable line 72 but change line 67 to:
$decoded = json_decode(html_entity_decode($data));

Despite completely understanding why this error check was added and supporting good practices, it is very inconvenient in terms of timing (unpredictable, obviously) with Joomla 3.6.4 fixing critical security issues released yesterday.
I have not tried the proposed fix yet, but I believe Joomla should update its upgrade tool to run the fix during the process (as it has been shown most of people are experiencing the problem due to core misconfiguration).

This is nothing to do with the security release Joomla 3.6.4 - this change was released before that in Joomla 3.6.3 - please refrain from starting rumour that give people reasons not to apply urgent security releases to their site.

No, People are not experiencing the problem due to core misconfiguration, again wrong, most people are experiencing this because they have invalid data in their database,

Oh I'm sorry but this has everything to do with the 3.6.4 update : when I tried to update to 3.6.3, I got my entire site unavailable. Back + front.
So between having a 3.6.4 secure website only displaying an error page to my users, and having a functional site, I choose the latter.

I will take time to create a test environment to test the proposed fix, but until I have time for that, upgrading is not an option.

+1 on Phil's comment. It's not a configuration issue nor is the common culprit a result of the core code inserting invalid JSON data (actually I've not found a place in core that results in broken JSON). The culprit is usually third party tools issuing bad queries.

Joomla core updates generally do not touch the data records in one's website. SQL changes are only made to alter table structures or to make changes in places we cannot trigger changes otherwise, such as inserting records for new core extensions since the updater does not trigger the extension install API. So again, there is not a code fix that can be reliably issued short of removing the error check, which is not going to happen. The only fix is to review the database for records which contain invalid JSON strings.

Oh I'm sorry but this has everything to do with the 3.6.4 update

Factually incorrect.

While hiding the error isn't a solution, introducing the error check while now afterwards knowing how many are affected by the supposedly invalid data (that wasn't causing any trouble what so ever before), and that obviously the users themselves had no idea were there, should be considered a breaking change.

Let's face it; most people using Joomla aren't exactly that experienced developers that know coding or SQL that well, which probably is why they also chose a CMS platform like this to begin with. For those, just updating to a new minor version, and then not being able to login or get any information except "Invalid JSON data" or similar as error message, is devastating.

The least thing that should be done is to improve the error message to include which module/plugin or whatever that last called with the invalid data, as well as some extra information pointing the user to a guide of some sort to address the issue in the database.

Oh I'm sorry but this has everything to do with the 3.6.4 update

This has NOTHING to do with the 3.6.4 update and stating such only serves to spread FUD. I wrote the patch that added the error check for invalid JSON data. And as the security team lead I wrote the patch fixing the security issues after confirmation with the rest of the team.

Please do NOT come onto this forum and discredit any contributor to this project by spreading FUD. Frankly all it does is make you look less trustworthy.

The least thing that should be done is to improve the error message to include which module/plugin or whatever that last called with the invalid data, as well as some extra information pointing the user to a guide of some sort to address the issue in the database.

If possible that is the only thing that can be done. Sadly Joomla cannot be responsible for badly coded extensions of which several have now been identified. And Joomla or any system should never mask errors

should be considered a breaking change

Checks ensuring data integrity are most assuredly NOT a breaking change.

Ok, sorry if you felt insulted, let's rephrase : my original message said that it was a really bad time for introducing such site-breaking change.
And THAT (the fact that it is a very bad timing), has to do with 3.6.4. Without this security update, the 3.6.3 upgrade could have been waiting some time.

ok in future I will personally ensure that critical security releases are only ever released when it is convenient for you and you have fixed your broken web site.

(unsubscribed)

@mbabker While the check itself isn't a breaking change, the consequence is that many sites are affected and they were working before 3.6.3, and now they aren't. I don't know the figures on how many sites around the world that use Joomla! today, but from the homepage it's stated that 75 million+ downloads have been made and millions of sites use Joomla!, so let's just speculate on the affected sites, assuming they are up to date, and say that it's 1 in 100 that get this error when updating to 3.6.3. That could mean tens of thousands of sites affected, and most of which probably has a developer who might not know where to go for a solution. I don't think that's a very good thing for Joomla! as a "trademark", breaking change or not in formal terms. I think this shouldn't be looked lightly at.

I am preparing a patch now that reverts all data integrity checks I have introduced this year to the core code as the community at large has indicated they would prefer Joomla silently absorb errors with processing database records or allow the code to reach fatal error conditions due to misconfigurations in the site's global configuration or at the server level. Since the community feels integrity checks that "break" misconfigured sites or cause issues with invalid data records are too risky, apparently the only acceptable approach is to allow these broken sites to function.

tens of thousands of sites affected

pah

24 Participants

pah...

@PhilETaylor Please don't use big corporation ideas like that... Just because all affected user don't sign up to github or a corporate forum and take time to write their complains or concerns, doesn't mean they don't suffer from the issue. Most don't even know where to go or that they can bring up issues here. Faulty TVs/computers or broken Joomla! sites, doesn't matter, there's an issue and it's not pretty. What if you were faced with your computer not starting all of a sudden, with a BSOD saying a non-critical app has faulty data and therefore you cannot use your computer any more. That's the severity of whats going on right now in Joomla 3.6.3. If someone can point to a good fix that works for everyone, then sure. then you can start taking this more lightly and point to that. But trying to make it out as if there's no issue to fix, I think is the wrong approach.

suggestion, when having broken JSON data in your database:

1. TEMPORARY comment line https://github.com/joomla/joomla-cms/blob/staging/libraries/vendor/joomla/registry/src/Format/Json.php#L72 to supress the exception.

2. Next solve the invalid JSON data in database (check your JSON data with any online json validator if you need - https://www.google.com/search?q=json+validator)

3. When finish, reenable the exception by uncommenting the line you comment in 1.

tens of thousands of sites affected

FUD. Nothing at all to back your claims.

If someone can point to a good fix that works for everyone

@mbabker did:

The only fix is to review the database for records which contain invalid JSON strings.

But you don't like the ONLY fix for this problem.

But trying to make it out as if there's no issue to fix, I think is the wrong approach.

This is open source - submit a PR if you want to propose a solution.

Talking further is not going to resolve your thought, you have pissed off some very senior Joomla Contributors this evening and made no friends.

@mbabker is going to propose a PR to revert this change - which is a massive backward step for Joomla and one I personally hope the PLT rejects and doesn't merge.

Now I think this issue should be locked and as previous stated, moved to a FAQ on the wiki https://docs.joomla.org/Category:Version_3.6.3_FAQ

Have not been able able to resolve the issue, nor replicate it on a test box.

I've only been using Joomla for 5 or 6 years, but I am glad that after all these years that the team decided to implement data integrity measures. You know- years later, out of the blue, no warning. But the best part of it all is releasing a critical security patch a day or two later- while many of us are still trying to recover from the 3.6.3 release.

In real life, a move like this would equal unemployment.
While I have great respect for team Joomla and the efforts therein, this has become the kind of issue that makes Joomla a bit less popular then some of the others. Not so much the issues themselves, but the high and mighty responses to frustrated users.

Not so much the issues themselves, but the high and mighty responses to frustrated users.

Like:

1. Clearly explaining the solid foundational technical reasons for the change - repeatedly.

2. Providing some helpful hints and tips on how to IMMEDIATELY fix an offline site temporarily

3. Providing information on how to resolve the root problem by searching for, and fixing invalid JSON data in your database

4. Encouraging you to see that this is NOT a Joomla Core issue, but one caused by 3rd party code creating invalid data in your database - shout at those crap developers, not the core contributors!

out of the blue, no warning

5. Taking time to release THREE RELEASE CANDIDATES with this change to handling JSON data in that you could have tested since 28 June 2016 until the Joomla 3.6.3 release on 18 October 2016 - not one single report of this issue was raised during this time - once can only assume no one bothered testing...

releasing a critical security patch a day or two later

Yes, a whole team of people you are now abusing gave up THEIR weekend to test and prepare a fix for YOUR SITE.

All while trying to co-ordinate one of the most critical security releases in Joomla's history... yeah, I think you should have "respect" for a team that manages to pull this off!

#12562 reverts the checks. Take it or leave it.

Closing this issue per standard protocol since there is a patch available for testing. God help this project.

If all else fails and you'd like to hire someone, send us an email. Our charge is only $50 and we guarantee a fix or your money back.

https://www.joomlashack.com/blog/tutorials/error-decoding-json-data-syntax/

I rest my case...

Just my two cents in case someone else with my problem is reading this.

It took me a couple of hours to find the corrupt data in my database. The problem was that I was searching for the error in the "XXX_modules" and "xxx_extensions" tables. I even read each record one by one and couldn't find it because it wasn't there. So, my first comment/suggestion for you is: The problem can be anywhere! as was stated by many people in this thread but if you only do a fast read of the comments you could miss it (like I did).

After wasting those hours, I did what I should have done from the beginning: Carefully read all available threads on the matter. And to my surprise the fastest way to solve the problem has been online since Oct 19!

Here https://forum.joomla.org/viewtopic.php?f=706&t=937036 on Oct 19 the user ibrentlam offered the following solution:

The error is pulled in the file:
/libraries/vendor/joomla/registry/src/Format/Json.php`
line 72
I changed it to:
throw new \RuntimeException(sprintf('Error decoding JSON data: %s the bad data is: %s', json_last_error_msg(), $data));
so I could actually see the data.

And oh! magic! In a matter of seconds I:

1) Reloaded my website and found the offending param.
2) With that info I went to phpmyadmin and performed a search of such param on all database tables
3) Found the table with the problem: xxx_template_styles (which I never would have guessed)
4) Fixed it (It was an space that shouldn't be there)
5) Returned line 72 to its original state

So, as a final thought: I know that when you have an offline site you panic, and if on top of that you are like me and you have a client shouting at you over the phone because his site is offline you can't think straight. But instead of wasting time fighting with other people on threads, use it to google and read everything you can about the problem, the solution might already be there.

Hope this helps someone and I want to thank Brent Laminack (ibrentlam in the joomla forums) because his answer solved my problem in a matter of seconds.

mbabker I think you took too emotionally, and I think that can be understood as some people are a bit aggressive on the thread.

@PhilETaylor made a good point telling that no one ever test the release candidate on a broken site. On the other side, joomla is rarely a critical point of failure for organisations, so I don't know who have the budget to do such methodical testing (but everyone should then be ok to pay consequence for that).

On the other side no one can just ignore frustration from user, you have a semi-automatic update process (that is, you allow non techie to do it) and your site is now no more working.

All that to say, the problem was hardly predictable, and inserting data validation is, per se, a good move, so @mbabker did not make anything wrong.

I think @pablotll suggestion could be the right patch (or just entry in the FAQ).
Another suggestion would be to be more merciful in JSON decoder, and for now emit a Warning, exposing the data which is not ok. Then with 3.7 release, throw exception.
Sorry I'm not able to do the PR myself, as I'm just occasional joomla user, with no big experience of it's core.

@pablotll I did the same as you did, adding the $data to the exception and found the problem in another area. In the xxxx_content there were old articles created with joomla v1.5 or so. The attribs column is filled with HTML entities and json_decode doesn't like that. I think for the stability the check is okay but run-time solving due to feedback from users on the next patch should also be supported. As proposed earlier, do not disable line 72 but change line 67 to:
$decoded = json_decode(html_entity_decode($data));

Thank you @pablotll that actually worked ( i believe this line of code should be integrated in the joomla in order to see what actually is not in the correct format).
In my case was no plugin or component as i had already search thorowlly in every database for many hours without any results. What was causing it was incorrectly saved json string for com_users component in extensions table.

Thanks you @pablotll, dat was perrrfect!
My site Admins are up and running again thanks to you!
Cheers

@hamsteg Be careful, your solution may corrupt fronpage

@N1kell why? (and its still working ;)) The bug itself was long time present (since v1.5). Another solution lies in the question: Can I remove the data from the attribs column from the older created articles? (like the later created articles).

I have debugged a live site this morning for a paying customer, and on close inspection of three sites I see an absolute mess of json in params... It looks exclusively 3rd party extensions that have caused this in their rows in several different database tables.

The current SQL to "fix" this corruption of data is as follows - which is provided here with no support/warranty at all - "it works on my machine" and is known not to be an exhaustive list of all columns in all tables that support JSON entry.

Don't forget to replace #_ with your database prefix._

sql moved to https://gist.github.com/PhilETaylor/8737e58ebba822ca72fe053e2b4f1047

There also seems to be some issue Im still playing with there some JSON in the db is slashes quotes and some places its not.

Thanks to the paying customer for supporting this free Joomla support financially, and with providing their broken sites for inspection to identify this issue.

@PhilETaylor {} is perfectly legal value for params, indeed its the one that Joomla uses by default in many places. I would set the params to {} instead of empty string, as while it works, empty string will make Joomla to go to legacy mode (read Joomla 1.5) and try to load the value as INI.

The SQL Im working on, that has worked on 3 sites so far, is now hosted and maintained here:

https://gist.github.com/PhilETaylor/8737e58ebba822ca72fe053e2b4f1047

Nice to see updated SQL which makes your site more future proof if Joomla ever decides to remove backwards compatibility from Registry class. ?

The second part may break your site, though. This is because of its perfectly legal to have " inside the parameter values. Try modifying some text field by adding " into it and then run the query. It does break as JSON escapes double quotes with \"...

Yes I know that - but if you are using quotes in your json values then you are an idiot anyway, the params fields are meant for small data snippets

Today I have even seen hard and soft carriage returns in json, before json, after json and even broken json from a 3rd party extension.

Its clear there are some less than perfect JSON in Joomla db's !!!

I'm not talking extensions here. Users can go to admin, edit any text field by adding double quote into there and hit save. Many times fields may also contain some html (read modules, which save all data into params). This is why second part of your script dangerous.

Also its perfectly legal to save JSON string (as a string) into JRegistry. In fact I'm doing that in some quickly wrote modules as I was too lazy to make my quick JavaScript field to convert the JSON into an array. The value is passed as a JSON string to JavaScript, which updates the field and encodes the value back to JSON. I bet this method is widely used as opposed to a lot of code, you will just need 2 lines of JavaScript.

But yeah, after fighting a few days trying to figure out what's going on in numerous customer sites, I can totally agree that way too many extensions are just badly written...

I just wrote a script to check all params and rules columns.

https://gist.github.com/robwent/2b945d860cdecf8f877fcf5955b00c94

Use at your own risk :)

I've added an updated version here: https://github.com/robwent/joomla-json-db-check

It will first fix any bad empty param fields (Jupgrade?) and then can do a full check for invalid values and report the problems (Without any modifications).

This is my error. Can i ask you what's 'not ok in #_extensions table (com_config i guess) ?

0 Error decoding JSON data: Syntax error the bad data is: {"filters":{"1":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"9":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"6":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"7":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"2":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"3":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"4":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"5":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"8":{"filter_type":"BL","filter_tags":"","filter_attributes":""}},}}}

The last part should not have ,}}:

{"filters":{"1":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"9":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"6":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"7":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"2":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"3":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"4":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"5":{"filter_type":"BL","filter_tags":"","filter_attributes":""},"8":{"filter_type":"BL","filter_tags":"","filter_attributes":""}}}

Check it with https://jsonformatter.curiousconcept.com

Thank you robwent!

xxxxx_assets, column rules
18 rows modified.

xxxxx_categories, column params
5 rows modified.

xxxxx_contact_details, column params
6 rows modified.

xxxxx_content_types, column rules
24 rows modified.

xxxxx_dpcalendar_events, column params
1 rows modified.

xxxxx_dpcalendar_locations, column params
72 rows modified.

xxxxx_droppics_pictures, column params
26 rows modified.

xxxxx_extensions, column params
66 rows modified.

xxxxx_kunena_topics, column params
9 rows modified.

xxxxx_kunena_user_categories, column params
2 rows modified.

xxxxx_kunena_user_topics, column params
9 rows modified.

xxxxx_menu, column params
25 rows modified.

xxxxx_modules, column params
11 rows modified.

xxxxx_rokcandy, column params
139 rows modified.

xxxxx_tags, column params
23 rows modified.

xxxxx_timetable_categories, column params
14 rows modified.

xxxxx_timetable_columns, column params
18 rows modified.

xxxxx_timetable_events, column params
45 rows modified.

ah wow lol, was that a site updated with jupgrade?

@rob...
No. Built bottom up, though some of the calendar data (events / locations) was imported from J1.5.
The template and its baggage appears to be the biggest culprit.

Quick install package ... let me guess?