Related issue/PR/discussion:
#14750
#14958

@wilsonge As stated already in the comments of the referenced PRs, it was wrong to merge #14750 and it should be reverted and the original issue fixed with the PR #14958 instead (which you closed).

@wilsonge this is waiting on you!!

I can write a new PR to revert part of #14750.

Should I revert only custom_data column or both?
If I do this, I change the default column value to NULL. (before it was an empty string).

To be sure that the new PR will work on Postgresql. I would like to wait until #19707 will be merged.

I would restore both. There is no reason to remove them to begin with once the underlying issues is solved.

There is no reason to remove them to begin with once the underlying issues is solved.

And IMO there is no reason to keep them if their presence can't be justified beyond "well, they've always been in the schema". What's their purpose, how are they used, etc. So if we're seriously putting them back then make sure they have a reason to be there, change for the sake of change is a bad idea but so is holding onto something with no rhyme or reason.

Both were used in the past as was explained multiple times already.

• custom_data for 3rd party extensions. Eg a module or a plugin without own database tables can store some small data in there.
• system_data was used by core for storing compatibility information. It may used in future again. One can argue about this one of course but imho it doesn't hurt. And it may of course also be used by 3rd parties due to lack of documentation.

I agree that some sort of documentation should be added as well.

Unless someone can give me an actual use case for these fields I plan on leaving them out. Currently the only use case was from @Bakual who used them for storing major version compatibility info - which has now been superseded by the native Joomla Update functionality from 3.10.

@wilsonge The custom_data still is used by 3rd party extensions. What proof do you need other than a developer actually opening an issue about that?
I can't give you more details as I don't use it myself, but I can see its use for them.

I don't know yet how the thing works in 3.10, but there may or may not other usecases in future or by 3rd party.

I don't understand where your hesitation comes from. It absolutely doesn't hurt anyone to put them back. It didn't hurt the last 10 years and will not in the years to come ?
The fix to solve the original issue is adding two properties to a class. Doesn't sound terrible to me.

the real problem is that there is not a "schema" nor a "data model" re-design

I think @Bakual is right these fields should stay, as they are very useful to storing custom_data as related to the extension.

Lets add documentation explaining to the developers that they can use it, but common don't remove it.

Is there anything else needed to proof the custom_data field is needed and should not be removed?
This issue is open for over a year now and multiple people stated the field should be kept.

One little use in core could be for plugins that run periodically and store their last run timestamp. Having it stored along with params causes the plugin to run every time a plugin is edited. Just a nitpick though.

Right now I'm sitting on this. Right now we have two developers who are using the custom_data field. This clearly doesn't fall under the 90% rule. HOWEVER i totally accept right now not many people are testing extensions on Joomla 4 yet until we get a beta out. So I'm aware this exists but unless i see more developers saying they use this table I'm not planning on the custom_data field yet.

Really? so we have to setup a place to take a poll for this and then do a marketing campaign to get all those not following Joomla on github to come and vote? that seems like a bit much... ?

I know many, many Joomla 3rd party developers who are not following the Joomla issue on github, that is why respond on this forum is not a poll.

Secondly I have not been using this field for the very reason that I can't trust it to always be there... that is just sad. So at this point I am seeing a very common problem, we can't please everyone... and we can't trust the Joomla core tables... lol

So @wilsonge if you think it is a waist of time.. and really should just be forgotten as it is the cause of some irritation... then dump it and lest move on. But don't say it is due to the lack of support... it does not have the support because of this very open issue, which is open for over a year now.

Who asked that it should be removed anyway? (I would like to see how many supported that :)

One little use in core could be for plugins that run periodically and store their last run timestamp. Having it stored along with params causes the plugin to run every time a plugin is edited. Just a nitpick though.

I'm currently developing a plugin for 3.9 and I just wanted to use this field to store technical values (timestamp and key). It will be a pity if J4 doesn't have this! Then we will have to make separate tables!..

OK Fair enough. I call quits. Put back in custom_data. system_data is still getting the chop though

@wilsonge can you add the custom_data column again for the next J4 beta? We want to test our YOOtheme Pro theme

I'll check on weekend and will see if I can make a PR.

Do I get it right, it is sufficient to bring back the only custom_data and not the system_data?

Correct

@wilsonge We have the problem now - similar to the other recent problem - that we can't add the columns back with an update SQL script when updating a 4.0 Beta-1, -2 or -3, because this would break updating from 3.10 where the columns still exist. A way out is to add them back with an additonal step in script.php. Would that be ok for you?

Update 11:36 UTC: If you decide not to accept PR #30369 so we live with broken update from 4.0 Beta-1, -2 or -3, then my PR #30373 is sufficient as it is. But if PR #30369 is ok and gets merged, it's worth to extend my PR here a bit as mentioned above.

I'm tempted to accept that to go from beta 3 to 4 we're going to need people to run a one-off manual step. Thing is even a script.php entry for the other issue is going to fail because we try and load the template too early (which is why we had to backport to version 3.10 in the first place).

@wilsonge We could add back the creation of these database columns to an update sql for J4 and could make the installer which runs the updates catch exceptions and check if the statement is an "alter table .. add column" statement and if the caught exception is an "column ... already exists", and in this case just silently continue. Sorry, wrong Issue. See PR #30369 for where it belongs to.

Right, for the other issue it would not help.

Closing as having a pull request. Please test #30373 . Thanks in advance.

Wait with testing a bit, I have to finish something on that PR. Will be ready latest tomorrow.

Now PR #30373 is ready for testing. Finally.