? Success

User tests: Successful: Unsuccessful:

avatar alex7r
alex7r
2 Jun 2016

Pull Request for Installer run all SQL updates.
Installer run all SQL updates starting from previous version, and not paying attention to current version that we are updating to.

Summary of Changes

Added "top_version" range

Testing Instructions

Update extension from version 3.1.020 to 3.1.24
With
3.1.019
3.1.021
3.1.024
3.1.025
sql update files
result: version installed is 3.1.024 and DB version is 3.1.024 (instead of 3.1.024 and 3.1.025 as before)

avatar alex7r alex7r - open - 2 Jun 2016
avatar alex7r alex7r - change - 2 Jun 2016
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 2 Jun 2016
Labels Added: ?
avatar mbabker
mbabker - comment - 2 Jun 2016

The database schema version is not directly related to the extension version (though that is generally used). This would also break the CMS' updates as it uses dated schema files (i.e. 3.6.0-2016-06-02.sql) and your use of version compare in the check breaks this functionality.

avatar brianteeman brianteeman - change - 2 Jun 2016
Category Installation SQL
avatar alex7r
alex7r - comment - 2 Jun 2016
  1. Joomla using this version compare function now and nothing is broken.
  2. I passing extra var and if it not passed(as in CMS' updates, so it's affect components installation) then additional check is not running.

schema version is not directly related to the extension version

  1. so I guess we should provide ability to relate them if some one need
avatar mbabker
mbabker - comment - 2 Jun 2016

so I guess we should provide ability to relate them if some one need

If you want them to match you pass empty .sql files in your schema update directory. The long and short is that an extension's SQL schema version is not directly related to an extension's version and an effort to mandate they are to me is restricting that aspect of the API.

To me there should only be two options on the "apply schema updates" API. Either apply a single update or apply all updates newer than the current version (this is a pretty standard convention too). If you're trying to apply a filter that says "don't apply diffs newer than X version", to me that smells like a packaging or versioning error.

avatar alex7r
alex7r - comment - 2 Jun 2016

If you want them to match you pass empty .sql files in your schema update directory.

Sorry, but I don't really understand what this is about. I mean why you want to add empty file to package?

If you're trying to apply a filter that says "don't apply diffs newer than X version", to me that smells like a packaging or versioning error

or site owner wants to rollback without uninstalling component

Also as "Joomla! member" can you provide some link or explain even right here:
How to set up 2 spaces in PHPDoc? PhpStorm doesn't allowing to do so, and all instructions on docs.joomla.org is useless when it comes to this.

avatar mbabker
mbabker - comment - 2 Jun 2016

That part of PhpStorm's code style is still buggy, I've never really gotten that to work. Suggest a bug report with them.

or site owner wants to rollback without uninstalling component

You'd need custom code in your extension's install script to handle a rollback situation. The Joomla APIs aren't really built at all to cope with this scenario right now.

Sorry, but I don't really understand what this is about. I mean why you want to add empty file to package?

Right now you can manage an extension version and a database schema version somewhat independently. A new extension version does not always include database schema changes, in which case the schema version would not be the same as the extension version. If you want to force this, you either have to add an empty file to the update folder or even worse manually update the #__schemas table to change the version for your extension. Honestly though, unless you're displaying an extension's schema version in your UI, you really don't need to know or care about the schema version.

avatar alex7r
alex7r - comment - 2 Jun 2016

That part of PhpStorm's code style is still buggy, I've never really gotten that to work. Suggest a bug report with them.

So why don't you make life of developers easier? Why don't use standard formatting? Why 2 or 10 000 spaces?
And what you suggest to format files? To do this manually?

You'd need custom code in your extension's install script to handle a rollback situation. The Joomla APIs aren't really built at all to cope with this scenario right now.

really not all the time.

If you want to force this, you either have to add an empty file to the update folder or even worse manually update

So you really want to flood instead of adding update from to range in update of extension?
It will not prevent from running all queries if this is needed, but not automatically on install.

Here scenario were is needed:
Client gets component.
Damages DB or because of development scenarios changed over few years something goes wrong and his DB is not up to date. Client is a simple web-admin and know nothing about coding or files of components. But knows all about security and backups. But his live site and site from backup DB is different. He provides you with backup and need you to give him page so he can fix live site so he can update further with no troubles.
What would you do? DB is not equal, so you can't really create needed sql.
the only way you can fix something is to implement DB fix interface, like joomla does.
You do this.
But everything is senseless because he already tried to update to newest version, he has newest sql files in his live site folder. And each time he attempts to install version that will help him he gets sql errors. Errors that he need to deal with by installing this package. But he can't install it because he get those errors. Recursion.
And what if we have top_version? we can provide package with build that would not trigger sql errors and will sole DB mismatch.

avatar mbabker
mbabker - comment - 2 Jun 2016

So why don't you make life of developers easier? Why don't use standard formatting? Why 2 or 10 000 spaces?

Joomla's codestyle rules pre-date PSR-2 or PhpStorm becoming a top player in the IDE marketspace. It's not Joomla's responsibility to cater to market trends just because it's the cool thing to do.

And what you suggest to format files? To do this manually?

For doc blocks, yes, because AFAIK no IDE's formatters 100% align to Joomla's codestyle rules.

Frankly the scenario you've presented wouldn't work correctly without providing a custom SQL diff specific to that installation, it's not something the API can automagically handle because Joomla doesn't have a real schema tracking/detection/management API. So at that point it's a custom job anyway and any installation is going to get that user out of sync with the source distribution.

In any case what you're suggesting IMO is not something that the Joomla API needs to be able to do. If you need to parse a single file, use the existing database API to load it up and execute it. If you need to run migrations for an extension, use the existing API to do it.

All I see here is the creation of new rules regarding extension database schemas and their co-relation to their extensions, namely rules regarding what the expected database schema version must be and the format it must be in. Also, by specifying a "top" version, this means that any existing core APIs cannot be relied on for validating a database schema is in the expected state. Going back to this example:

Update extension from version 3.1.020 to 3.1.024
With 
3.1.019
3.1.021
3.1.024
3.1.025

With your "top" version argument, the 021 and 024 updates would be applied but all existing API tools would tell you that your schema is out of date because 025 is not applied.

This is not a fully implemented "idea" and IMO has more issues than pros for inclusion in the core platform.

avatar alex7r
alex7r - comment - 2 Jun 2016

Joomla's codestyle rules pre-date PSR-2 or PhpStorm becoming a top player in the IDE marketspace. It's not Joomla's responsibility to cater to market trends just because it's the cool thing to do.

And that's how you planning on involving new developers to commit and spread Joomla!? You better be kidding me... Joomla real thirsty, young blood developers blood thirsty.
You know it will be good meme, or smth like:

You'll love Joomla it gives you so much space for extension of it's core
You'll hate Joomla it gives you 2 much "space" for committing to it's core

Seriously - 2 or 10 000... to cater to market trends... Just remove those lines from build check and so much developers love you'll get. Nobody ask you to rebuild your core to be in this kind of trend. Seriously, think about it. Or think about forgiving about Joomla at all if 3 years trend is nothing for you. Don't use bootstrap and jquery, stick with mootools and table templates as it's not Joomla's responsibility.

Frankly the scenario you've presented wouldn't work correctly without providing a custom SQL diff specific to that installation, it's not something the API can automagically handle because Joomla doesn't have a real schema tracking/detection/management API

Well you need to do some job definitely, but there is solution without custom sql. To preserve issues from happening you may create 0.0.001.sql and it will be applied only but "db fix" button.
And it need to contain all tables in presented like: table creating + each column creating.

The one good question that you added is

With your "top" version argument, the 021 and 024 updates would be applied but all existing API tools would tell you that your schema is out of date because 025 is not applied.

But really to use API in your extension you need to call it right? In one extension I use this workaround - I call API one way. In other I don't - other way. And so there is issues.
What better implement 2 ways by adding variable, or only one way of implementation? That's not flexible.
And rhetorical question: did you seen many extensions using it? I mean I strongly believe this problem was not previously mentioned because there ain't much people using this API.

Also: "Oh this solution need to more changes in other file to be good... Well let's reject it and do nothing, as it needs some pointing to some additional work." - is not long ride solution.

avatar mbabker
mbabker - comment - 2 Jun 2016

So if you seriously believe that a project should not have a consistent code style ruleset or should change those rules to follow whatever new trends are on the market feel free to propose a pull request and see what happens. It's there for a reason. Newsflash, Joomla's codestyle rules date back to 2010-2011-ish and in 2016 there are still codestyle violations in the code base. Do you really think anyone's going to go through the effort to reformat all code files to follow a new standard? They are all consistently formatted for a reason, for better or worse, and a major change is a lot of work for little gain.

I suppose Joomla should get ahead of the power curve and implement PSR-12 to keep developers coming in by the dozen, because code style rules are what make or break contributions.

And rhetorical question: did you seen many extensions using it?

I actually can't believe core uses such a busted API, but alas, architecture seems to be nothing more than a joke in the Joomlaverse.

avatar brianteeman brianteeman - change - 10 Jun 2016
Status Pending Needs Review
avatar brianteeman
brianteeman - comment - 10 Jun 2016

Setting Needs Review so the CMS Maintainers can make a decision


This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/10711.

avatar wojsmol
wojsmol - comment - 19 Jun 2016

@alex7r Please see alex7r#2

avatar rdeutz
rdeutz - comment - 6 Sep 2016

We are not going to change something in this area, I think it is good as it is. Can't really find a good argument for a change.

Thanks all for working on this.

avatar rdeutz rdeutz - change - 6 Sep 2016
Status Needs Review Closed
Closed_Date 0000-00-00 00:00:00 2016-09-06 16:05:53
Closed_By rdeutz
avatar rdeutz rdeutz - close - 6 Sep 2016
avatar joomla-cms-bot joomla-cms-bot - change - 6 Sep 2016
Category Installation SQL Libraries
avatar alex7r
alex7r - comment - 18 Oct 2016

Hi everyone,

@rdeutz can you revise this issue once again as I found one more reason to apply this patch:

If something that is not suppose to made to the package - it's dev problem, builders fault and so on...

But as soon as this package gets to end-user you're dead stuck. Any changes to packages are ineffective as user already have those unneeded files at their server, so no matter if you fixed your package - end-user can't install it.

Add a Comment

Login with GitHub to post a comment