? Success

User tests: Successful: Unsuccessful:

avatar Bakual
Bakual
27 Oct 2014

Issue

Install Joomla (current staging) on PostgreSQL without Sample Data.
Try to create a module. It will fail with an error message related to the assets id.

Reason

During installation, we insert data together with a fixed ID into various tables. In PostgreSQL you need to adjust the sequence number (= autoincrement value) after you manually inserted an ID.
We did that for some tables but not for all. The #__assets one was missing for example.

Solution

This PR adds the queries to set the next sequence value for those tables that have a serial ID and after we manually inserted IDs.
It also removes some unneeded nextval (which only increments the sequence number) and one setval (because there is no serial ID in that table).

Testing

Make sure you can create modules, categories and the like.

avatar Bakual Bakual - open - 27 Oct 2014
avatar jissues-bot jissues-bot - change - 27 Oct 2014
Labels Added: ?
avatar wilsonge
wilsonge - comment - 28 Oct 2014

test: When installing with no sample data this issue became apparent and the PR indeed fixed the issues.

avatar sovainfo
sovainfo - comment - 28 Oct 2014

Would prefer removing the id from fieldlist and valuelist, like they have done with most recent addition of table #__postinstall_messages. No need to fix a problem if you can make it so you don't have the problem!

avatar wilsonge
wilsonge - comment - 28 Oct 2014

@sovainfo if that fixes the issue then that definitely seems like the better fix!

avatar mbabker
mbabker - comment - 28 Oct 2014

The IDs are needed for any table where there is a reference between
objects. Assets, categories, extensions, updates, menu, and modules all
are in that boat on a new install without sample data. Add that in and
banners, contacts, content, etc. are affected. Possibly more tables too.

On Monday, October 27, 2014, sovainfo notifications@github.com wrote:

Would prefer removing the id from fieldlist and valuelist, like they have
done with most recent addition of table #__postinstall_messages. No need to
fix a problem if you can make it so you don't have the problem!


Reply to this email directly or view it on GitHub
#4942 (comment).

avatar sovainfo
sovainfo - comment - 28 Oct 2014

Ofcourse they are needed! The same ID's are going to be created in that way. That is even possible for those tables where they made the mistake of putting the id's in a range. The order that the rows are inserted determine the sequence number. The benefit is that you don't need to maintain the numbers.

avatar mbabker
mbabker - comment - 28 Oct 2014

Yes, we have to manually maintain the numbers. Since the sample data isn't installed by using the components or library APIs to insert the records, the SQL files have to emulate what they would do. In the case of categories, they have a asset_id which must match the id column of the #__assets table. Its asset's name is stored in the format of <extension>.<section>.<id> where <id> is the item's ID from its main table. Being able to get those references correct without manually sequencing items is a task that is much too complex IMO.

avatar sovainfo
sovainfo - comment - 28 Oct 2014

Just did it on #__assets, The content is the same! Nothing to do, the database does it!

avatar Bakual
Bakual - comment - 28 Oct 2014

I actually thought about this before submitting the PR :smile:

They are certainly needed in the #__assets table because we reference them in other queries. It could be done with but would make the other queries more complex since you need to look up the asset id first.

In other places they may indeed not be needed, I still opted to leave it as is for multiple reasons:

  • It makes the PR simple to test as it only impacts PostgreSQL installations. Removing the ids would have to be done in all installation and sample data SQL files.
  • It's very simple to review since it's only a few lines changed.
  • It makes it easier to target specific rows if we want to update them in a later release or with the sample data.

We can always revisit that later on, but it's not in the scope of this PR, which is to fix installation for PostgreSQL.

avatar brianteeman brianteeman - change - 28 Oct 2014
Category Postgresql
avatar sovainfo
sovainfo - comment - 28 Oct 2014

@Bakual Looks like you missed my post: Just did it on #__assets, The content is the same! Nothing to do, the database does it!

avatar wilsonge
wilsonge - comment - 28 Oct 2014

@sovainfo I think what @Bakual means is that you need the module id when you insert the data into the assets table (as an example) so you're still effectively hardcoding the id's in there. Except now that's just your best guess as to what the id is - because it's all being handled by postgres

avatar Bakual
Bakual - comment - 28 Oct 2014

Looks like you missed my post: Just did it on #__assets, The content is the same! Nothing to do, the database does it!

As long as the table is clean and the sequence is at zero and all queries are run in order, it will work, yes.
But the current queries also work, without any conditionals. It's just safer this way. Personally I would not rely on the autoincrement if I'm going to use that ID later on. It's also easier to adjust the queries if needed since you don't have to count the rows to get the IDs. You can just read them and know what it will be. Less likely to introduce bugs this way.

avatar sovainfo
sovainfo - comment - 28 Oct 2014

We should only be talking about the primary key, most of the time called id (another mistake!), with a sequence number generated upon insert. As mentioned several times now, the data inserted remains the same. It only relies on the database to create the numbers, just when you add them with Joomla.
All databases have statements to manipulate the sequence number, as you should know, that is suggested as a solution to the problem created by hardcoding! So, there is absolutely no reason to hard code them, even when using reserved ranges. Only for tables with too many gaps it would be easier to stick with hardcoding
.
As for the foreign keys, they could/should be calculated instead of hard coded. The only issue is that it makes the order of importing data relevant. You can't calculate them when the data they rely on is not populated yet. Hardcoding the id's makes the order irrelevant. I don't suggest changes to foreign keys, despite that they would be improvements. Rather solve the many problems of using SQL format by changing it to xml. This should be for both structure, data and demodata. Definitely not in scope for this PR.

avatar Bakual
Bakual - comment - 28 Oct 2014

As for the foreign keys, they could/should be calculated instead of hard coded. The only issue is that it makes the order of importing data relevant.

They could, but it would make the queries more complex and imho you don't gain anything by doing so.

Rather solve the many problems of using SQL format by changing it to xml. This should be for both structure, data and demodata. Definitely not in scope for this PR.

I agree that this would be a good improvement. If someone is going to create a PR for that, I'm sure it will be considered.

Meanwhile I set this to RTC since it will not break anything and will only improve PostgreSQL installation.

avatar Bakual Bakual - change - 28 Oct 2014
Labels Added: ?
avatar zero-24 zero-24 - change - 31 Oct 2014
Status Pending Ready to Commit
avatar zero-24
zero-24 - comment - 31 Oct 2014

moving also on JIssues to RTC

This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/4942.

avatar alikon alikon - test_item - 1 Nov 2014 - Tested successfully
avatar alikon
alikon - comment - 1 Nov 2014

@test success

This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/4942.

avatar infograf768 infograf768 - close - 3 Nov 2014
avatar infograf768 infograf768 - change - 3 Nov 2014
Status Ready to Commit Closed
Closed_Date 0000-00-00 00:00:00 2014-11-03 12:21:50

Add a Comment

Login with GitHub to post a comment