? Pending

User tests: Successful: Unsuccessful:

avatar csthomas
csthomas
14 Mar 2017

Pull Request for Issue #14590

Summary of Changes

Indexed column can not have more than 767 bytes for mysql database.

  • shrink index to first 191 chars for column context in #__fields, #__fields_groups.
  • shrink index to first 191 chars for column item_id in #__fields_values
  • add/sync unsigned to asset_id and group_id which should have that.
  • add missing label for indexes in #__fields_values

Testing Instructions

Installation should works as before.
It is enough to test installation without testing data.

How to emulate installation error.
Mysql <= 5.6 with old configuration has by default ROW_FORMAT=COMPACT.
To emulate that, you have to replace text in installation sql file from:
) ENGINE=InnoDB
to:
) ROW_FORMAT=COMPACT ENGINE=InnoDB

for all lines.

Expected result

No errors on mysql <=5.6 with old configuration.

Actual result

Error on installation.

Documentation Changes Required

None

avatar csthomas csthomas - open - 14 Mar 2017
avatar csthomas csthomas - change - 14 Mar 2017
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 14 Mar 2017
Category SQL Administration com_admin Postgresql MS SQL Installation
avatar csthomas
csthomas - comment - 14 Mar 2017

@laoneo Can you take a look and check whether context column is long enough?

avatar laoneo
laoneo - comment - 15 Mar 2017

I guess for most extensions it is enough, but as soon as you wont start to do more advanced things I guess the context can grow.
Under which circumstances does this happen, when the default ROW_FORMAT value of mysql is set to compact? Never saw that issue before, so wondering what is the cause of it.

avatar csthomas
csthomas - comment - 15 Mar 2017

https://dev.mysql.com/doc/refman/5.6/en/innodb-row-format-antelope.html

To preserve compatibility with prior versions of InnoDB, InnoDB tables created in MySQL 5.6 default to the COMPACT row format rather than the newer DYNAMIC row format.

avatar laoneo
laoneo - comment - 15 Mar 2017

So what do we do here then? We had this problem discussed already in another issue, but can't remember the number. Honestly I would not go less that 255 characters in the context column.

avatar csthomas
csthomas - comment - 15 Mar 2017

I can change it to max 191 characters, and it still be OK.
I have changed it to 50 because I found such column name in other table which has 50.

avatar csthomas
csthomas - comment - 15 Mar 2017

Why: 191* 4 (bytes utf8) = 764 < 767 bytes.

The simpler way may be to set column as LATIN1 or ASCII then there won't be any limit.

avatar laoneo
laoneo - comment - 15 Mar 2017

Would this be the only place where we have indexes on varchar(255) columns? Just wondering if we don't have that issue somewhere else.

avatar csthomas
csthomas - comment - 15 Mar 2017

For now yes, the similar is session_id in #__sessions. Others column use some limit in chars to fit.

avatar laoneo
laoneo - comment - 15 Mar 2017

What's the solution there?

avatar csthomas
csthomas - comment - 15 Mar 2017

In general I see 3 options:

  • leave 255 and change characters coding to 1 byte, like ASCII, latin1
  • leave 255 and set/force for table ROW_FORMAT=DYNAMIC (it may be non B/C for older mysql)
  • set max 191 chars as it is done in #__sessions table, session_id
avatar laoneo
laoneo - comment - 15 Mar 2017

Then I would go for option 1 if that is acceptable in the Joomla land.

avatar csthomas
csthomas - comment - 15 Mar 2017

OK, then can someone approve option 1. I would prefer ASCII as it is a subset of utf8.

avatar alikon
alikon - comment - 31 Mar 2017

The simpler way may be to set column as LATIN1 or ASCII then there won't be any limit.
for me simple is better
?

avatar wilsonge
wilsonge - comment - 31 Mar 2017

Move to 191. That's how we should be doing it like the alias in most extensions after 3.6

avatar franz-wohlkoenig franz-wohlkoenig - change - 1 Apr 2017
Status Pending Discussion
avatar laoneo
laoneo - comment - 1 Apr 2017

We are not talking about the alias, its the context field which should not get truncated under every circumstance.

avatar Bakual
Bakual - comment - 1 Apr 2017

I'm with George.
There is no reason to have a context length of 255 to begin with, 191 will be far more than enough and solves our issue without introducing inconsistencies into the system.
Very likely 50 would be enough as well. It is the length of the context field in #__associations (which is a similar, but not exactly the same context). I'm fine with 50 as well. Honestly if an extension has a context longer than that, then it likely did something worng.

avatar laoneo
laoneo - comment - 1 Apr 2017

When an extension will have dynamic contextes (similar to a CCK), then it will become very likely to get a context bigger than 50. If there is a way to leave it with 255, and fix the issue, then we should go that way.

avatar Bakual
Bakual - comment - 1 Apr 2017

When an extension will have dynamic contextes (similar to a CCK), then it will become very likely to get a context bigger than 50.

191 will be far than enough then. Besides, CCKs likely will not use com_fields anyway. They offer that functionality themself.

If there is a way to leave it with 255, and fix the issue, then we should go that way.

Changing charset to ascii would mean we have a single field which isn't utf8mb4. Doesn't sound like a good approach to me.

avatar joomla-cms-bot joomla-cms-bot - change - 1 Apr 2017
Category SQL Administration com_admin Postgresql MS SQL Installation SQL Administration com_admin Installation
avatar csthomas csthomas - change - 1 Apr 2017
Labels Added: ?
avatar csthomas csthomas - change - 1 Apr 2017
The description was changed
avatar csthomas csthomas - edited - 1 Apr 2017
avatar csthomas csthomas - change - 1 Apr 2017
The description was changed
avatar csthomas csthomas - edited - 1 Apr 2017
avatar csthomas csthomas - change - 1 Apr 2017
The description was changed
avatar csthomas csthomas - edited - 1 Apr 2017
avatar csthomas csthomas - change - 1 Apr 2017
The description was changed
avatar csthomas csthomas - edited - 1 Apr 2017
avatar csthomas
csthomas - comment - 1 Apr 2017

After all I took a decision to leave for column 255 chars and shrink index key to first 191 characters.

In postgresql I could not set only a few columns to different character set.
There is only one possibility for all varchar/text columns - UTF-8.

avatar csthomas
csthomas - comment - 1 Apr 2017

@laoneo It would be good to remove #__fields_categories table from sql files and related php code.

avatar laoneo
laoneo - comment - 2 Apr 2017

Why? This is needed.

avatar csthomas
csthomas - comment - 2 Apr 2017

I thought that #__fields_categories table is not used any more.

avatar laoneo
laoneo - comment - 2 Apr 2017

Its the mapping table between a field and a category.

avatar csthomas
csthomas - comment - 2 Apr 2017

Thanks for explanation, I have not know that.

avatar rdeutz
rdeutz - comment - 4 Apr 2017

Could we have some testers here, so that we can add it to the RC1, thanks

avatar rdeutz rdeutz - change - 6 Apr 2017
Status Discussion Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2017-04-06 17:35:22
Closed_By rdeutz
avatar rdeutz rdeutz - close - 6 Apr 2017
avatar rdeutz rdeutz - merge - 6 Apr 2017

Add a Comment

Login with GitHub to post a comment