No Code Attached Yet bug
avatar pabloarias
pabloarias
27 May 2024

Steps to reproduce the issue

Indexing for Smart Search with CLI with command:

php -d memory_limit=512M joomla.php finder:index -vvv

I obtain an error: Data too long for column 'route' at row 1

Expected result

All indexed correctly without errors.

Actual result

If alias is big, near to 400 characters: I obtain the error.

It is because the route field, in finder_links table, has a 400 chars limit.

avatar pabloarias pabloarias - open - 27 May 2024
avatar pabloarias pabloarias - change - 27 May 2024
Labels Removed: ?
avatar joomla-cms-bot joomla-cms-bot - change - 27 May 2024
Labels Added: No Code Attached Yet
avatar joomla-cms-bot joomla-cms-bot - labeled - 27 May 2024
avatar rdeutz
rdeutz - comment - 1 Jun 2024

@Hackwar why we have the alias made to 400 chars?

avatar rdeutz rdeutz - change - 1 Jun 2024
Labels Added: bug
avatar rdeutz rdeutz - labeled - 1 Jun 2024
avatar brianteeman
brianteeman - comment - 1 Jun 2024

The alias is generated automatically from the title
BUT if you have set unicode alias to no then the alias can be longer than the title

avatar Hackwar
Hackwar - comment - 1 Jun 2024

This was unified at one point and for some reason it was decided on 400 chars. I changed the last few columns over to 400 chars when I stumbled upon this. But there is no real reason for this. This is also a general issue. However, the alias is not meant to be a short story, so 400 chars is pretty excessive, even if we use unicode...

avatar schultz-it-solutions
schultz-it-solutions - comment - 26 Jul 2024

I ran into this issue "Error: Data too long for column 'alias' at row 1" from a slightly different angle.
I have a "custom field" (textarea) which I like to be indexed as well. Indexing alone seems to work, but when I activate the option (in the custom field's configuration) to "index and add to taxonomy", then I seem to run into the mentioned error:
"Error: Data too long for column 'alias' at row 1"

The "value" of that textfield (it is connected to the articles) can indeed be longer than 400 chars. However, custom fields do not have an alias, and creating the alias from the field title for indexing does not make a whole lot of sense (since the title is the same for all instances of that field).

So my guess is, the alias for the search taxonomy is rendered from the "value" of the custom field - and that can (at least for the field type "textarea") easily exceed 400 chars.
If that is the case, then maybe a different approach for the alias of custom fields is needed...

also: some additional information in the error message would be helpful in identifying which entry is actually causing the error.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/43545.

avatar Hackwar
Hackwar - comment - 26 Jul 2024

Why do you set this to "index and add to taxonomy" when it is over 400 chars long? This would result in a select list of gigantic entries with all the text of the field. I agree in a way that we should catch the errors when indexing. My solution would be to truncate the data properly when adding the taxonomy to the database. Would you be willing to create a PR for that?

avatar schultz-it-solutions
schultz-it-solutions - comment - 29 Jul 2024

I seem to be unable to create a PR for this, I am getting the following error message:
Pull request creation failed. Validation failed: must be a collaborator

However I do have a fix prepared for this issue...

avatar Quy
Quy - comment - 31 Jul 2024

@schultz-it-solutions If the changes are minimal, please post code changes here and I can do the pull request. Thanks.

avatar schultz-it-solutions
schultz-it-solutions - comment - 31 Jul 2024

yeah, it is actually only a few lines of code:

in [JOOMLAROOT]/administrator/components/com_finder/indexer/Helper.php
in method "addCustomFields"

            // We want to add this field as a taxonomy; but only to a maximum of 40 words, less than 255 characters
            if (
                ($searchindex == self::CUSTOMFIELDS_ADD_TO_TAXONOMY || $searchindex == self::CUSTOMFIELDS_ADD_TO_BOTH)
                && $field->value
            ) {
				$fieldValue = $field->value;				
				if (strlen($fieldValue) > 255) {				
					$fieldValue = implode(' ',array_slice(array_filter(explode(' ', $fieldValue)), 0, 40));
				}
				$fieldTitle = 	$field->title;
				if (strlen($fieldTitle) > 255) {				
					$fieldTitle = implode(' ',array_slice(array_filter(explode(' ', $fieldTitle)), 0, 40));
				}
                $item->addTaxonomy($fieldTitle, $fieldValue, $field->state, $field->access, $field->language);
            }
avatar particthistle
particthistle - comment - 22 Oct 2024

After encountering the issue described in #44210 today more specifically, I tried the code above, but it didn't directly resolve the issue with the title value in the database in my case.

In the case of the title for taxonomy, it's a 255 character cap. In my case it's the field type "List" with ability to select multiple options that has come into play.

Some further investigation shows that when the indexer is going through the custom fields, it's indexing the list of selected values for the custom field as a single combined value, instead of identifying the parts of the selection as individual taxonomy values.

In my case, the combined title for the taxonomy value ended up >255 characters due to the number of options that could be selected for one of the lists on the article (it's used as a resource catalogue, and so there's 7 fields with 3-10 tags in each field)

Would the solution be to identify the field type, and then if it's a list process it by looping through the values and triggering an addTaxonomy instance for each value in the array?

To test the scenario:

  • Create a Custom Field of type "List"
  • Set it to multiple selections
  • Add a variety of lengthy phrases to select from as options
  • Create an article and select all the options for that list so that the total character count including the name of the field is 255 characters
  • Save the article.

You will get a message
Save failed with the following error: Data too long for column 'title' at row 1

Additionally...

  • Where the error comes up when you go to save an individual article, the error doesn't identify what the cause is. Adding something to indicate the issue is with Smart Search would dramatically help users.
  • It would be much easier to troubleshoot the issue when Smart Search Indexer runs by also displaying some extra information eg the ID of the article that is having an issue indexing... the site today I've been working on has 167 articles but only 6 had the new custom fields functionality to then discover what the issue was. Another site I have has 4000+ articles indexed with 7 different content filters... I'd really not want to troubleshoot that one to find the needle in a haystack!
avatar particthistle
particthistle - comment - 22 Oct 2024

Also, the helper file in J5 is actually administrator/components/com_finder/src/Indexer/Helper.php

Add a Comment

Login with GitHub to post a comment