Indexing for Smart Search with CLI with command:
php -d memory_limit=512M joomla.php finder:index -vvv
I obtain an error: Data too long for column 'route' at row 1
All indexed correctly without errors.
If alias is big, near to 400 characters: I obtain the error.
It is because the route field, in finder_links table, has a 400 chars limit.
Labels |
Removed:
?
|
Labels |
Added:
No Code Attached Yet
|
Labels |
Added:
bug
|
The alias is generated automatically from the title
BUT if you have set unicode alias to no then the alias can be longer than the title
This was unified at one point and for some reason it was decided on 400 chars. I changed the last few columns over to 400 chars when I stumbled upon this. But there is no real reason for this. This is also a general issue. However, the alias is not meant to be a short story, so 400 chars is pretty excessive, even if we use unicode...
I ran into this issue "Error: Data too long for column 'alias' at row 1" from a slightly different angle.
I have a "custom field" (textarea) which I like to be indexed as well. Indexing alone seems to work, but when I activate the option (in the custom field's configuration) to "index and add to taxonomy", then I seem to run into the mentioned error:
"Error: Data too long for column 'alias' at row 1"
The "value" of that textfield (it is connected to the articles) can indeed be longer than 400 chars. However, custom fields do not have an alias, and creating the alias from the field title for indexing does not make a whole lot of sense (since the title is the same for all instances of that field).
So my guess is, the alias for the search taxonomy is rendered from the "value" of the custom field - and that can (at least for the field type "textarea") easily exceed 400 chars.
If that is the case, then maybe a different approach for the alias of custom fields is needed...
also: some additional information in the error message would be helpful in identifying which entry is actually causing the error.
Why do you set this to "index and add to taxonomy" when it is over 400 chars long? This would result in a select list of gigantic entries with all the text of the field. I agree in a way that we should catch the errors when indexing. My solution would be to truncate the data properly when adding the taxonomy to the database. Would you be willing to create a PR for that?
I seem to be unable to create a PR for this, I am getting the following error message:
Pull request creation failed. Validation failed: must be a collaborator
However I do have a fix prepared for this issue...
@schultz-it-solutions If the changes are minimal, please post code changes here and I can do the pull request. Thanks.
yeah, it is actually only a few lines of code:
in [JOOMLAROOT]/administrator/components/com_finder/indexer/Helper.php
in method "addCustomFields"
// We want to add this field as a taxonomy; but only to a maximum of 40 words, less than 255 characters
if (
($searchindex == self::CUSTOMFIELDS_ADD_TO_TAXONOMY || $searchindex == self::CUSTOMFIELDS_ADD_TO_BOTH)
&& $field->value
) {
$fieldValue = $field->value;
if (strlen($fieldValue) > 255) {
$fieldValue = implode(' ',array_slice(array_filter(explode(' ', $fieldValue)), 0, 40));
}
$fieldTitle = $field->title;
if (strlen($fieldTitle) > 255) {
$fieldTitle = implode(' ',array_slice(array_filter(explode(' ', $fieldTitle)), 0, 40));
}
$item->addTaxonomy($fieldTitle, $fieldValue, $field->state, $field->access, $field->language);
}
After encountering the issue described in #44210 today more specifically, I tried the code above, but it didn't directly resolve the issue with the title value in the database in my case.
In the case of the title for taxonomy, it's a 255 character cap. In my case it's the field type "List" with ability to select multiple options that has come into play.
Some further investigation shows that when the indexer is going through the custom fields, it's indexing the list of selected values for the custom field as a single combined value, instead of identifying the parts of the selection as individual taxonomy values.
In my case, the combined title for the taxonomy value ended up >255 characters due to the number of options that could be selected for one of the lists on the article (it's used as a resource catalogue, and so there's 7 fields with 3-10 tags in each field)
Would the solution be to identify the field type, and then if it's a list process it by looping through the values and triggering an addTaxonomy instance for each value in the array?
To test the scenario:
You will get a message
Save failed with the following error: Data too long for column 'title' at row 1
Additionally...
Also, the helper file in J5 is actually administrator/components/com_finder/src/Indexer/Helper.php
@Hackwar why we have the alias made to 400 chars?