Joomla! Issue Tracker | Joomla! CMS #27807 - "Smart Search

Information Required ?

Closed
5 Jul 2020
Medium
Build: Joomla! 3.9.15 Stable
# 27807

webtimgist
5 Feb 2020

Steps to reproduce issue

Create an article with 30 pages of content, a very large article.

Expected result

File would save without issue

Actual result

When I save I get the following error:

Error -

Article save failed with the following error: SQLSTATE[HY000]: General error: 1114 The table 'lny6d_finder_tokens' is full

I first submitted a ticket to Siteground.com and the tech that worked with me came to the same conclusion I did, so the ticket got bumped upstairs.

The higher level tech that took over came to the conclusion that a search plugin was causing the problem.

I went through the list of search plugins disabling them one at a time and determined that the "Smart Search - Content" plugin was the offending plugin.

System information (as much as possible)

PHP Built On Linux giow1036.siteground.us 3.12.18-clouder0 #1 SMP Tue Oct 9 18:14:47 EEST 2018 x86_64
Database Type mysql
Database Version 5.7.25-28-log
Database Collation utf8mb4_general_ci
Database Connection Collation utf8mb4_general_ci
PHP Version 7.2.27
Web Server Apache
WebServer to PHP Interface cgi-fcgi
Joomla! Version Joomla! 3.9.15 Stable [ Amani ] 27-January-2020 15:00 GMT
Joomla! Platform Version Joomla Platform 13.1.0 Stable [ Curiosity ] 24-Apr-2013 00:00 GMT
User Agent Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0

Forum Post Assistant (v1.4.9 (lambrusca) : 4th February 2020 wrote:

    Problem Description :: wrote:
    SQL Error 1114 

    Log/Error Message :: wrote:
    N?A

    Last PHP Error(s) Reported :: wrote:
    [03-Feb-2020 23:29:13 CST6CDT] PHP Deprecated: Function create_function() is deprecated in /home/customer/www/boston911truth-dev.org/public_html/media/widgetkit/widgets/lightbox/lightbox.php on line 49

    Actions Taken To Resolve wrote:
    Disable Content Smart Search

    Basic Environment :: wrote:
    Joomla! Instance :: Joomla! 3.9.15-Stable (Amani) 27-January-2020
    Joomla! Platform :: Joomla Platform 13.1.0-Stable (Curiosity) 24-Apr-2013
    Joomla! Configured :: Yes | Read-Only (444) | Owner: --protected-- . (uid: 1/gid: 1) | Group: --protected-- (gid: 1) | Valid For: 3.9
    Configuration Options :: Offline: true | SEF: true | SEF Suffix: false | SEF ReWrite: false | .htaccess/web.config: Yes | GZip: false | Cache: false | CacheTime: 15 | CacheHandler: file | CachePlatformPrefix: false | FTP Layer: false | Proxy: false | LiveSite: | Session lifetime: 60 | Session handler: database | Shared sessions: false | SSL: 0 | Error Reporting: default | Site Debug: false | Language Debug: false | Default Access: 1 | Unicode Slugs: false | dbConnection Type: pdomysql | PHP Supports J! 3.9.15: Yes | Database Supports J! 3.9.15: Unknown | Database Credentials Present: Yes |

    Host Configuration :: OS: Linux | OS Version: 3.12.18-clouder0 | Technology: x86_64 | Web Server: Apache | Encoding: | Doc Root: --protected-- | System TMP Writable: Yes | Free Disk Space : 1081.06 GiB |

    PHP Configuration :: Version: 7.2.27 | PHP API: cgi-fcgi | Session Path Writable: Yes | Display Errors: | Error Reporting: 30709 | Log Errors To: php_errorlog | Last Known Error: 03rd February 2020 23:29:13. | Register Globals: | Magic Quotes: | Safe Mode: | Allow url fopen: 1 | Open Base: | Uploads: 1 | Max. Upload Size: 128M | Max. POST Size: 128M | Max. Input Time: 120 | Max. Execution Time: 120 | Memory Limit: 768M

    Database Configuration :: Connection Error: : Database Credentials Present? in Configuration...

    Detailed Environment :: wrote:
    PHP Extensions :: Core (7.2.27) | date (7.2.27) | libxml (7.2.27) | openssl (7.2.27) | pcre (7.2.27) | sqlite3 (7.2.27) | zlib (7.2.27) | bcmath (7.2.27) | bz2 (7.2.27) | calendar (7.2.27) | ctype (7.2.27) | curl (7.2.27) | dba (7.2.27) | dom (20031129) | enchant (7.2.27) | hash (1.0) | fileinfo (1.0.5) | filter (7.2.27) | ftp (7.2.27) | gd (7.2.27) | gettext (7.2.27) | gmp (7.2.27) | SPL (7.2.27) | iconv (7.2.27) | session (7.2.27) | intl (1.1.0) | json (1.6.0) | ldap (7.2.27) | mbstring (7.2.27) | standard (7.2.27) | mysqlnd (mysqlnd 5.0.12-dev - 20150407 - $Id: 3591daad22de08524295e1bd073aceeff11e6579 $) | pcntl (7.2.27) | mysqli (7.2.27) | PDO (7.2.27) | pdo_mysql (7.2.27) | pdo_pgsql (7.2.27) | pdo_sqlite (7.2.27) | pgsql (7.2.27) | Phar (2.0.2) | posix (7.2.27) | pspell (7.2.27) | readline (7.2.27) | Reflection (7.2.27) | imap (7.2.27) | shmop (7.2.27) | SimpleXML (7.2.27) | soap (7.2.27) | sockets (7.2.27) | sodium (7.2.27) | exif (7.2.27) | sysvmsg (7.2.27) | sysvsem (7.2.27) | tidy (7.2.27) | tokenizer (7.2.27) | wddx (7.2.27) | xml (7.2.27) | xmlreader (7.2.27) | xmlrpc (7.2.27) | xmlwriter (7.2.27) | xsl (7.2.27) | zip (1.15.4) | cgi-fcgi () | memcached (3.0.4) | ionCube Loader (10.3.9) | Zend OPcache (7.2.27) | Zend Engine (3.2.0) |
    Potential Missing Extensions ::

    Switch User Environment (Experimental) :: PHP CGI: Yes | Server SU: Yes | PHP SU: Yes | Custom SU (LiteSpeed/Cloud/Grid): Yes
    Potential Ownership Issues: No 

    Folder Permissions :: wrote:
    Core Folders :: images/ (755) | components/ (755) | modules/ (755) | plugins/ (755) | language/ (755) | templates/ (755) | cache/ (755) | logs/ (755) | tmp/ (755) | administrator/components/ (755) | administrator/modules/ (755) | administrator/language/ (755) | administrator/templates/ (755) | administrator/logs/ (755) |

    Elevated Permissions (First 10) :: 

    Extensions Discovered :: wrote:
    Components :: SITE ::
    Core :: com_wrapper (3.0.0) ? | com_mailto (3.0.0) ? |
    3rd Party:: kunena_tmpl_crypsis (5.1.15) ? | kunena_tmpl_crypsisb3 (5.1.15) ? | WF_FONTSELECT_TITLE (2.8.5) ? | WF_TABLE_TITLE (2.8.5) ? | WF_EMOTIONS_TITLE (2.8.5) ? | WF_MEDIAMANAGER_TITLE (2.8.5) ? | WF_AUTOSAVE_TITLE (2.8.5) ? | WF_ARTICLE_TITLE (2.8.5) ? | WF_PREVIEW_TITLE (2.8.5) ? | WF_PRINT_TITLE (2.8.5) ? | WF_FULLSCREEN_TITLE (2.8.5) ? | WF_FILEMANAGER_TITLE (2.8.5) ? | WF_TEXTPATTERN_TITLE (2.8.5) ? | WF_LAYER_TITLE (2.8.3) ? | WF_DIRECTIONALITY_TITLE (2.8.5) ? | WF_SPELLCHECKER_TITLE (2.8.5) ? | WF_KITCHENSINK_TITLE (2.8.5) ? | WF_IMGMANAGER_TITLE (2.8.5) ? | WF_TEXTCASE_TITLE (2.8.5) ? | WF_TEMPLATEMANAGER_TITLE (2.8.5) ? | WF_MEDIA_TITLE (2.8.5) ? | WF_CLIPBOARD_TITLE (2.8.5) ? | WF_CONTEXTMENU_TITLE (2.8.5) ? | WF_VISUALCHARS_TITLE (2.8.5) ? | WF_ANCHOR_TITLE (2.8.5) ? | WF_STYLE_TITLE (2.8.5) ? | WF_NONBREAKING_TITLE (2.8.5) ? | WF_FONTCOLOR_TITLE (2.8.5) ? | WF_STYLESELECT_TITLE (2.8.5) ? | WF_IFRAME_TITLE (2.8.5) ? | WF_IMGMANAGER_EXT_TITLE (2.8.5) ? | JCE - Noneditable (1.0.0) ? | WF_WORDCOUNT_TITLE (2.8.5) ? | WF_FORMATSELECT_TITLE (2.8.5) ? | JCE - Columns (1.0.0) ? | WF_HR_TITLE (2.8.5) ? | WF_LISTS_TITLE (2.8.5) ? | WF_XHTMLXTRAS_TITLE (2.8.5) ? | WF_FONTSIZESELECT_TITLE (2.8.5) ? | WF_BROWSER_TITLE (2.8.5) ? | WF_CHARMAP_TITLE (2.8.5) ? | WF_CLEANUP_TITLE (2.8.5) ? | WF_CAPTION_TITLE (2.8.5) ? | WF_SEARCHREPLACE_TITLE (2.8.5) ? | WF_MICRODATA_TITLE (2.8.5) ? | WF_SOURCE_TITLE (2.8.5) ? | WF_LINK_TITLE (2.8.5) ? | WF_VISUALBLOCKS_TITLE (2.8.5) ? | WF_LINKS_JOOMLALINKS_TITLE (2.8.5) ? | WF_POPUPS_JCEMEDIABOX_TITLE (2.8.5) ? | WF_LINK_SEARCH_TITLE (2.8.5) ? | WF_FILESYSTEM_JOOMLA_TITLE (2.8.5) ? | WF_AGGREGATOR_VIMEO_TITLE (2.8.5) ? | WF_AGGREGATOR_DAILYMOTION_TITLE (2.8.5) ? | WF_AGGREGATOR_VIDEO_TITLE (2.8.5) ? | WF_AGGREGATOR_AUDIO_TITLE (2.8.5) ? | WF_AGGREGATOR_[youtube]_TITLE (2.8.5) ? |

    Components :: ADMIN ::
    Core :: com_postinstall (3.2.0) ? | com_redirect (3.0.0) ? | com_admin (3.0.0) ? | com_languages (3.0.0) ? | com_search (3.0.0) ? | com_checkin (3.0.0) ? | com_contenthistory (3.2.0) ? | com_joomlaupdate (3.6.2) ? | com_associations (3.7.0) ? | com_media (3.0.0) ? | com_config (3.0.0) ? | com_users (3.0.0) ? | com_tags (3.1.0) ? | com_modules (3.0.0) ? | com_templates (3.0.0) ? | com_content (3.0.0) ? | com_finder (3.0.0) ? | com_installer (3.0.0) ? | com_weblinks (3.0.0) ? | com_banners (3.0.0) ? | com_menus (3.0.0) ? | com_privacy (3.9.0) ? | com_plugins (3.0.0) ? | com_cpanel (3.0.0) ? | com_fields (3.7.0) ? | com_actionlogs (3.9.0) ? | com_messages (3.0.0) ? | com_login (3.0.0) ? | com_newsfeeds (3.0.0) ? | com_cache (3.0.0) ? | com_ajax (3.2.0) ? | com_categories (3.0.0) ? |
    3rd Party:: Akeeba (7.0.1) ? | plg_kunena_altauserpoints (5.1.15) ? | plg_kunena_easysocial (5.1.15) ? | plg_kunena_gravatar (5.1.15) ? | plg_kunena_finder (5.1.15) ? | plg_finder_kunena (5.1.15) ? | PLG_KUNENA_EASYPROFILE (5.1.15) ? | plg_kunena_joomla (5.1.15) ? | PLG_KUNENA_EASYBLOG (0.0.1) ? | plg_kunena_uddeim (5.1.15) ? | plg_kunena_kunena (5.1.15) ? | plg_kunena_community (5.1.15) ? | plg_kunena_comprofiler (5.1.15) ? | com_kunena (5.1.15) ? | Widgetkit (1.5.8) ? | COM_JCE (2.8.5) ? | Blank Component (3.0.0) ? | COM_JEVENTS (3.4.50) ? |

    Modules :: SITE ::
    Core :: mod_banners (3.0.0) ? | mod_wrapper (3.0.0) ? | mod_menu (3.0.0) ? | mod_login (3.0.0) ? | mod_languages (3.5.0) ? | mod_syndicate (3.0.0) ? | mod_whosonline (3.0.0) ? | mod_related_items (3.0.0) ? | mod_search (3.0.0) ? | mod_articles_category (3.0.0) ? | mod_random_image (3.0.0) ? | mod_stats (3.0.0) ? | mod_feed (3.0.0) ? | mod_articles_archive (3.0.0) ? | mod_custom (3.0.0) ? | mod_articles_latest (3.0.0) ? | mod_users_latest (3.0.0) ? | mod_breadcrumbs (3.0.0) ? | mod_articles_categories (3.0.0) ? | mod_tags_similar (3.1.0) ? | mod_articles_popular (3.0.0) ? | mod_weblinks (3.0.0) ? | mod_finder (3.0.0) ? | mod_articles_news (3.0.0) ? | mod_footer (3.0.0) ? | mod_tags_popular (3.1.0) ? |
    3rd Party:: MOD_JEV_CUSTOM_MODULE_TITLE (3.4.50) ? | Widgetkit (1.0.0) ? | MOD_JEV_SWITCH_VIEW_TITLE (3.4.50) ? | MOD_JEV_LATEST_EVENTS_TITLE (3.4.50) ? | MOD_JEV_FILTER_MODULE_TITLE (3.4.50) ? | MOD_JEV_LEGEND_TITLE (3.4.50) ? | MOD_JEV_CALENDAR_TITLE (3.4.50) ? | Widgetkit Twitter (1.0.0) ? |

    Modules :: ADMIN ::
    Core :: mod_popular (3.0.0) ? | mod_logged (3.0.0) ? | mod_menu (3.0.0) ? | mod_login (3.0.0) ? | mod_submenu (3.0.0) ? | mod_title (3.0.0) ? | mod_latestactions (3.9.0) ? | mod_stats_admin (3.0.0) ? | mod_feed (3.0.0) ? | mod_version (3.0.0) ? | mod_sampledata (3.8.0) ? | mod_custom (3.0.0) ? | mod_toolbar (3.0.0) ? | mod_status (3.0.0) ? | mod_quickicon (3.0.0) ? | mod_privacy_dashboard (3.9.0) ? | mod_latest (3.0.0) ? | mod_multilangstatus (3.0.0) ? |
    3rd Party::

    Libraries ::
    Core ::
    3rd Party:: file_fof30 (3.5.1) ? |

    Plugins ::
    Core :: PLG_INSTALLER_FOLDERINSTALLER (3.6.0) ? | PLG_INSTALLER_URLINSTALLER (3.6.0) ? | plg_installer_packageinstaller (3.6.0) ? | plg_content_emailcloak (3.0.0) ? | plg_content_joomla (3.0.0) ? | plg_content_confirmconsent (3.9.0) ? | plg_content_pagebreak (3.0.0) ? | plg_content_pagenavigation (3.0.0) ? | plg_content_loadmodule (3.0.0) ? | plg_content_geshi (2.5.0) ? | plg_content_fields (3.7.0) ? | plg_content_finder (3.0.0) ? | plg_content_vote (3.0.0) ? | plg_quickicon_joomlaupdate (3.0.0) ? | plg_quickicon_privacycheck (3.9.0) ? | plg_quickicon_phpversioncheck (3.7.0) ? | plg_quickicon_extensionupdate (3.0.0) ? | PLG_ACTIONLOG_JOOMLA (3.9.0) ? | plg_extension_joomla (3.0.0) ? | plg_privacy_content (3.9.0) ? | plg_privacy_message (3.9.0) ? | plg_privacy_actionlogs (3.9.0) ? | plg_privacy_user (3.9.0) ? | plg_privacy_consents (3.9.0) ? | plg_system_debug (3.0.0) ? | plg_system_logout (3.0.0) ? | plg_system_remember (3.0.0) ? | plg_system_languagecode (3.0.0) ? | plg_system_stats (3.5.0) ? | plg_system_highlight (3.0.0) ? | plg_system_updatenotification (3.5.0) ? | plg_system_redirect (3.0.0) ? | PLG_SYSTEM_ACTIONLOGS (3.9.0) ? | plg_system_privacyconsent (3.9.0) ? | plg_system_log (3.0.0) ? | plg_system_languagefilter (3.0.0) ? | plg_system_sef (3.0.0) ? | plg_system_cache (3.0.0) ? | plg_system_logrotation (3.9.0) ? | plg_system_p3p (3.0.0) ? | plg_system_sessiongc (3.8.6) ? | plg_system_fields (3.7.0) ? | plg_editors-xtd_readmore (3.0.0) ? | plg_editors-xtd_menu (3.7.0) ? | plg_editors-xtd_article (3.0.0) ? | plg_editors-xtd_pagebreak (3.0.0) ? | plg_editors-xtd_module (3.5.0) ? | plg_editors-xtd_image (3.0.0) ? | plg_editors-xtd_fields (3.7.0) ? | plg_captcha_recaptcha (3.4.0) ? | plg_captcha_recaptcha_invisible (3.8) ? | plg_search_weblinks (3.0.0) ? | plg_search_content (3.0.0) ? | plg_search_tags (3.0.0) ? | plg_search_newsfeeds (3.0.0) ? | plg_search_categories (3.0.0) ? | plg_search_contacts (3.0.0) ? | plg_user_contactcreator (3.0.0) ? | plg_user_profile (3.0.0) ? | plg_user_terms (3.9.0) ? | plg_user_joomla (3.0.0) ? | plg_twofactorauth_totp (3.2.0) ? | plg_twofactorauth_yubikey (3.2.0) ? | plg_fields_calendar (3.7.0) ? | plg_fields_radio (3.7.0) ? | plg_fields_sql (3.7.0) ? | plg_fields_text (3.7.0) ? | plg_fields_media (3.7.0) ? | plg_fields_checkboxes (3.7.0) ? | plg_fields_integer (3.7.0) ? | plg_fields_repeatable (3.9.0) ? | plg_fields_list (3.7.0) ? | plg_fields_usergrouplist (3.7.0) ? | plg_fields_color (3.7.0) ? | plg_fields_user (3.7.0) ? | plg_fields_editor (3.7.0) ? | plg_fields_textarea (3.7.0) ? | plg_fields_imagelist (3.7.0) ? | plg_fields_url (3.7.0) ? | plg_authentication_cookie (3.0.0) ? | plg_authentication_gmail (3.0.0) ? | plg_authentication_joomla (3.0.0) ? | plg_authentication_ldap (3.0.0) ? | plg_finder_weblinks (3.0.0) ? | plg_finder_content (3.0.0) ? | plg_finder_tags (3.0.0) ? | plg_finder_newsfeeds (3.0.0) ? | plg_finder_categories (3.0.0) ? | plg_finder_contacts (3.0.0) ? |
    3rd Party:: plg_installer_jeventsinstaller (3.4.50) ? | plg_installer_jce (2.8.5) ? | plg_editors_codemirror (5.40.0) ? | plg_editors_jce (2.8.5) ? | plg_editors_tinymce (4.5.11) ? | PLG_JEV_CORE_CONTENT_PLUGIN_TITLE (3.4.50) ? | plg_content_jce (2.8.5) ? | Content - Widgetkit (1.0.0) ? | plg_quickicon_jce (2.8.5) ? | plg_quickicon_kunena (5.1.15) ? | plg_quickicon_akeebabackup (7.0.1) ? | PLG_ACTIONLOG_AKEEBABACKUP (7.0.1) ? | plg_extension_jce (2.8.5) ? | PLG_SYSTEM_GWEJSON (3.4.50) ? | System - Widgetkit (1.0.0) ? | System - Widgetkit ZOO (3.1.0) ? | PLG_SYSTEM_AKEEBAUPDATECHECK (7.0.1) ? | plg_system_jcemediabox (2.0.13) ? | System - Widgetkit Joomla (1.0.0) ? | PLG_SYSTEM_BACKUPONUPDATE (7.0.1) ? | plg_system_jce (2.8.5) ? | plg_system_kunena (5.1.15) ? | PLG_JEV_SEARCH_TITLE (3.4.50) ? | plg_kunena_comprofiler (5.1.15) ? | plg_kunena_community (5.1.15) ? | plg_kunena_altauserpoints (5.1.15) ? | plg_kunena_joomla (5.1.15) ? | plg_kunena_uddeim (5.1.15) ? | plg_kunena_gravatar (5.1.15) ? | plg_kunena_easysocial (5.1.15) ? | plg_kunena_kunena (5.1.15) ? | PLG_KUNENA_EASYPROFILE (5.1.15) ? | plg_kunena_finder (5.1.15) ? | plg_fields_mediajce (2.8.5) ? | PLG_FINDER_JEVENTS (3.4.50) ? | plg_finder_kunena (5.1.15) ? | 

    Templates Discovered :: wrote:
    Templates :: SITE :: jp-ultimate (1.0.005) ? | beez3 (3.1.0) ? | protostar (1.0) ? |
    Templates :: ADMIN :: hathor (3.0.0) ? | isis (1.0) ? |

Additional comments

This is what virtual DBA had to say about the error:

[quote]It is possible to impose user-defined maximums by explicitly defining the variable innodb_data_file_path. For example setting it to a value of ibdata1:10M:autoextend:max:256M will limit the data in InnoDB tables to a total of 256MB. Removing the max:256MB term will eliminate the imposed maximum.

MySQL reports a “Table is full” error where, in most cases, the issue involves running out of disk space. By default, limits are not imposed on MySQL tables however there are relatively large maximums inherent to the database and those maximums have not been the issue in our experience. If you are seeing this error, first check the disk space on the partition to ensure that this is not the cause of the error. If disk space is not a concern, check the variable innodb_data_file_path to see if a maximum table size has been set explicitly.[/quote]

webtimgist - open - 5 Feb 2020

joomla-cms-bot - labeled - 5 Feb 2020

chrisdavenport - comment - 5 Feb 2020

The #__finder_tokens table is a MEMORY table which has a relatively low limit on size for obvious reasons. You're not running out of disk space, you're running out of memory. Normally, the indexer should detect when you are approaching the limit and switch to using a disk-based table instead (with a consequent performance hit), but it is not happening in your case, probably because you have a single very large article which is filling the table on its own.

If you go into Components -> Smart Search -> Options, then on the Index tab you should see a field called Memory Table Limit. The default is 30000 which is the number of rows that may be added to the MEMORY tables before they switch to disk tables automatically. If you reduce that number (try, say, 10000) then it should switch sooner and give you a better chance of being able to complete the indexing process.

If that doesn't work, try changing the engine used for the #__finder_tokens and #__finder_tokens_aggregate tables from MEMORY to MYISAM or InnoDB. Expect the indexing process to be slower.

There are some notes about this problem here: https://docs.joomla.org/Smart_Search_on_large_sites

Quy - change - 6 Feb 2020

Labels

Added: Information Required

Quy - labeled - 6 Feb 2020

webtimgist - comment - 6 Feb 2020

Thank you for your reply.

No lowering the threshold to 10000 didn't have any effect.

I will report this to Siteground.com and see if they can increase memory limit.

Thanks for the reference to the Smart Search documentations as well. I will pass this along to siteground.com

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/27807.}

infograf768 - change - 6 Feb 2020

Status

New

⇒

Confirmed

richard67 - change - 6 Feb 2020

Status

Confirmed

⇒

New

richard67 - change - 6 Feb 2020

Status

New

⇒

Information Required

richard67 - comment - 6 Feb 2020

@webtimgist Please report back what was the result, if they could increase it and if so, if it has helped. We leave the issue open for a while.

webtimgist - comment - 9 Feb 2020

Yes, as soon as I hear from them I will post that information here. One of the solution was to use Apache Solr instead of a script based search engine. Most hosts don't make that available, (shame), on shared hosting accounts, and comes at a premium if we have a VPS or dedicated server. If Joomla could tap into Solr, that would put this issue to rest.

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/27807.}

webtimgist - comment - 9 Feb 2020

This is what Siteground had to say:

Kuzman Stoyanov

Feb 8, 2020 02:09 PM
Hello Timothy,

I was able to recreate the error, but sadly did not find a solution for it. The current memory_limit value is 768M:

https://boston911truth-dev.org/i.php

This is a default value for all our Shared hosting plans. Note that it can not be increased on a shared hosting plan, its value is global and changing it would affect all sites hosted on the machine. Such changes could only be performed on our Cloud hosting servers.

It is advised to ask Joomla devs for assistance on the matter, as this case is out of the scope of the services we provide.

Best Regards,

Kuzman Stoyanov
Senior Technical Support

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/27807.}

Quy - comment - 10 Feb 2020

@chrisdavenport Would your PR #16621 help in this case?

chrisdavenport - comment - 10 Feb 2020

It might, but probably not. The problem is the very large article, which presumably has a large number of indexable terms in it. During the indexing process each term is added to a temporary MEMORY table which is held in the server's RAM. Since RAM is a limited resource there is, in MySQL at least, a configuration variable which limits how much RAM each MEMORY table is allowed to use before a disk full error is returned. This limit is given by max_heap_table_size. Now, Smart Search is aware that MEMORY tables can get full so it tries to check to see if the table is nearing its size limit. However, there's no way to determine how much memory a table is actually using, so instead it uses a limit on the number of rows in the table. By default this is set to 30,000 rows. If the indexer reaches this limit it automatically switches the table from being a MEMORY table to being a MyISAM table which uses disk instead of RAM and has a size limit that you're never likely to reach, but is very much slower.

So the problem that https://github.com/webtimgist is encountering is that his/her server is reaching max_heap_table_size before the indexer has reached memory_table_limit rows. Hence the advice to reduce memory_table_limit so that it reaches that limit sooner and switches to a disk table well before max_heap_table_size is reached.

To solve the problem we need to find out why that isn't happening. Since reducing memory_table_limit to 10,000 didn't fix the problem, I'm thinking that the indexer is encountering a batch of tokens that itself is larger than memory_table_limit. I've noticed before that there is a flaw in the tokenizeToDbShort method in that it only checks memory_table_limit after it has already attempted to add a batch of tokens to the table. If there are sufficient tokens in a single batch such that max_heap_table_size is exceeded then you would encounter a "disk full" error before reaching the point were memory_table_limit is checked. Since I've never come across an article that would trigger that bug, I haven't attempted to fix it before and it has always remained a theoretical issue for me.

If that is what is causing this particular problem, then doing the check before adding the tokens to the table should fix the issue. Moving the check in https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_finder/helpers/indexer/indexer.php on lines 491 - 495 to just after line 486 and changing $count to count($tokens) should do the job, but you might have to check that $tokens is an array (because it might be a FinderIndexerToken object according to the code).

Incidentally, #16621 is addressing a problem where you really are running out memory. At present tokenisations are cached in memory but there is no limit placed on the size of that cache. With enough tokens the cache can grow large enough for PHP to run out of memory. That PR applies a simple LRU algorithm to limit the size of the cache to something reasonable. If you encounter that problem you'd see an "out of memory" error rather than "disk full".

Phrixos-IT - comment - 10 Jun 2020

I have a workaround for this problem that partially alleviates it. It does not solve the problem with "too large" articles and means manual work. I am also looking for someone who can implement the necessary search. If the indexer gets out, new articles and old ones that are opened and closed are still included in the index. Opening a lot of articles manually is very time-consuming. It would therefore be a relief if you could only display the articles in the article view that are not in the index in order to open and close them. Can anyone implement this here?

_{This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/27807.}

Quy - comment - 5 Jul 2020

Pleas test PR #30008

Quy - change - 5 Jul 2020

Status	Information Required	⇒	Closed
Closed_Date	0000-00-00 00:00:00	⇒	2020-07-05 20:55:41
Closed_By		⇒	Quy

Quy - close - 5 Jul 2020

Add a Comment

Older
Newer

Joomla! Issue Tracker - CMS

[#27807] - "Smart Search - Content" Causes Error to be Thrown.

Steps to reproduce issue

Expected result

Actual result

System information (as much as possible)

Additional comments

Add a Comment