Joomla! Issue Tracker - CMS

Download
Launch
Feeling Lucky

[#121] - Finder / Smart Search Index Hangs on Large Content Sites

?
avatar cspratt
cspratt
5 Feb 2012

I have a #_content table of just about 3Gigs, 30k+ Articles and 4k+ Categories. In earlier versions of Finder before it was integrated into the latest release of Joomla 2.5 Stable I had the same problem I'm having now. It hangs and sometimes crashes my server requiring a Hard Reboot. I think this is due to having specific articles that are in pagebreak standards almost 400 book pages long. I have been building an online library using Joomla (which in retrospect may have been a bad idea, due to Joomla not really having the search functionality to deal with EXTREMELY large content based sites, and also that it uses MySQL which can be a huge bottleneck with large sites) and I have been letting the GUI Indexer run for about 9 hours now and it is hung up on 10% which was what happened earlier on in development when I tried this. I have used the CLI as well and let it run for 3 days and it never finished, and eventually shot up an error which ultimately nullified all the prior indexing.

I have submit this issue to the Joomla! Bug Squad on Google Groups, yet it seems to have never been approved by a Moderator :(

System Specs:
Rackspace Cloud Server
12 GIGS DDR3
PHP Built On Linux 2.6.18-238.19.1.el5xen #1 SMP Fri Jul 15 08:16:59 EDT 2011 x86_64
Database Version 5.5.15
Database Collation utf8_general_ci
PHP Version 5.3.8
Web Server Apache/2.2.3 (CentOS)
WebServer to PHP Interface cgi-fcgi
Joomla! Version Joomla! 2.5.1 Stable [ Ember ] 2-Feb-2012 14:00 GMT
Joomla! Platform Version Joomla Platform 11.4.0 Stable [ Brian Kernighan ] 03-Jan-2012 00:00 GMT

avatar cspratt cspratt - open - 5 Feb 2012
avatar cspratt
cspratt - comment - 5 Feb 2012

So I decided to resort to the CLI finder_indexer.php script and I get the following (I purged the Index from Joomla! Backend prior to running CLI script):

  • Processed batch 2 in 8.225 seconds.
  • Processed batch 3 in 9.309 seconds.
  • Processed batch 4 in 15.008 seconds.
  • Processed batch 5 in 10.954 seconds.
  • Processed batch 6 in 8.943 seconds.
  • Processed batch 7 in 11.124 seconds.
  • Processed batch 8 in 11.733 seconds.
  • Processed batch 9 in 11.457 seconds.
  • Processed batch 10 in 9.973 seconds.
    PHP Fatal error: Uncaught exception 'Exception' with message 'Duplicate entry '3235-93612' for key 'PRIMARY' SQL=INSERT INTO d0y2o_finder_links_terms0 (link_id, term_id, weight) SELECT 3235, term_id, ROUND(SUM(context_weight), 8) FROM d0y2o_finder_tokens_aggregate WHERE map_suffix = '0' GROUP BY term ORDER BY term DESC' in /var/www/clients/client0/web7/web/administrator/components/com_finder/helpers/indexer/indexer.php:762
    Stack trace:
    #0 /var/www/clients/client0/web7/web/plugins/finder/content/content.php(317): FinderIndexer::index(Object(FinderIndexerR esult))
    #1 /var/www/clients/client0/web7/web/administrator/components/com_finder/helpers/indexer/adapter.php(242): plgFinderCont ent->index(Object(FinderIndexerResult))
    #2 [internal function]: FinderIndexerAdapter->onBuildIndex()
    #3 /var/www/clients/client0/web7/web/libraries/joomla/event/event.php(71): call_user_func_array(Array, Array)
    #4 /var/www/clients/client0/web7/web/libraries/joomla/event/dispatcher.php(161): JEvent->update(Array)
    #5 /var/www/clients in /var/www/clients/client0/web7/web/administrator/components/com_finder/helpers/indexer/indexer.php on line 762

Fatal error: Uncaught exception 'Exception' with message 'Duplicate entry '3235-93612' for key 'PRIMARY' SQL=INSERT INTO d0y2o_finder_links_terms0 (link_id, term_id, weight) SELECT 3235, term_id, ROUND(SUM(context_weight), 8) FROM d0y2o_finder_tokens_aggregate WHERE map_suffix = '0' GROUP BY term ORDER BY term DESC' in /var/www/clients/client0/web7/web/administrator/components/com_finder/helpers/indexer/indexer.php:762

Stack trace:

#0 /var/www/clients/client0/web7/web/plugins/finder/content/content.php(317): FinderIndexer::index(Object(FinderIndexerR
esult))

#1 /var/www/clients/client0/web7/web/administrator/components/com_finder/helpers/indexer/adapter.php(242): plgFinderCont
ent->index(Object(FinderIndexerResult))

#2 [internal function]: FinderIndexerAdapter->onBuildIndex()

#3 /var/www/clients/client0/web7/web/libraries/joomla/event/event.php(71): call_user_func_array(Array, Array)

#4 /var/www/clients/client0/web7/web/libraries/joomla/event/dispatcher.php(161): JEvent->update(Array)

#5 /var/www/clients in /var/www/clients/client0/web7/web/administrator/components/com_finder/helpers/indexer/indexer.php
on line 762

avatar zero-24 zero-24 - close - 5 May 2013
avatar nicksavov
nicksavov - comment - 5 May 2013

Thanks for posting this, Christopher! Can you send a pull request? If you need to learn how, check out this tutorial:
http://docs.joomla.org/Git_for_Coders

I also created a tracker item for this issue:
http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=30864

You can receive notifications from it by signing in, going to the item, and clicking "Monitor Item" in the left column.

While we’re transitioning to a new integrated tracker, the JoomlaCode tracker is where we do most of Joomla's bug fixing. You can find out more about the process at:
http://docs.joomla.org/Filing_bugs_and_issues

I'm going to close this item, since we have the above JoomlaCode Tracker Item. If and when you submit a pull request, please reference the JoomlaCode Tracker Item URL:
http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=30864

Let me know if you have any questions and I'll be glad to answer. Thanks again, Christopher!

avatar zero-24 zero-24 - change - 7 Jul 2015
Labels Added: ?
Removed: ?
Build ⇒ staging

Add a Comment

Login with GitHub to post a comment