Language Change PR-staging RTC

Pending

User tests: Successful: Unsuccessful:

avatar frankmayer
frankmayer
7 Jan 2017

New issue:
With thousands of articles or K2 items with lots of different text each, the server might become unresponsive, mostly as a result of a lot of processing and IO/WAIT.

Summary of Changes

This patch implements a pausing mechanism that pauses for a defined or dynamically adjusted amount of time between batches, therefore giving the server a little time to catch up.

Testing Instructions

Note:
For all of the tests, you may break the processing, at any point, after the findings are confirmed. No need to run them completely on very large sets of data.

  1. Do not apply the patch, yet. Run the command with the --purge parameter to also clean up the database.

Note: You might or might not notice (on huge number of articles with a lot of different text) that the machine(server) might get unresponsive.

Even if you don't notice that, you will notice that the apache/php- and mysql-processes are each constantly fully consuming a CPU core. Which depending on the cores available (and the settings of mysql in core usage) might translate to 50% on a dual core, 25% on a quad core or 12,5% on an octa-core processor. While this alone might not be an issue, it does add up the the problem at hand.

You also might or might not notice (depending on the storage subsystem [hdd, ssd or hybrid]) that there is a lot of IO/WAIT during the task. It gets worse, if it is a very long running task. I have seen this resulting in an unresponsive site, during indexing. Where unresponsive can mean that the response could take from 5-10 seconds to a lot more, or a complete timeout.

  1. Now apply the patch and run the same command again (with the --purge argument).
    You should notice pauses being inserted between each batch, according to the time the finished batch took to process. The division to determine the pause length is rounded[processed-batch-running-time / divisor]. The default divisor is 5.

You should also notice that the site/admin interface is now a lot more responsive, as the machine has less IO/WAITS and less constantly fully consuming of CPU cores.

  1. Now run the same command again (with the --purge argument), and add --pause=division --divisor=3
    Notice that the pauses are correctly calculated using the new divisor.

  2. Now run the same command again (with the --purge argument), and replace the --pause=division --divisor=3 with --pause=3
    Notice that the pauses are constantly three seconds long.

Documentation Changes Required

Yes. The new arguments should be included in the documentation here: https://docs.joomla.org/Setting_up_automatic_Smart_Search_indexing

avatar frankmayer frankmayer - open - 7 Jan 2017
avatar frankmayer frankmayer - change - 7 Jan 2017
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 7 Jan 2017
Category CLI Language & Strings
avatar frankmayer frankmayer - change - 7 Jan 2017
The description was changed
avatar frankmayer frankmayer - edited - 7 Jan 2017
avatar frankmayer frankmayer - change - 7 Jan 2017
The description was changed
avatar frankmayer frankmayer - edited - 7 Jan 2017
avatar frankmayer frankmayer - change - 7 Jan 2017
The description was changed
avatar frankmayer frankmayer - edited - 7 Jan 2017
avatar frankmayer frankmayer - change - 7 Jan 2017
The description was changed
avatar frankmayer frankmayer - edited - 7 Jan 2017
avatar frankmayer frankmayer - change - 7 Jan 2017
The description was changed
avatar frankmayer frankmayer - edited - 7 Jan 2017
avatar rdeutz rdeutz - change - 27 May 2017
Labels Added: Language Change PR-staging
avatar frankmayer frankmayer - change - 1 Jun 2017
The description was changed
avatar frankmayer frankmayer - edited - 1 Jun 2017
avatar frankmayer frankmayer - change - 11 Jun 2017
Title
Implemented pausing mechanisms into cli/finder_indexer.php.
[Smart Search] Implemented pausing mechanisms into cli/finder_indexer.php.
avatar frankmayer frankmayer - edited - 11 Jun 2017
avatar frankmayer frankmayer - change - 11 Jun 2017
Title
Implemented pausing mechanisms into cli/finder_indexer.php.
[Smart Search] Implemented pausing mechanisms into cli/finder_indexer.php.
avatar Quy Quy - test_item - 18 May 2018 - Tested successfully
avatar Quy
Quy - comment - 18 May 2018

I have tested this item successfully on b7134e2


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/13502.

avatar Quy
Quy - comment - 19 May 2018

I have tested this item successfully on d6a4e1a

Thank you!


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/13502.

avatar Quy Quy - test_item - 19 May 2018 - Tested successfully
avatar franz-wohlkoenig franz-wohlkoenig - change - 11 Apr 2019
Category CLI Language & Strings CLI
avatar franz-wohlkoenig franz-wohlkoenig - change - 19 Apr 2019
Title
[Smart Search] Implemented pausing mechanisms into cli/finder_indexer.php.
Implemented pausing mechanisms into cli/finder_indexer.php.
avatar franz-wohlkoenig franz-wohlkoenig - edited - 19 Apr 2019
avatar joomla-cms-bot joomla-cms-bot - change - 19 Apr 2019
Category CLI CLI Language & Strings
avatar franz-wohlkoenig
franz-wohlkoenig - comment - 19 Jul 2019

@SharkyKZ can you please look at this PR if you may test it?

avatar SharkyKZ
SharkyKZ - comment - 19 Jul 2019

I think conflicts need to be solved first.

avatar franz-wohlkoenig
franz-wohlkoenig - comment - 19 Jul 2019

true. @frankmayer can you please resolve conflicting files?

avatar franz-wohlkoenig franz-wohlkoenig - change - 19 Jul 2019
Status Pending Information Required
avatar HLeithner
HLeithner - comment - 19 Jul 2019

It's a new feature so should go into J4 maybe @Hackwar can look at this?

avatar brianteeman
brianteeman - comment - 19 Jul 2019

It's fixing a bug

the server might become unresponsive, mostly as a result of a lot of processing and IO/WAIT.

avatar HLeithner
HLeithner - comment - 19 Jul 2019

you can reduce the load of the server if you use ionice and nice to reduce the priority of the process.

avatar frankmayer
frankmayer - comment - 19 Jul 2019

Wow... two and a half years later... 😄

@HLeithner yes, one could do that, too.
However, this PR gives the user more control than just setting a priority on the whole process and/or its IO.
Also, I am not sure if nice and ionice would indeed be much help, as the IO-heavy process(es) would be the one(s) of the database server, not the script itself.
And the issue would be with the massive read/writes of the database server for the script, which could stress the server and under certain loads making it not able to actually serve content in parallel to running the script, as I wrote in the PR-description. This PR introduces user-defined pauses to the process, so that whatever medium is used for the db, can catch up and also serve other purposes in between.

@franz-wohlkoenig should I go on and fix the conflicts, then?

avatar franz-wohlkoenig
franz-wohlkoenig - comment - 19 Jul 2019

Wow... two and a half years later... 😄

it takes time if you have less People volunteering.

should I go on and fix the conflicts, then?

If you fix the Conflicts @SharkyKZ can take a look. So "Yes".

avatar frankmayer
frankmayer - comment - 19 Jul 2019

Wow... two and a half years later... smile

it takes time if you have less People volunteering.

@franz-wohlkoenig I know, be sure there was no pun intended. I just noticed how long ago I created the PR

should I go on and fix the conflicts, then?

If you fix the Conflicts @SharkyKZ can take a look. So "Yes".

Ok, will do. Thanks

avatar Hackwar
Hackwar - comment - 19 Jul 2019

Thank you for this feature. I do consider it a new feature and not just a bugfix, and thus I would recommend to add this to 4.0 instead of 3.x. I would actually be interested if this issue persists with the new DB structure in 4.0.

And yes, some things take a VERY long time. sigh

avatar frankmayer
frankmayer - comment - 19 Jul 2019

@Hackwar Yes, it can be seen both ways. But it fixes a problem that is happening on the 3.x version. And since people will be on that version for some time to come, we might as well provide a better experience.

As for 4.0, I have not tested, and unfortunately I didn't have any time to check out all the great stuff you people have done with the new version. But if we have this on 3.x it shouldn't be too much of an effort, if needed, to port it to 4.0.

avatar frankmayer frankmayer - change - 19 Jul 2019
Labels Removed: J3 Issue
avatar frankmayer
frankmayer - comment - 19 Jul 2019

@franz-wohlkoenig resolved conflicts.
@SharkyKZ want to give it a try?
Thanks

avatar SharkyKZ
SharkyKZ - comment - 22 Jul 2019

I have tested this item successfully on 9b4cd19


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/13502.

avatar SharkyKZ SharkyKZ - test_item - 22 Jul 2019 - Tested successfully
avatar franz-wohlkoenig
franz-wohlkoenig - comment - 22 Jul 2019

@Quy can you please retest?

avatar Quy
Quy - comment - 22 Jul 2019

I have tested this item successfully on 9b4cd19


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/13502.

avatar Quy Quy - test_item - 22 Jul 2019 - Tested successfully
avatar Quy Quy - change - 22 Jul 2019
Status Information Required Ready to Commit
avatar Quy
Quy - comment - 22 Jul 2019

RTC


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/13502.

avatar HLeithner HLeithner - change - 17 Sep 2019
Status Ready to Commit Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2019-09-17 16:13:17
Closed_By HLeithner
Labels Added: RTC
avatar HLeithner HLeithner - close - 17 Sep 2019
avatar HLeithner HLeithner - merge - 17 Sep 2019
avatar HLeithner
HLeithner - comment - 17 Sep 2019

Thanks for this PR. The reason I'm merging this into 3.9 series is to protect servers overloading.

Add a Comment

Login with GitHub to post a comment