User tests: Successful: Unsuccessful:
The last iteration of #25022, porting #22310 to 4.x
a way to run scheduled "quick" tasks in approximately regular intervals
In a normal hosting situation that would be job for cron, but if your hosting doesn’t offer cron, then a common solution in these cases is “poor man’s cron:” on every page load (onAfterRespond
) the scheduler system plugin check if itself is expected to run and then trigger a new event onExecuteScheduledTask
then any job plugin wich listen to that event check if itself is expected to run and if so execute his task.
onAfterRespond
)install these 2 silly job plugin:
plg_job_jobthree.zip
plg_job_jobone.zip
this one is an example on how to schedule an sql export
plg_job_jobexportdb.zip
job one simply sleep for 1 seconds
job two simply sleep for 3 seconds
set options, publish the plugin and save plugin ONE
Frequency (in unit) set 1
Unit of time set minute
means that this plugin run 1xminute = every minute
if onExecuteScheduledTask
has been triggered
depending on your (plugins) settings
take a coffe/smoke/whatever
randomly click a couple of frontend pages
check the log file at yoursite/administrator/logs/joomla_web.php
you'll see something like this
#
#<?php die('Forbidden.'); ?>
#Date: 2019-05-30 17:24:39 UTC
#Software: Joomla! 4.0.0-alpha10-dev Development [ Amani ] 16-May-2019 19:22 GMT
#Fields: datetime priority clientip category message
2019-05-30T17:24:39+00:00 INFO 127.0.0.1 scheduler Scheduler>start
2019-05-30T17:24:40+00:00 INFO 127.0.0.1 scheduler Job>jobone Task#221 Processing Time: 1.01 seconds.
2019-05-30T17:24:43+00:00 INFO 127.0.0.1 scheduler Job>jobtwo Task#166 Processing Time: 3.01 seconds.
2019-05-30T17:24:43+00:00 INFO 127.0.0.1 scheduler Scheduler>stop>task#1564 Total Processing Time: 4.04 seconds.
2019-05-30T17:24:47+00:00 INFO 127.0.0.1 scheduler Scheduler>start
2019-05-30T17:24:47+00:00 INFO 127.0.0.1 scheduler Scheduler>stop>task#1565 Total Processing Time: 0.04 seconds.
1st run
the scheduler plugin (wich runs always frequency=0) trigger the onExecuteScheduledTask
event
so plugin ONE and plugin TWO are executed (cause no previous run)
2nd run
the scheduler plugin (wich runs always frequency=0) trigger the onExecuteScheduledTask
event
but this time both plugin have an expected scheduled time (for example ONE each 1min TWO each 5min)
the 2nd run was at 17:24:47+00:00
the 1st run was at 17:24:43+00:00
they have only 4 seconds of difference form the previous execution so both plugin are not expected to run
3rd run
it depends on how fast you click & settings
on scheduler system plugin
set Webcron to yes/enable
set Key to mywebcronactivationkey
then use yoursiteurl/index.php?webcronkey=mywebcronactivationkey
as the url for webcron services to be triggred
Go to the cli folder of your website and run
php joomla.php job:run
check the log file at yoursite/administrator/logs/joomla_cli.php
N/A
it can be used for tasks like:
yes
Status | New | ⇒ | Pending |
Category | ⇒ | Administration SQL com_admin Postgresql com_plugins Language & Strings Installation Libraries Front End Plugins |
Update SQL script for adding the new table seems to be missing.
I have tested this item
Tested web version. Had to apply database fix using System > Database first.
Changed the frequency to 5 min on Job 3 to test the frequency cycle, and sleep time to 10 seconds, which is why the figures for Job>jobthree Task#3 is delayed compared to the other instances.
Refreshed page a few times a few minutes apart. Log shown below.
#Date: 2020-06-13 14:47:47 UTC
#Software: Joomla! 4.0.0-beta2-dev Development [ Ma単ana ] 30-May-2020 21:13 GMT
#Fields: datetime priority clientip category message
2020-06-13T14:47:47+00:00 INFO 101.116.160.42 scheduler Starting Scheduler
2020-06-13T14:47:52+00:00 INFO 101.116.160.42 scheduler Job>jobthree Task#1 Return Code is: 0 Processing Time: 5.06 seconds.
2020-06-13T14:47:53+00:00 INFO 101.116.160.42 scheduler Job>jobone Task#1 Return Code is: 0 Processing Time: 1.06 seconds.
2020-06-13T14:47:53+00:00 INFO 101.116.160.42 scheduler Scheduler tooks 6.17 seconds.
2020-06-13T14:48:05+00:00 INFO 101.116.160.42 scheduler Starting Scheduler
2020-06-13T14:48:05+00:00 INFO 101.116.160.42 scheduler Scheduler tooks 0.03 seconds.
2020-06-13T14:48:57+00:00 INFO 101.116.160.42 scheduler Starting Scheduler
2020-06-13T14:49:01+00:00 INFO 101.116.160.42 scheduler PLG_SYSTEM_SCHEDULER_LOCKED
2020-06-13T14:49:02+00:00 INFO 101.116.160.42 scheduler Job>jobthree Task#2 Return Code is: 0 Processing Time: 5.03 seconds.
2020-06-13T14:49:03+00:00 INFO 101.116.160.42 scheduler Job>jobone Task#2 Return Code is: 0 Processing Time: 1.04 seconds.
2020-06-13T14:49:03+00:00 INFO 101.116.160.42 scheduler Scheduler tooks 6.11 seconds.
2020-06-13T14:49:05+00:00 INFO 101.116.160.42 scheduler Starting Scheduler
2020-06-13T14:49:05+00:00 INFO 101.116.160.42 scheduler Scheduler tooks 0.04 seconds.
2020-06-13T14:50:42+00:00 INFO 101.116.160.42 scheduler Starting Scheduler
2020-06-13T14:50:43+00:00 INFO 101.116.160.42 scheduler Job>jobone Task#3 Return Code is: 0 Processing Time: 1.02 seconds.
2020-06-13T14:50:43+00:00 INFO 101.116.160.42 scheduler Scheduler tooks 1.05 seconds.
2020-06-13T14:51:42+00:00 INFO 101.116.160.42 scheduler Starting Scheduler
2020-06-13T14:51:42+00:00 INFO 101.116.160.42 scheduler Scheduler tooks 0.06 seconds.
2020-06-13T14:51:56+00:00 INFO 101.116.160.42 scheduler Starting Scheduler
2020-06-13T14:51:57+00:00 INFO 101.116.160.42 scheduler Job>jobone Task#4 Return Code is: 0 Processing Time: 1.04 seconds.
2020-06-13T14:51:57+00:00 INFO 101.116.160.42 scheduler Scheduler tooks 1.07 seconds.
2020-06-13T14:53:40+00:00 INFO 101.116.160.42 scheduler Starting Scheduler
2020-06-13T14:53:41+00:00 INFO 101.116.160.42 scheduler Job>jobone Task#5 Return Code is: 0 Processing Time: 1.05 seconds.
2020-06-13T14:53:41+00:00 INFO 101.116.160.42 scheduler Scheduler tooks 1.11 seconds.
2020-06-13T14:59:30+00:00 INFO 101.116.160.42 scheduler Starting Scheduler
2020-06-13T14:59:40+00:00 INFO 101.116.160.42 scheduler Job>jobthree Task#3 Return Code is: 0 Processing Time: 10.03 seconds.
2020-06-13T14:59:41+00:00 INFO 101.116.160.42 scheduler Job>jobone Task#6 Return Code is: 0 Processing Time: 1.02 seconds.
2020-06-13T14:59:41+00:00 INFO 101.116.160.42 scheduler Scheduler tooks 11.10 seconds.
Labels |
Added:
?
?
|
I have tested this item
Took the tip and updated the database before installing patch. Ran a few tests and got sensible output. joomla_cli.php:
2020-06-13T19:22:53+00:00 INFO - scheduler Scheduler started
2020-06-13T19:22:53+00:00 INFO - scheduler Scheduler tooks:0.008
2020-06-13T19:24:07+00:00 INFO - scheduler Scheduler started
2020-06-13T19:24:08+00:00 INFO - scheduler Job>jobone Task#3 Return Code is: 0 Processing Time: 1.02 seconds.
2020-06-13T19:24:08+00:00 INFO - scheduler Scheduler tooks:1.026
2020-06-13T19:24:40+00:00 INFO - scheduler Scheduler started
2020-06-13T19:24:45+00:00 INFO - scheduler Job>jobthree Task#2 Return Code is: 0 Processing Time: 5.03 seconds.
2020-06-13T19:24:45+00:00 INFO - scheduler Scheduler tooks:5.036
2020-06-13T19:29:52+00:00 INFO - scheduler Scheduler started
2020-06-13T19:29:57+00:00 INFO - scheduler Job>jobthree Task#3 Return Code is: 0 Processing Time: 5.03 seconds.
2020-06-13T19:29:58+00:00 INFO - scheduler Job>jobone Task#4 Return Code is: 0 Processing Time: 1.01 seconds.
2020-06-13T19:30:01+00:00 INFO - scheduler Job>jobexportdb Task#2 Return Code is: 0 Processing Time: 3.65 seconds.
2020-06-13T19:30:01+00:00 INFO - scheduler Scheduler tooks:9.708
Click Tested successfully - but one last rerun of CLI ended up with a long stack trace. Looks as though it is trying to write a db.zip file to somewhere without write access. Here is the first part:
Scheduler started
PHP Notice: Undefined variable: directory in /Users/ceford/Sites/j4beta2/plugins/job/jobexportdb/jobexportdb.php on line 120
PHP Stack trace:
PHP 1. {main}() /Users/ceford/Sites/j4beta2/cli/joomla.php:0
PHP 2. Joomla\CMS\Application\ConsoleApplication->execute() /Users/ceford/Sites/j4beta2/cli/joomla.php:69
PHP 3. Joomla\CMS\Application\ConsoleApplication->execute() /Users/ceford/Sites/j4beta2/libraries/src/Application/ConsoleApplication.php:227
PHP 4. Joomla\CMS\Application\ConsoleApplication->doExecute() /Users/ceford/Sites/j4beta2/libraries/vendor/joomla/console/src/Application.php:438
PHP 5. Joomla\CMS\Application\ConsoleApplication->doExecute() /Users/ceford/Sites/j4beta2/libraries/src/Application/ConsoleApplication.php:184
PHP 6. Joomla\CMS\Application\ConsoleApplication->runCommand() /Users/ceford/Sites/j4beta2/libraries/vendor/joomla/console/src/Application.php:393
PHP 7. Joomla\CMS\Console\SchedulerCommand->execute() /Users/ceford/Sites/j4beta2/libraries/vendor/joomla/console/src/Application.php:1165
PHP 8. Joomla\CMS\Console\SchedulerCommand->doExecute() /Users/ceford/Sites/j4beta2/libraries/vendor/joomla/console/src/Command/AbstractCommand.php:235
PHP 9. Joomla\CMS\Console\SchedulerCommand->triggerJobs() /Users/ceford/Sites/j4beta2/libraries/src/Console/SchedulerCommand.php:94
PHP 10. Joomla\CMS\Application\ConsoleApplication->triggerEvent() /Users/ceford/Sites/j4beta2/libraries/src/Console/SchedulerCommand.php:142
PHP 11. Joomla\Event\Dispatcher->dispatch() /Users/ceford/Sites/j4beta2/libraries/src/Application/EventAware.php:111
PHP 12. PlgJobJobexportdb->Joomla\CMS\Plugin{closure:/Users/ceford/Sites/j4beta2/libraries/src/Plugin/CMSPlugin.php:267-290}() /Users/ceford/Sites/j4beta2/libraries/vendor/joomla/event/src/Dispatcher.php:495
PHP 13. PlgJobJobexportdb->onExecuteScheduledTask() /Users/ceford/Sites/j4beta2/libraries/src/Plugin/CMSPlugin.php:285
PHP 14. PlgJobJobexportdb->jobexportdbTask() /Users/ceford/Sites/j4beta2/plugins/job/jobexportdb/jobexportdb.php:85
PHP Warning: file_put_contents(/j4-beta_DB_2020-06-13T19-29-58.zip): failed to open stream: Read-only file system in /Users/ceford/Sites/j4beta2/libraries/vendor/joomla/filesystem/src/File.php on line 232
@ceford there was a silly error on the exportdb demo plugin fixed now
plg_job_jobexportdb.zip
I have tested this item
A few comments
1 The web log is OK:
2020-06-14T06:54:36+00:00 INFO ::1 scheduler Starting Scheduler
2020-06-14T06:54:41+00:00 INFO ::1 scheduler Job>jobthree Task#7 Return Code is: 0 Processing Time: 5.08 seconds.
2020-06-14T06:54:42+00:00 INFO ::1 scheduler Job>jobone Task#10 Return Code is: 0 Processing Time: 1.08 seconds.
2020-06-14T06:54:45+00:00 INFO ::1 scheduler Job>jobexportdb Task#5 Return Code is: 0 Processing Time: 2.93 seconds.
2020-06-14T06:54:45+00:00 INFO ::1 scheduler Scheduler tooks 9.14 seconds.
Except that tooks should be took.
2. The cli log is missing seconds:
2020-06-14T06:38:07+00:00 INFO - scheduler Scheduler started
2020-06-14T06:38:12+00:00 INFO - scheduler Job>jobthree Task#6 Return Code is: 0 Processing Time: 5.01 seconds.
2020-06-14T06:38:13+00:00 INFO - scheduler Job>jobone Task#9 Return Code is: 0 Processing Time: 1.02 seconds.
2020-06-14T06:38:16+00:00 INFO - scheduler Job>jobexportdb Task#4 Return Code is: 0 Processing Time: 2.59 seconds.
2020-06-14T06:38:16+00:00 INFO - scheduler Scheduler tooks:8.628
3. On documentation
I have been revising Help screens recently and this has reminded me that plugins still need some work. At the moment all plugins share the same Help screen URL. I pulled out the list of 92 plugins into a separate chunk but that is still too long and does not yet have J4 images. The best solution is a separate chunk for each plugin. Another week locked down/up could swing it. But where would the cli documentation be located?
should be here https://docs.joomla.org/Category:Joomla_CLI
conflict solved
That constant doesn't mean no execution but means execution ko is with
errors
Il dom 14 giu 2020, 16:47 Richard Fath notifications@github.com ha
scritto:
@richard67 commented on this pull request.
In administrator/components/com_plugins/src/Jobs/JobsPlugin.php
#29592 (comment):
- Exit Code For no time to run
- */
- public const JOB_NO_TIME = 1;
- /**
- Exit Code For lock failure
- */
- public const JOB_NO_LOCK = 2;
- /**
- Exit Code For execution failure
- */
- public const JOB_KO_RUN = 3;
But it should be consistent, or not? The others are JOB_NO_....
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#29592 (comment),
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AABMLMOZVTACMX5BZH3PRPLRWTPGVANCNFSM4N44RQ3Q
.
I see .. well it looks like a typo if you don't know that.
Add to the list of core extensions.
Add to the list of core extensions.
Done
I have tested this item
Tested successfully in Beta2-dev of 15 May.
Test environment: Wampserver 3.2.2 Apache 2.4.43a MySQL 8.0.20 PHP 7.4.7
@richard67 done
@alikon When having applied the patch and the update SQL so that the new plugin is enabled and configured, but not having installed any of the jobs plugins yet, and am using the cli, then I get following for the job:run
command:
The jobs:list
command works well, it lists no jobs, which is correct.
@ceford @toivo Please wait with testing, there might be some correction necessary.
I have tested this item
Worked as described. Here is my log:
2020-06-22T18:59:16+00:00 INFO ::1 scheduler Starting Scheduler
2020-06-22T18:59:22+00:00 INFO ::1 scheduler Job>jobthree Task#8 Return Code is: 0 Processing Time: 5.08 seconds.
2020-06-22T18:59:23+00:00 INFO ::1 scheduler Job>jobone Task#11 Return Code is: 0 Processing Time: 1.08 seconds.
2020-06-22T18:59:26+00:00 INFO ::1 scheduler Job>jobexportdb Task#6 Return Code is: 0 Processing Time: 3.08 seconds.
2020-06-22T18:59:26+00:00 INFO ::1 scheduler Scheduler took 9.31 seconds.
2020-06-22T18:59:29+00:00 INFO ::1 scheduler Starting Scheduler
2020-06-22T18:59:29+00:00 INFO ::1 scheduler Scheduler took 0.02 seconds.
2020-06-22T18:59:31+00:00 INFO ::1 scheduler Starting Scheduler
2020-06-22T18:59:31+00:00 INFO ::1 scheduler Scheduler took 0.02 seconds.
2020-06-22T19:04:27+00:00 INFO ::1 scheduler Starting Scheduler
2020-06-22T19:04:32+00:00 INFO ::1 scheduler Job>jobthree Task#9 Return Code is: 0 Processing Time: 5.08 seconds.
2020-06-22T19:04:33+00:00 INFO ::1 scheduler Job>jobone Task#12 Return Code is: 0 Processing Time: 1.08 seconds.
2020-06-22T19:04:33+00:00 INFO ::1 scheduler Scheduler took 6.19 seconds.
On documentation: I added a section to the System Plugin Help screen:
https://docs.joomla.org/Chunk4x:Extensions_Plugin_Manager_Edit_System_Group
That will get into the Help screen when the /en version gets regenerated.
There is a link to a Tutorial on Writing A CLI Application. Now looking for a tutorial on how to write a Job Plugin.
I have tested this item
Tested successfully in Beta2-dev of 23 June. The scheduler, webcron and cli/cron worked as expected.
Here is the log file from webcron, joomla_web.php:
#
#Date: 2020-06-23 03:12:48 UTC
#Software: Joomla! 4.0.0-beta2-dev Development [ Mañana ] 30-May-2020 21:13 GMT
#Fields: datetime priority clientip category message
2020-06-23T03:12:48+00:00 INFO 127.0.0.1 scheduler Starting Scheduler
2020-06-23T03:12:49+00:00 INFO 127.0.0.1 scheduler Job>jobexportdb Task#3 Return Code is: 0 Processing Time: 1.59 seconds.
2020-06-23T03:12:50+00:00 INFO 127.0.0.1 scheduler Job>jobone Task#10 Return Code is: 0 Processing Time: 1.01 seconds.
2020-06-23T03:12:52+00:00 INFO 127.0.0.1 scheduler Job>jobthree Task#4 Return Code is: 3 Processing Time: 2.01 seconds.
2020-06-23T03:12:52+00:00 INFO 127.0.0.1 scheduler Scheduler took 4.64 seconds.
2020-06-23T03:13:13+00:00 INFO 127.0.0.1 scheduler Starting Scheduler
2020-06-23T03:13:13+00:00 INFO 127.0.0.1 scheduler Scheduler took 0.01 seconds.
Here is the log file from cli/cron, joomla_cli.php:
#
#Date: 2020-06-23 02:59:51 UTC
#Software: Joomla! 4.0.0-beta2-dev Development [ Mañana ] 30-May-2020 21:13 GMT
#Fields: datetime priority clientip category message
2020-06-23T02:59:51+00:00 INFO - scheduler Scheduler started
2020-06-23T02:59:52+00:00 INFO - scheduler Job>jobexportdb Task#2 Return Code is: 0 Processing Time: 1.45 seconds.
2020-06-23T02:59:53+00:00 INFO - scheduler Job>jobone Task#9 Return Code is: 0 Processing Time: 1.03 seconds.
2020-06-23T02:59:55+00:00 INFO - scheduler Job>jobthree Task#3 Return Code is: 3 Processing Time: 2.01 seconds.
2020-06-23T02:59:55+00:00 INFO - scheduler Scheduler took: 4.52
Status | Pending | ⇒ | Ready to Commit |
RTC
thanks for the doc part
Labels |
Added:
?
|
Labels |
Added:
Conflicting Files
|
Title |
|
Labels |
Added:
?
Removed: Conflicting Files |
Labels |
Removed:
?
|
Also one more. Why do we use onExecuteScheduledTask
and onAfterJob
as plugin events?
Why not
onExecuteScheduledTask
& onAfterScheduledTask
or
onBeforeJob
& onAfterJob
I personally would choose the first one but I would not mix it
Status | Ready to Commit | ⇒ | Pending |
Taking of RTC until the comments are adressed.
Labels |
Removed:
?
|
this pr was born for 3.9...... then has been decided it was late for 3.9..... so supposed to be in 4.0, then yet again has been decided it was late for 4.0
.... i really hope that it still can have a chance to to land in 4.1
.... i really hope that it still can have a chance to to land in 4.1
Sure lets move it forward and get it merged for 4.1 :)
@alikon any update on #29592 (comment) and can you please update the updatesql files to 4.1.0 too?
well mostly matter of taste or english mistake from myside.... i've no preference as i'm an english dumb....
In the hope that all remarks has been addressed at least for now ....
I have just updated 4.1-dev with 4.0 and updated this PR acordingly so once drone is happy this can be fully tested :)
the "Download — Prebuilt packages are available for download." is missing from the 4.1-dev pipeline ?
I really like the idea of having a Cron fallback in the Joomla! and it's also a feature on the roadmap, so really thank you for bringing this code in.
As we have still some time for Joomla! 4.1 I really would love to see to go a few steps further, so that we get a solution where the end user can manage crons over an interface and at the end the entries will be saved in a cron tab like way in the database.
It would also be great if cron is available on the server there could be some integration.
Also a CLI functionality and some API (helper?) would really be great.
Last but not least probably we should also take care of the user executing the Cron. In the worse case it's a user who has not the privileges.
(Thanks to @nibra and @HLeithner for a lot of valuable feedback here)
tl;tr; I really like what you did but I think more is possible, would you be up to spend some more time and enhance your great start to bring it on the next level?
I think more is possible
of course,
but what we have now in this pr is a good start.....
the 1st step when and if this pr will be merged should be to use the poor man cron's for tasks like :
but i'll not spent more time on this, if this pr will not be merged (i've proposed this before 3.9 goes stable)
p.s
you don't need to manage complex crontab entry
something like this is enough
* * * * * /usr/local/bin/php /shared/httpd/j4/htdocs/cli/joomla.php job:run >> /shared/httpd/cron.log
Status | Pending | ⇒ | Closed |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2021-06-26 08:08:47 |
Closed_By | ⇒ | alikon |
if there is still someone interested i've decoupled it to be an external component https://github.com/alikon/testcom
Would it be possible that you recover the branch for a few days, so I can copy it into my repository? I still like the base idea and implementation.
I still like the base idea and implementation.
So much so that you farmed it out to a "Student" to complete? https://summer.iscas.ac.cn/#/org/prodetail/210880722?lang=en
I am getting this in the joomla_web.php log:
#
#Date: 2020-06-13 09:28:59 UTC
#Software: Joomla! 4.0.0-beta2-dev Development [ Mañana ] 30-May-2020 21:13 GMT
#Fields: datetime priority clientip category message
2020-06-13T09:28:59+00:00 INFO ::1 scheduler PLG_SYSTEM_SCHEDULER_LOCKED
2020-06-13T09:30:05+00:00 INFO ::1 scheduler PLG_SYSTEM_SCHEDULER_LOCKED
2020-06-13T09:32:07+00:00 INFO ::1 scheduler PLG_SYSTEM_SCHEDULER_LOCKED
2020-06-13T09:36:45+00:00 INFO ::1 scheduler PLG_SYSTEM_SCHEDULER_LOCKED
The CLI gives this:
Scheduler
Scheduler started
[OK] Scheduler finished in 0.009
joomla_cli.php log has this:
#
#Date: 2020-06-13 09:38:03 UTC
#Software: Joomla! 4.0.0-beta2-dev Development [ Mañana ] 30-May-2020 21:13 GMT
#Fields: datetime priority clientip category message
2020-06-13T09:38:03+00:00 INFO - scheduler Scheduler started
2020-06-13T09:38:03+00:00 INFO - scheduler Scheduler tooks:0.008
My System - Job Scheduler plugin is showing Yes/No radio buttons, not a slider as in your screen grab. So has this mornings Nightly Build update not worked properly for me?
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/29592.