[#45143] - [5.4] Automated Core Update Client
- Fixed in Code Base
- 20 May 2025
- Medium
- Build: 5.3-dev
- # 45143
- Diff
- SniperSister:automated-updates/client
User tests: Successful: Unsuccessful:
Summary of Changes
This PR implements automated core updates for Joomla. It's the "client" implementation, the server implementation can be found here: https://github.com/joomla-projects/Automated-Updates-Server
In general, the implemented concept utilizes existing logic and functionality ans has been built as a thin "remote control" layer around the current code:
- Update retrieval is handled by TUF, guaranteeing reliable, trustworthy update information. The information about a new version is therefore not "pushed" by the autoupdate server (causing potential loopholes if the server is compromised) but fetched by the site itself
- Update download and checksum handling is performed by the existing model, that com_joomlaupdate uses
- Update extraction is performed using the standalone extraction script of com_joomlaupdate - as this script normally resides in the administrator folder and that folder is often protected by additional measures (IP blocks, basic auth checks), a statement in the main index.php has been ended to require that script directly, allowing direct access
- Postinstall scripts are again performed by the existing com_joomlaupdate models
The communication between the update server (which handles periodic health checks and triggers the updates) and the site happens via a bunch of newly added webservice endpoints. For access control, an auth token, that is generated in the site and is sent to the server on registration, is used.
Besides these endpoints, the PR adds multiple "supporting" extensions and tweaks:
- a postinstall message, highlighting the new feature on existing installations, pushing site owners to enable it
- a guided tour, highlighting the new feature too
- a quick icon, displaying the current feature and connection status
- new mail templates to notify admins about successful or failed updates
- CLI commands to register and unregister a site in the autoupdate server.
This PR is joint effort together with @rdeutz @bembelimen @HLeithner - thank you guys! Thank you @brianteeman for the language support :) and thank you @richard67 for taking care of the adjustments to 5.4
Testing Instructions
The following instructions will cover a full autoupdate cycle. In order for the update to be performed, you have to install the site on a publicly accessible server. A local enviroment will not work.
1a. Install a new Joomla site using the package provided at Full Download
1b. Update a Joomla site using the package provided at Update Download
1c. Update without automated update, change in the database table #_update_sites the column location to https://update.joomla.org/alpha/, change in the file administrator/components/com_joomlaupdate/src/Model/UpdateModel.php in line 123 to $updateURL = 'https://update.joomla.org/alpha/';
4. Log into the administrator site and check that the guided tour for the new feature is shown; that login will also trigger the registration of your site in the update server
5. As a (faked) newer version is available, the update server will trigger an autoupdate of your newly installed site automatically. This process will take approx. 5-10 minutes.
6. When the update is completed (or failed) you'll receive an email to the address provided during the installation process
7. Verify the update result after the mail has been received
8. Manually disable autoupdates using the provided option in com_joomlaupdate; verify the registration state using the quickicon in the dashboard
9. Re-Enable autoupdates and verify the state again
Link to documentations
Please select:
-
Documentation link for docs.joomla.org: Todo
-
No documentation changes for docs.joomla.org needed
-
Pull Request link for manual.joomla.org:
-
No documentation changes for manual.joomla.org needed
| Status | New | ⇒ | Pending |
joomla-cms-bot
- | Category | ⇒ | SQL Administration com_admin Postgresql com_joomlaupdate Language & Strings Repository NPM Change JavaScript Installation |
| Labels |
Added:
Language Change
NPM Resource Changed
PR-5.3-dev
|
||
Thx @brianteeman for the fixes
When trying to save the options in order to get an update token I got the following error. I assume that is because j51.test is not a real domain and just local but still the error message should be better. (i did get an updated id however)
Last Checked??
I assumed that this would get a value each time I manually check for updates in the component. Do I assume incorrectly
Post Installation Message
What do I have to do to see the new message
Auth Token
For access control, an auth token, that is generated in the site and is sent to the server on registration, is used.
How unique is this? What happens if I copy an entire site to a new domain.
When trying to save the options in order to get an update token I got the following error. I assume that is because j51.test is not a real domain and just local but still the error message should be better. (i did get an updated id however)
yes you assume right, the reason is that your webserver couldn't be reached, server message is generated server side so a couple of issues cloud be have more human readable error messages.
Last Checked??
I assumed that this would get a value each time I manually check for updates in the component. Do I assume incorrectly
No, the Update Server checks every 24 hours the health of the website, when this is done this field is updates. A indication for you that your side will get updates, this timestamp is also used in the quick icon, if it's older then 4 days you get a warning.
Post Installation Message What do I have to do to see the new message
disable the autoupdate in the configuration, or update a joomla 5.2 side.
Auth Token
For access control, an auth token, that is generated in the site and is sent to the server on registration, is used.
How unique is this? What happens if I copy an entire site to a new domain.
combination of domain and token is unique, if you copy a site the same token will be used for registration, which is not a problem.
When you disable automated updates in the options it appears to be trying to connect to the update service and deactivate it there. However the error message given is the following which doesnt make sense
shouldn't this pr made against 6 ?
@SniperSister The update SQL scripts need to be renamed. Currently the newest update SQL in the 5.3-dev branch is "5.3.0-2025-03-14.sql", and the newest one which has already been released with a beta is "5.3.0-2025-02-22.sql".
So anything coming with the next release has to be newer than "5.3.0-2025-02-22.sql".
If we would grant updating between nightly builds to work, we would have to rename all new scripts to something newer than "5.3.0-2025-03-14.sql". But the "5.3.0-2025-03-14.sql" is not new, and it should not be modified if not absolutely necessary.
This would mean: Rename "5.3.0-2025-03-15.sql" to "5.3.0-2025-03-17.sql", rename "5.3.0-2025-01-18.sql" to "5.3.0-2025-03-15.sql", leave "5.3.0-2025-03-14.sql" as it is in the 5.3-dev branch, and create a new update SQL "5.3.0-2025-03-16.sql" which updates the guided tours from "5.3.0-2025-03-14.sql" to contain your changes.
But it seems your "5.3.0-2025-01-18.sql" shall run before the others, as it defined the new extension, and if we only grant updates between the betas, i.e. from beta 2 to beta 3, we could do it easier and modify the "5.3.0-2025-03-14.sql" like you did.
This would mean: Rename "5.3.0-2025-01-18.sql" to "5.3.0-2025-03-13.sql" and leave the other 2 as they are in your PR,
But in this case the "5.3.0-2025-03-13.sql" will not run if someone updates to beta 3 from beta 2 a 5.3 nightly build which contained already the "5.3.0-2025-03-14.sql".
Please chose what you prefer, I can help then if necessary with a PR against your branch.
shouldn't this pr made against 6 ?
It’s not a b/c break?
5.3 had already feature freeze, or not?
but it's a new feature or it's not ? a lot of more silly pr's have been moved massively to 6
i'm sorry still trying to understand the "criteria"
| Title |
|
||||||
FYI: we decided to move the feature to 5.4, I’ll rebase the PR tomorrow.
| Status | Pending | ⇒ | Closed |
| Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2025-03-16 10:25:35 |
| Closed_By | ⇒ | SniperSister |
| Status | Closed | ⇒ | New |
| Closed_Date | 2025-03-16 10:25:35 | ⇒ | |
| Closed_By | SniperSister | ⇒ |
| Status | New | ⇒ | Pending |
FYI: we decided to move the feature to 5.4, I’ll rebase the PR tomorrow.
all other proposed new features that did not make the 5.3 feature freeze last month were moved by the maintainers to j6
i would like to clarify, i'm happy for this feature and i would like to see it in wild as soon as possible, and thanks to you all for your big effort, i'm just trying to understand, what is the criteria, just that, we have 4.x branch , 5.2 branch , 5.3 branch, 5.4 branch , and 6 branch so once again what is the criteria ?
@brianteeman Yes, 5.4 shall not be a feature release, it shall be the bridge version to 6.0, like 3.10 was for 4 and 4.4 is for 5, and so 5.4 focuses on stable and safe updating to 6, and of course it will have bug fixes.
However, there are some exceptions if a new feature helps with the updating to the next major. In 3.10 we had the pre-update checker. In 4.4 we had the b/c plugin as a stub so it was available when updating to 5. And in 5.4 it makes sense to have already the infrastructure for the automated updates so these do not have to introduced when updating to 6, even if we won't use the automated update for updating to 6 because as far as I understand it will not update to new major versions, only new minors or patch versions. But having the automated updates already working on 5.4 reduces the risk when updating to 6 in my opinion.
That's why I am ok with it for going into 5.4.
| Title |
|
||||||
| Category | SQL Administration com_admin Postgresql com_joomlaupdate Language & Strings Repository NPM Change JavaScript Installation | ⇒ | Unit Tests SQL Administration com_admin Postgresql com_categories com_config com_joomlaupdate com_tags Language & Strings Modules |
i would like to clarify, i'm happy for this feature and i would like to see it in wild as soon as possible, and thanks to you all for your big effort, i'm just trying to understand, what is the criteria, just that, we have 4.x branch , 5.2 branch , 5.3 branch, 5.4 branch , and 6 branch so once again what is the criteria ?
@alikon Se my previous comment about new features. See the road map about the rest.
Unrelated changed files shown due to the rebase will disappear after the next upmerge from 5.3-dev to 5.4-dev and then a branch update of this PR. Am working on it right now. Done.
| Labels |
Added:
Unit/System Tests
PR-5.4-dev
Removed: NPM Resource Changed |
||
| Category | SQL Administration com_admin Postgresql com_joomlaupdate Language & Strings Unit Tests com_categories com_config com_tags Modules | ⇒ | SQL Administration com_admin Postgresql com_joomlaupdate Language & Strings Repository NPM Change JavaScript Installation |
| Labels |
Added:
Feature
NPM Resource Changed
Removed: Unit/System Tests PR-5.3-dev |
||
@SniperSister Testing instructions should be adjusted to 5.4.0-alpha1. Maybe we can find a way to test it before that, e.g. when this PR is merged and we have 5.4-dev nightlies which include this PR? When someone is on such a nightly and we could offer some kind of pre-release in Tuf, maybe 5.4.0-alpha1-0 or so, so PHP version_compare says it is higher than "5.4.0-alpha1-dev" but lower than "5.4.0-alpha1", could that work? The targetplatform of such pre-release should of course be limited to 5.4.0, and the stability should be "Development".
That could even work without this PR being merged yet, without nightlies, when someone is on the patched package of this PR or has applied this PR on a current 5.4-dev branch.
@SniperSister and everyone else, thank you in advance for including this feature in version 5.4 ❤️.
And we require a detailed test plan to confirm that automated core updates are functioning correctly on all existing sites under all circumstances
new mail templates to notify admins about successful or failed updates
Did these get missed from the pr
I think @SniperSister branch has not been sync with the development repository yet.
I would suggest we take the Guided Tour out of this PR, because it may not be the only information we want to add to the 'What's new tour' for 5.4. I have personally kept the code/language keys/image that were created in this PR so that we can create a separate one with the tour and what may be added to it or modified.
I really appreciate the work that was done here.
| Category | SQL Administration com_admin Postgresql com_joomlaupdate Language & Strings Repository NPM Change JavaScript Installation | ⇒ | SQL Administration com_admin Postgresql com_joomlaupdate Language & Strings Repository NPM Change JavaScript |
@SniperSister Please rename the update SQL scripts to something newer than the "5.4.0-2025-04-23.sql" currently present in the 5.4-dev branch, e.g. rename "5.4.0-2025-03-16.sql" to "5.4.0-2025-05-09.sql", "5.4.0-2025-03-17.sql" to "5.4.0-2025-05-10.sql" and "5.4.0-2025-03-18.sql" to "5.4.0-2025-05-11.sql".
@SniperSister P.S.: ... and add the check of the postinstall message to the testing instructions. Thanks in advance.
| Category | SQL Administration com_admin Postgresql com_joomlaupdate Language & Strings Repository NPM Change JavaScript | ⇒ | SQL Administration com_admin Postgresql com_joomlaupdate Language & Strings Repository NPM Change JavaScript Installation |
@SniperSister Please rename the update SQL scripts to something newer than the "5.4.0-2025-04-23.sql" currently present in the 5.4-dev branch, e.g. rename "5.4.0-2025-03-16.sql" to "5.4.0-2025-05-09.sql", "5.4.0-2025-03-17.sql" to "5.4.0-2025-05-10.sql" and "5.4.0-2025-03-18.sql" to "5.4.0-2025-05-11.sql".
I combined the 3 files to one file with todays date
I've merged the suggestions by @richard67 and @brianteeman (thank you guys) and fixed a missing use statement that was discovered during internal testing
and fixed a missing
usestatement that was discovered during internal testing
@SniperSister I don't see a fix like that in the latest commits. Could you double check if that statement is there?
@richard67 see 7646135
My test as requested in Mattermost (looks good):
- From previous test installation, deleted Joomla files and emptied database
- Installed Joomla and Logged in to Admin
- Tour started is Welcome to Joomla - not the new feature
- Message arrived: Joomla! update failed for J541 – https://fford.me.uk/j541/api/ BUT
Your site has been successfully automatically updated from 5.4.100 to 5.4.102. - Normal intro tour finished
- Dismissed the Help Joomla Stats invite.
- The Home Dashboard shows 5.4.100
- Reload Home Dashboard and it shows 5.4.102
- Changed Automated Update from Yes to No
- Home Dashboard icon: Automated Updates are disabled in amber
- Enabled Autoupdate - Information message says Registered to autoupdate service
- Home Dashboard icon says Automated Updates working in green.
cPanel account on my own WHM VPS. Extract from System Information:
Database Type mysql
Database Version 8.0.42
Database Collation utf8mb4_0900_ai_ci
Database Connection Collation utf8mb4_0900_ai_ci
Database Connection Encryption None
Database Server Supports Connection Encryption Yes
PHP Version 8.2.28
Web Server Apache
WebServer to PHP Interface fpm-fcgi
Joomla! Version Joomla! 5.4.102 Stable [ Kutegemea ] 10-May-2025 19:50 GMT
Partly successfully tested
- Updated an existing Website from 5.3.0 to 5.4.103
- Make sure you are on Update Channel default and Minimum Stability: Stable,
- Change in Joomla Update to Update Channel default and Minimum Stability Stable, enabled Autoupdate
- Upon Save & Close first time:
- Save & Close second time (no changes made), OK (Registered to automated update service.)
- Automated Update was done almost immedatly, Email Sent
- oomla Update to Update Channel default and Minimum Stability Stable, enabled Autoupdate
was caching activated for this site?
- oomla Update to Update Channel default and Minimum Stability Stable, enabled Autoupdate
was caching activated for this site?
Yes - without caching (conservative & progressive) Save & Close works every time.
- oomla Update to Update Channel default and Minimum Stability Stable, enabled Autoupdate
was caching activated for this site?
Yes - without caching (conservative & progressive) Save & Close works every time.
ok make sense because joomla caches the component options and we likely load it from the cache in the next second, we will see how we can solve this. Cache invalidation would be the best case but might be tricky. As alternative we could load the information directly...
I added the 2 known issues to the pr description and would like to postpone the fix for this to a follow up pr to get this merged into the alpha1 so it can be tested easier.
I have tested this item ✅ successfully on b25bba3
Works! From existing 5.3.0 to 5.4.103 and automatically to 5.4.104
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/45143.
I added the 2 known issues to the pr description and would like to postpone the fix for this to a follow up pr to get this merged into the alpha1 so it can be tested easier.
The known issues have been resolved
I have tested this item ✅ successfully on b25bba3
I've successfully tested that it works, also when the site is set to offline.
In addition, I've tested updating a 5.3 to the patched package of this PR with MySQL and PostgreSQL to see if the update SQL scripts are right.
Finally I've downloaded the 5.4.105 packages used for the test and have verified that they don't contain any unexpected differences to the patched packages for this PR, except of the wanted patch for the update site and the version and that it is behind the 5.4-dev branch with some changes, but that is not related to the functionality added by this PR.
What is not nice is that if you use the button in the Guided Tour for activating the auto updates, the settings show up for a short moment, but then you are back in the guided tours, and there is no visual indication if it has worked or not until you leave the guided tours and check the quick icon or the settings.
But I think this can be fixed with a follow up PR.
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/45143.
| Status | Pending | ⇒ | Ready to Commit |
RTC
This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/45143.
| Status | Ready to Commit | ⇒ | Fixed in Code Base |
| Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2025-05-20 14:06:06 |
| Closed_By | ⇒ | muhme | |
| Labels |
Added:
RTC
|
||
A big thank you to all contributers and testers!
From me a big thank you, too.
Sometimes it says "auto updates" and sometimes "automated updates"
I think it would be better to standardise on just one name and having two might lead to some mistranslations