? Success

User tests: Successful: Unsuccessful:

avatar nikosdion
nikosdion
13 Jun 2014

See Joomla! Tracker 33862

Summary

This Pull Request makes minor modifications to com_installer for the benefit of end users who have problems fetching updates and extension developers who no longer have to explain to users that they have an impossible to fix Joomla! problem and not a bug with their extensions.

The first change is that the Purge button is reinstated in the toolbar. Reasoning: an extension provides two subsequent updates in a time period less than the "Updates caching (in hours)" (default: 6 hours) setting in the Options page. The update will be invisible until these many hours have elapsed. Even if the user knows the update is there they can't see it and report it as a bug to the developers.

The second change is the addition of an Update Sites management page. Reasoning: Joomla! will unpublish update sites without warning if it cannot contact the update site's server. The user is not notified and the update site cannot be re-enabled. This means that a temporary network issue at the wrong moment will disable updates without warning, putting end users' sites at risk. Even worse, it was IMPOSSIBLE to re-enable the disabled update sites without editing the database. This was also reported to extension developers as a bug with their software.

These are the most often reported issues with the Joomla! extension updater by end users. Narutally the project was completely unaware of the high impact as the receiving end of the complaints has always been the extensions developers, not the Joomla! PLT. As an extensions developer I felt the urge to provide a fix to them for the benefit of everyone using or developing for Joomla!.

Backwards compatibility

This PR is 100% backwards compatible.

Translation impact

This PR introduces 11 new translation strings in 3 INI files. The average time to translate them is 3 minutes.

Testing instructions

Feature 1: Purge button

Log in to your back-end. Go to the Extensions, Extension Manager menu item. Click on Update. Before the patch the toolbar buttons on the left are only Update and Find Updates. After the patch there is a new third button called Purge.

Make sure there are updates shown on the page (you may want to install an outdated extension for that matter). Click on Find Updates. The updates list is cleared. Click on Find Updates. The update list is populated again.

Feature 2: Update Sites

Log in to your back-end. Go to the Extensions, Extension Manager menu item. Look at the right hand link bar. Before the path the items are Install, Update, Manage, Discover, Database, Warnings, Install Languages. After the patch there is a new item called Update Sites. Click on it.

You see a list of update sites. Find an update site of an extension which currently has updates available and unpublish it. Go to Update, click on Purge and then click on Find Updates. You do not see the updates for the extension whose update site you just unpublished.

Go back to Update Sites. Publish the update site you had unpublished in the previous step. Go to Update, click on Purge and then click on Find Updates. You now do see the updates for the extension whose update site you just published

avatar nikosdion nikosdion - open - 13 Jun 2014
avatar infograf768
infograf768 - comment - 14 Jun 2014

+JLIB_DATABASE_ERROR_MUSTCONTAIN_A_TITLE_UPDATESITE="Update site must have a title"
has to be added also in admin en-GB

avatar infograf768
infograf768 - comment - 14 Jun 2014

Feature 2:
Disabling language packs update site
Then going to Update
Purge
Result: Purged updates****, 1 disabled site was enabled**
And then Find Updates works without going back to Update Sites to re-enable the language pack update sites.

avatar nikosdion nikosdion - change - 14 Jun 2014
Title
Improving the Joomla! extensions update for end-users
[#33862] Improving the Joomla! extensions update for end-users
avatar nikosdion nikosdion - change - 14 Jun 2014
Title
Improving the Joomla! extensions update for end-users
[#33862] Improving the Joomla! extensions update for end-users
avatar nikosdion
nikosdion - comment - 14 Jun 2014

The update site purge feature (InstallerControllerUpdate::purge) was calling InstallerModelUpdate::enableSites. That task was put in Joomla! long before this PR because there was no easy way to enable the already disabled update sites. However, now that we are giving our users the option of managing update sites it shouldn't be doing that.

You're right, it's a bug. And I just fixed it :) Can you please test this again?

avatar nikosdion
nikosdion - comment - 14 Jun 2014

@infograf768 One more thing. Could you please provide the URLs of the out of date package you are using? It will help other people with testing this PR. Thank you very much!

avatar roland-d
roland-d - comment - 14 Jun 2014

@test:
There is an untranslated string in the browser bar called COM_INSTALLER_TITLE_updatesites when I click on the update sites link.


There are 2 entries in the update sites list that are the same:
updatesites

The 2 entries called Joomla CMS are actually the Joomla Core and Joomla Extension Directory, see here:
updatesites_db
I think the name should be taken from the #__update_sites table.


When I try to sort the update sites table by Type column I get this error:

Column 'type' in order clause is ambiguous SQL=SELECT s.*,e.extension_id,e.name as extension_name,e.type,e.element,e.folder,e.client_id,e.state,e.manifest_cache FROM jos_update_sites AS s INNER JOIN jos_update_sites_extensions AS se on(se.update_site_id = s.update_site_id) INNER JOIN jos_extensions AS e ON(e.extension_id = se.extension_id) WHERE state=0 ORDER BY `type` asc LIMIT 0, 20

When I try to sort by Update Site ID, Status or Name it is not sorting by these columns but it is sorting by the Update Site column.


Testing the check for updates works as described.

That is all for now :)

avatar b2z
b2z - comment - 14 Jun 2014

I am confirming @roland-d test issues.

avatar nikosdion nikosdion - change - 14 Jun 2014
Labels Added: ?
Removed: ?
avatar jurianeven
jurianeven - comment - 14 Jun 2014

Thanks for your contribution, Nicholas!

Typo in the test instructions:

Feature 1: Purge button
... Click on Find Updates Purge. The updates list is cleared. Click on Find Updates. The update list is populated again.

Purging and finding updates works for me.

Feature 2: Update Sites

Enabling/disabling update sites works.

  1. How about adding the Update Sites menu-item after the Updates menu-item in stead of the last in the list?
  2. UX improvement: can you move the status column to the second column, right after the checkboxes?
  3. Filter by status doesn't work. Ordering works fine.

Besides that I don't see anything wrong.

I've used Joomla 3.3.1 for testing.

avatar brianteeman
brianteeman - comment - 14 Jun 2014

Why reinstate the purge button. Why not just make Find Updates do an automatic purge

avatar JoomliC
JoomliC - comment - 14 Jun 2014

@test success for latest fixes. (great PR Nic!)
@test success for feature 1 & feature 2

One suggestion :
in Updates Sites, we have duplicated id (one for each language) for Accredited Joomla! Translations, but it's only one sever. Maybe having only one line, and replace name by "Translation Packs" ?

I agree too with Brian about an automatic purge (More convenient for the average user)

Tested on 3.3.1

avatar brianteeman
brianteeman - comment - 14 Jun 2014

When you Purge the cache you get two messages.
Purged updates and There are no updates available at the moment. Please check again later.
screenshot 2014-06-14 11 17 27

That is confusing and bad UX. Surely the correct procedure would be to immediately (and automatically) run the Find Updates. This way you will get two messages
Purged updates AND either There are no updates available at the moment. Please check again later. OR a list of actual updates.

Why make me press two buttons. If I am purging the cache then obviously I want to see if there are any new updates available as well

avatar brianteeman
brianteeman - comment - 14 Jun 2014

Not created by this PR but would be good to fix at the same time.

The Notice displayed in the Extension Updater is not great
Before updating ensure that the update is compatible with your Joomla! installation. If updating Joomla!, ensure installed extensions are available for the new Joomla! version.

The second sentence is irrelevant as you cant update Joomla here.

screenshot 2014-06-14 11 21 52

However the second sentence belongs in the Com_joomlaupdate screen where there is no notice

screenshot 2014-06-14 11 22 04

avatar nikosdion
nikosdion - comment - 14 Jun 2014

@jurianeven I moved the Status column. The Update Sites was made the last item on the list as it's supposed to be the least frequently used.

@brianteeman The Find Updates button must not purge the updates. Each update site has its own last time checked column. This means that Find Updates will only fetch the update information for the extensions for which updates were not retrieved within X hours, as configured in the Options page. If you purge before finding updates you have to retrieve all updates. This is problematic since when you have too many extensions on a bandwidth limited shared host you may hit the PHP timeout limit. I'm trying to prevent that – unless you really want Joomla! to do that and you click on Purge.

Regarding the messages, they are not both messages. The green one is a message (only displayed after the action). The blue one is status information. I disagree with the "There are no updates" being displayed in a blue message-like alert box, but this is how Joomla! has been doing it since 3.0 and I am not going to change this in this PR. Simply put, I DESPERATELY NEED the Purge button and Update Sites management features for the benefit of my paying clients and I'd be damned if I risked my improvements being rejected or talked to death because I also happened to change the layout of a message.

Same goes for anything else regarding the UX of com_installer. It's deliberately out of scope of this PR. This PR has the sole purpose of saving me 20 hours of senseless support and bad reputation after each and every single release of my software, all caused by Joomla! not allowing my paying clients to easily reset the update information without going into the database. Pretty selfish? Yes, indeed. Also works for everyone else publishing extensions for Joomla!? Darn right. Everybody wins.

avatar brianteeman
brianteeman - comment - 14 Jun 2014

If we are going to make changes (and we should) then lets get them right this time and not have to make further changes in the future

avatar jurianeven
jurianeven - comment - 14 Jun 2014

Thanks Nicholas. Can you also fix this please?

Filter by status doesn't work.

avatar jurianeven
jurianeven - comment - 14 Jun 2014

Why make me press two buttons. If I am purging the cache then obviously I want to see if there are any new updates available as well

You could rename the Purge button to Purge and find updates and then of course also find updates when that button has been clicked.

avatar roland-d
roland-d - comment - 14 Jun 2014

@nikosdion All the issues I reported have been fixed. Thanks :+1:
As for @jurianeven suggestion to rename the button, looks good to me :)

avatar nikosdion
nikosdion - comment - 14 Jun 2014

@brianteeman I vehemently disagree. My PR intends to fix a very specific thing: Joomla! updates get stuck and Joomla! silently disables the update site. Anything else is out of scope of this PR.

I agree that there are UX issues, that's why I proposed a rewrite of com_installer and drafter a whitepaper on how to do that. There's a WG on that and I'm a technical advisor and sort-of-coordinator. If you want to be the UX advisor, please drop me a line. But let's keep it outside this completely unrelated PR.

avatar Bakual
Bakual - comment - 14 Jun 2014

I agree with the PR in general here. It's something which was on my mind for quite some time as well. So huge +:100: from me for that contribution.

From code review it looks mostly good. I wonder if it could be done without using the deprecated JError. I'd say either use exceptions or enqueue a message there. Or are they really needed? Sounds a bit wrong to use depecated classes and methods in new code :smile:

avatar nonumber
nonumber - comment - 14 Jun 2014

+1 for not adding back the purge button.
Purge should be done on 'Find updates' and should be done for ALL update sites (also the ones checked 2 seconds ago).

avatar nikosdion
nikosdion - comment - 14 Jun 2014

@Bakual I deliberately followed the design of other com_installer models / controllers even though I don't particularly enjoy using JError. IMHO everything should be moved to exceptions. However, for consistency's sake (and saving the sanity of whoever will be maintaining the code in 1-2 years), it should be an all or nothing approach.

@nonumber @roland-d @jurianeven @brianteeman I disagree with purging before fetching updates. Let me explain this in small words:

  • You are a typical Joomla! user on a low-quality shared host with limited bandwidth, PHP maximum execution time and PHP memory time limit
  • Being a typical user, you have installed 30 components, 50 modules, 120 plugins and six templates, each one with its own update site definition for a grand total of 209 update sites (including those shipped with Joomla!).
  • Assuming an optimistic 1 second to contact each of those update sites and 300Kb of PHP memory required to parse them you need 209 seconds and just over 61 Mb of RAM
  • Your host gives you 30 seconds and 32 Mb of RAM. You can only process up to 25-30 update sites at once.
  • However, the last update time is committed for those first 25-30 sites.
  • Therefore you can go back and hit Find Updates again. The next 25-30 sites are processed. The first ones have a last update time less than the min time between checks, therefore they are not checked.
  • Rinse and repeat and you have all of your updates.
  • If you purge before you find updates you will NEVER, EVER be able to fetch updates
  • This happens AUTOMATICALLY in the main Control Panel page of Joomla!, thanks to the AJAX magic I had baked in Joomla! 1.7's "Extensions updates" icon plugin 2+ years ago. You're welcome.
  • In realistic circumstances you only have around 20 update sites, but the time to download and process each one is in the 2-3 seconds territory. This problem is REAL, not imaginary. I left this bit of info in the end to see how many of you bothered reading this description.
  • Oh, by the way, it's even worse when you are developing locally ;)
  • Therefore, purging the updates before finding updates is a stupid idea, unless we consider Joomla! to only run on high-end dedicated hosts, with sites created by people who are not overloading their sites with useless crap (hint: neither holds true in our current target audience)

So, do you people REALLY want me to replace my extremely carefully thought out Purge feature (which I had contributed to Joomla! 1.7 and some random committer mindlessly removed) with something that doesn't really work in the real world? I hope you're joking.

avatar roland-d
roland-d - comment - 14 Jun 2014

@nikosdion As I understood there would be 2 buttons, Find updates and Purge + Find updates. This would also cover the scenario of not purging before updating. As writing that, I realize it will be confusing for users to have 2 buttons that say Find updates. The Purge button would only be needed if there are unpublished sites.

So yeah, let's stick with your original proposal, just a Purge button :100:

avatar nikosdion
nikosdion - comment - 14 Jun 2014

Purge should only be needed in the very rare cases where Joomla! soils its pants, leaving ghost update records in the #__updates table. Yeah, it also has the side-effect of enabling update sites but the main reason of its existence is ghost update records. In some cases they're super-annoying: version 4.1.0 is out, the user installs it, Joomla! insists for two weeks that version 4.0.0 is the latest, user vents out to the poor developer. Been there, done that, now there's a button to cure this ;)

avatar betweenbrain
betweenbrain - comment - 14 Jun 2014

Based on @nikosdion's explanation, I'm a +1 for keeping the Purge feature as he explains it.

avatar jurianeven
jurianeven - comment - 14 Jun 2014

Yep leave it like that for KISS reasons (I think the UX idea of Purge and find updates is not bad, but then you have to split the requests with AJAX and so on to make it work, and even then it has to be fine tuned. Way too much hassle for something which isn't used often anyway.). Btw I've read your whole description, @nikosdion. Hopefully it'll ease the pain of explaining.

avatar roland-d
roland-d - comment - 14 Jun 2014

Now we can move the JC tracker to Ready for review?

avatar Bakual
Bakual - comment - 14 Jun 2014

I deliberately followed the design of other com_installer models / controllers even though I don't particularly enjoy using JError. IMHO everything should be moved to exceptions. However, for consistency's sake (and saving the sanity of whoever will be maintaining the code in 1-2 years), it should be an all or nothing approach.

I know we have that JError stuff still in many places. The goal is to remove them and the class is already deprecated in favor of native exceptions. So I fully agree with you that it should be exceptions where possible. If you can do it in that PR, it would sure guide others how to properly remove JError in other places. I would very much appreciate if you could do it :+1:

avatar Kubik-Rubik
Kubik-Rubik - comment - 14 Jun 2014

Very good PR, @nikosdion! :+1:

avatar nikosdion
nikosdion - comment - 14 Jun 2014

All right, there you go @Bakual Bonus code clarity improvement: eliminating redundant else-blocks, see Rafael Dhoms' keynote in JaB14/

avatar kennst
kennst - comment - 14 Jun 2014

@test OK

What should the user do when a component have several update sites? My testside includes now 3 update sites for JCE, with three different update sites ID. Is there an easy way to clean up the update sites?

I hoped this would fix issue 33482 (JoomlaCode). But it didn't. @nikosdion do you have a solution for this too? The trouble is with updating Akeeba backup and admin tools
Error: Package download failed: https://www.akeebabackup.com/component/ars/?view=download&id=2690&dlid=&dummy=my.zip

http://joomlacode.org/gf/project/joomla/tracker/?action=TrackerItemEdit&tracker_item_id=33482&start=0

avatar nikosdion
nikosdion - comment - 15 Jun 2014

@kennst A single extension can define multiple update sites. Likewise, many extensions can all be updated from the same update site. This is not a bug or a problem, it's how Joomla! works. There's nothing to "do".

Regarding JC33842 you need to upgrade to Joomla! 3.3 and enter your Download ID in the component's Options page. However, this is out of topic. If you need support with our software please file a support ticket on our site, Monday to Friday and me or a co-worker will reply as soon as possible.

avatar Bakual
Bakual - comment - 15 Jun 2014

Thanks for getting rid of JError, much appreciated :+1:

avatar larapetersen
larapetersen - comment - 15 Jun 2014

@test
Works as expected, both feature 1 & 2.

avatar beat
beat - comment - 15 Jun 2014

@nikosdion Great work and good improvement !

Wondering if "Purge" button could have a less scary and clearer name ? like e.g. "Refresh entries" or "Reactivate updates" ?

avatar infograf768
infograf768 - comment - 16 Jun 2014

@nikosdion

Still need to sync both lib_joomla.ini as stated in my first post.

Also, we have 2 "Purge updates" string values in core:
one in the installer, one in the library
en-GB.com_installer.ini:150: COM_INSTALLER_PURGED_UPDATES="Purged updates"
en-GB.lib_joomla.ini:491: JLIB_INSTALLER_PURGED_UPDATES="Purged updates"
en-GB.lib_joomla.ini:492: JLIB_INSTALLER_PURGED_UPDATES="Purged updates"

Looks like it is the library one used here; I suggest to change the value to something like:
"Updates successfully purged. Click on 'Find Updates'"

avatar nikosdion
nikosdion - comment - 16 Jun 2014

@infograf768 We are using both, in different contexts. The JLIB string is used in com_joomlaupdate, the COM_INSTALLER string is used in com_installer. I will remove the COM_INSTALLER strings to keep everything consistent.

avatar phproberto
phproberto - comment - 17 Jun 2014

Added a comment about removing an (in my opinion) unneeded function but :+1: here. I was very happy when the purge button was removed but I can understand why it's needed. Thanks for the explanation.

avatar losedk
losedk - comment - 9 Jul 2014

Just tested the functionality of the PR, seem to work as expected. Great job @nikosdion!

avatar Bakual Bakual - reference | - 9 Jul 14
avatar Bakual
Bakual - comment - 9 Jul 2014

Merged into 3.4. Thanks all for testing and especially @nikosdion for coding! :+1:

avatar Bakual Bakual - change - 9 Jul 2014
Status New Closed
Closed_Date 0000-00-00 00:00:00 2014-07-09 20:09:57
avatar Bakual Bakual - close - 9 Jul 2014
avatar Bakual Bakual - close - 9 Jul 2014
avatar betweenbrain
betweenbrain - comment - 10 Jul 2014

w00t!

Add a Comment

Login with GitHub to post a comment