User tests: Successful: Unsuccessful:
This will partially solve #5274 by displaying a message in the Install Languages Manager if the Joomla! Accredited Translations site is disabled.
The message will be added to the default one when the _updates table is still empty ( you can empty it by going to the Update Manager and use Purge)
To test, enable and disable the site in Update sites Manager.
NOTE: The best solution would be to FORCE Purge/enable site/FindLanguages when loading the manager. If not proposed for 3.4.0, this PR will help.
Labels |
Added:
?
|
Category | ⇒ | Updating |
That would only be when someone clicks the Find Languages button, Thomas. It would not solve some of the issues described in #5274 where one gets weird results (like the weblink.xml pkg and/or a limited list.
This is why I suggested above to purge/enablesite/findlanguages
Forgive me for what probably is a stupid question, but, if "Accredited Joomla Translation server" is disabled on fresh install, where do the weird results are coming from?
Is anyone but you seeing the "weird results"?
On 2 December 2014 at 11:15, Sergio Manzi (smz) notifications@github.com
wrote:
Forgive me for what probably is a stupid question, but, if "Accredited
Joomla Translation server" is disabled on fresh install, where do the weird
results are coming from?β
Reply to this email directly or view it on GitHub
#5284 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
yes: @infograf768
That would only be when someone clicks the Find Languages button
I thought that's the time where languages are searched and the updateserver is used. If nobody presses the button, then the server isn't used and it doesn't matter if it's disabled.
The updateserver is probably also used when someone checks for updates, but that's a different view and affects all update servers there. There is no reason why the language update server should be threated differently than 3rd party extension server in the update view.
At least that's my understanding.
If it's only the issue that in rare cases the language list may become incomplete, we could as well just rename the button to "Refresh List" or something
Surry guys, but I'm not having hallucinations. Please, follow me with the following:
I performed a fresh install and then, even before logging in as Admin I inspected Joomla tables and this is what I found:
So, all updates sites seems to be enabled...
I checked if the _updates table was dirty:
nope! clean!
Then I logged in an proceeded to Extension Manager -> Update Sites:
Confirmed all enabled:
Then gone to "Install Languages". I won't show you the screenshot of that 'cause I've already posted it several times. I'll show you instead the content of _updates taken immediatly after:
Huston, we have a problem!
Don't you think that a ping @nikosdion can help?
further consideration: you can clearly see that the Update server used is the one with id=3, i.e. the one for "Accredited Joomla! Translation"
When you install what data set are you choosing
@brianteeman: data set = none
First, an observation. There are four (4) update sites installed and enabled with every single Joomla! 3 site. It's indifferent to your sample data. The actual SQL code which installs these update sites can be found in https://github.com/joomla/joomla-cms/blob/staging/installation/sql/mysql/joomla.sql#L1783 As you can see in the files inside administrator/components/com_admin/sql/updates/mysql the Joomla! Accredited Translations update site was added in Joomla! 2.5.7 and changed in Joomla! 3.0.0. The Joomla! Update Component update site was added in Joomla! 3.3.6 (I think me fucking up the restore.php has something to with it as @mbabker can attest). A fifth update site is added in Joomla! 3.4.0 for the Weblinks component which is now severed from the Joomla! core.
Now, why would an update site be automagically disabled? We come to the very reason I wrote the Update Sites page for the Extensions Manager.
When you access Joomla!'s back-end interface the two quick icon plugins will check for updates in Joomla! itself (eid=700) and other extensions (eid != 700). But what happens if Joomla! cannot fetch the contents of the update sites? You can found your answer in https://github.com/joomla/joomla-cms/blob/staging/libraries/joomla/updater/adapters/collection.php#L270 and https://github.com/joomla/joomla-cms/blob/staging/libraries/joomla/updater/adapters/extension.php#L238 The update sites are disabled immediately.
This behaviour in the JUpdater adapters is what lead to me writing the Update Sites page. We needed a way to re-enable the update sites. However, this is NOT enough! After enabling a site you need to Purge the updates (Manage Extensions, Update, Purge) and then fetch the updates again (Manage Extensions, Update, Find Updates). Only then will you see the full list of available updates, including languages available for installation.
Guys, I know that the updater makes less sense the more you dig into it. The thing is, we can't overhaul it before Joomla! 4.0. The other thing is, since there is no current planning for Joomla! 4 (goals, features, dates...) nobody is willing to spend any time on this. So we're stuck with this legacy updater solution with duplicated code and some obsolete design decisions. Don't get me wrong. Back in 2009/2010 when it was designed it was a fairly decent solution. Four to five years later? Not so much :(
@nikosdion: Nic, in my case no update server is disabled... problem is that on first access to the "Install Languages" page, the list is only partially populated and @infograf768 has seen it populated with non-relevant content...
@smanzi Then sorry, that's not in an area of the CMS I am deeply familiar with. Based on what you told me I'd say that the wrong update site is being read. I think @richard67 may be up to something. I'd also check the entire language installation page's logic to make sure it checks if the update site is enabled (if not, enable and fetch updates) and that there are updates for the expected extension_id present in the updates table. It should also be noted that using a hardcoded extension ID or update site ID is an invitation to major headaches, especially on sites migrated with extensions which do not maintain the ID column values.
@smanzi i made another video including the database tables - I just can NOT replicate the issue you are seeing
https://www.dropbox.com/s/m3tn01qvl5y35cc/langtest.mp4?dl=0
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5284.
@nikosdion Thanks and forgive me for "evoking" you in!
Maybe the strange results in the list which smanzi and infograf768 had are not created by the "find language updates" functionality but by the "find normal updates" functionality. As far as I could see in administrator/components/com_installer/models/languages.php, there is a way to find out which updates belong to language packs (by extension id = 600 and table #__update_sites_extensions, which tells the corresponding update site id), but i could not find any code to exclude the language pack updates from the list of all updates in the functionality for updating in general, so the normal updates still handles also the language updates (since theiy are in the same database table), so I guess the problem could be related to the initial check for updates after a new installation, which handles language packs in a wrong way. This is just an idea, but maybe it helps to locate where these strange partly filled lists come from.
@brianteeman I see... and really can't imagine why we are having different results...
FWIW, I'm on Windows 8, my WAMP stack is Uniform Server, I have PHP 5.3.29
From a methodological point of view I would say that while debugging a single positive and repeatable result has a higher weight than negative results, but this just my opinion...
I'd really love to see others with different configurations perform the same steps both you and me are performing and check if at least someone else is seeing the same things I'm seeing...
Really, I don't want to waste anybody's time, but from the evidence I have I'm profoundly convinced that a problem exists...
@richard67 from the fact that the_update table is populated with rows that have update_site_id=3 I'm inclined to think that this is not the case...
Are you doing ANYTHING different to the steps I showed in the video
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5284.
@brianteeman different username/password
@brianteeman your PHP?
I have tested this on localhost and TWO different hosts and still cannot replicate what you are seeing.
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5284.
can you tell me your PHP version? I wont to try with the same...
Fantastic! With exactly the same codebase (latest staging) uploaded to a CentOS VPS in Amsterdam (which is using almost the same PHP version, 5.3.28) I am not seeing the problem (i.e. same results as @brianteeman)
Repeated on localhost: same results as before, i.e. partial list.
I'm still more puzzled...
Now... the update server should be www.joomla.org, correct? This is seen at my localhost as having IP 72.29.124.146 How do you see it from your end?
no, from Amsterdam it is seen with the same IP, 72.29.124.146 this is not the cause...
In the update sites table, the location for the language updates site is "http://update.joomla.org/language/translationlist_3.xml", which has IP 198.232.124.192.
ah ok... let me check...
Please ensure 100000% that your local install is ALWAYS using a completely
new database
On 2 December 2014 at 14:55, Sergio Manzi (smz) notifications@github.com
wrote:
no, from Amsterdam it is seen with the same IP, 72.29.124.146 this is not
the cause...β
Reply to this email directly or view it on GitHub
#5284 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
yes, and completely empty folder (directory) before unpacking the installation package.
@brianteeman A.B.S.O.L.O.U.T.E.L.Y!
If you have TeamViewer I'll let you into my my PC and do it yourself...
@richard67: same as above!
ping me on skype Misterb1967
On 2 December 2014 at 15:00, Sergio Manzi (smz) notifications@github.com
wrote:
@richard67 https://github.com/richard67: same as above! [image: ]
β
Reply to this email directly or view it on GitHub
#5284 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
update.joomla.org is a Rochen CDN thing, but it seems to be comprised of a SINGLE IP which is 198.232.124.192 both here and in Amsterdam...
@smanzi Hmm, I am not really sure if I shall dare to ask if you cleared broswer cache ;-) Maybe your Apache on localhost sets high expiration times for XML files (mod_expires) so the browser still has cached an old one? You know, I ask such silly questions not because I think you don't know it yourself, I ask them because I myself often forgets such basic things, and so I think I should mention them.
@richard67 My FF is configured to clear everything (beside navigation history) every time I exit. I'm not sure about the local Apache config....
please contact me on skype and then we can teamviewer
On 2 December 2014 at 15:10, Sergio Manzi (smz) notifications@github.com
wrote:
@richard67 https://github.com/richard67 My FF is configured to clear
everything (beside navigation history) every time I exit. I'm not sure
about the local Apache config....β
Reply to this email directly or view it on GitHub
#5284 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
@brianteeman I added you to my Skype contacts but I still can't see you...
@brianteeman I think you must accept my invitation... or add me yourself: I'm "sergiomanzi"
I have not had a contact request from you
On 2 December 2014 at 15:12, richard67 notifications@github.com wrote:
@smanzi https://github.com/smanzi sorry, I deleted my comment about
broswer cache to which you answered, because my comment was silly. With the
last step of your test, the broswer has fetched the correct XML, so it
cannot be a cache issue.β
Reply to this email directly or view it on GitHub
#5284 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
@Γ #@#@! Try adding me on your side... sergiomanzi
As an update:
The conclusion I can draw is that in some circumstances the initial Languages List can be partially (and randomly) populated.
The exact circumstances bringing to this are yet to be defined...
maybe it is related to connection speed or latency to the update server?
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5284.
yes, makes sense, sorry for not having thought it through
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5284.
no prob! This is a real mistery!
Later I'll try on another localhost and connecting through a different network...
Maybe you should engage a dowser to look for waterlines or earth rays ;-) You know, you have this other issue with inconsistent title link, which also nobody else can reproduce ;-)
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5284.
Have you tried the title thing, Richard?
Not now, Dimitris... I'm going out to the pharmacy to grab something for my flu... will do later...
@dgt41 This is what I just started to try, too, to do the installation in another language than the default EN-GB.
@smanzi Yes, I've tried the title thing and cannot reproduce here. And I've checked the index.php of protostar uses $this->baseurl in any case. So I have no idea.
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5284.
I've re-tested again with fresh install of 3.4.0 alpha and language = german during installation, and I still cannot reproduce the problem. So here no difference with this.
@smanzi regarding the title thing I have switched SEF URLs off, but also cannot reproduce.
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5284.
This won't work. We cannot guarantee the language update server will always have ID 3; only for new sites installed after this feature was added (2.5.5 or 2.5.6 IIRC). The below screenshot comes from my site which was upgraded from 1.5 to 1.7 way back when.
We can use the name instead of the id, i.e. something like:
->where($db->quoteName('enabled') . ' = 0 AND ' . $db->quoteName('name') . '= "Accredited Joomla! Translations"');
But I still think that we should force Purge/enablesite/findlanguages when loading the manager...
Sergio, can you give us a screenshot of the language field expanded at the beginning of the installation?
This list has nothing to do with packs. These lang files are in the installation folder.
What has to do with packs is further in installation when one has the choice to install other languages than English en-GB.
@dgt41 Dimitris, I couldn't take a snapshot of the initial page as wished because I've been unable to convince the drop-down to not move while I was taking his portrait... Anyway I can assure you that the whole list of languages was there.
One thing I noticed is that the language automatically selected was en-US and not en-GB and this is probably because I'm using an en-US browser. I tried forcing en-GB and proceed with install. At the same time I used PHP 5.4.33 in place of the usual 5.3.29. No way: same problem.
I'm wondering if the en-US browser can have something to do with that (after all I'm visiting backend with it, and this could had in mysterious ways forced Joomla! to load a languages list...), but actually I don't think this is the reason, because, if it was, I have had the same problem when installing on my VPS in Amsterdam...
About the update sites I don't understand why are we disabling them automatically. I think we should revert the workflow. Sites will never be disabled automatically and the users can explicitly do it in the update servers management that @nikosdion contributed. I think this would save us a lot of issues and wouldn't break any site.
About the find languages issue I would enable update server automatically when clicking on Find languages
. If I'm clicking that button we can assume that I want to check for updates
So creating a enableUpdateServer on the model an call it from the findLanguages
method would be enough I think.
Also I'd maybe display a message in the extension updater like:
You have X update sites disabled. Click here to check them
So users are notified that something is happening and they can follow the bread crumbs to fix it
I wholeheartedly agree with all three suggestions @phproberto contributed.
I agree too. My proposal here was just a band aid
Go for it @phproberto
On 3 December 2014 at 09:11, infograf768 notifications@github.com wrote:
I agree too. My proposal here was just a band aid
β
Reply to this email directly or view it on GitHub
#5284 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
I'm on it ;)
I would leave the automatic disabling of update servers in.
The users probably just start complaining that the update fetching takes ages. You then have to tell them to find out which updateserver which isn't responding (maybe because it's setup wrong) and manually disabling it. I think it would be a pain to find out which updateserver is the one which doesn't respond to your system.
Personally I would create a quick icon check which shows red if an updateserver is disabled and maybe a message on the update view.
I would also enable the translation updateserver when pressing the "Find Languages" button.
I'm against automatic disabling: a temporary network issue can put all your servers on disabled and you'll probably even don't notice that...
If the feature is kept there should be a message in admin Home Page stating: "You have disabled update servers"
@bakual What you described is the common misconception of how Joomla! should behave. The reality is a bit different. If an update server is taking forever to respond you get a PHP timeout error (blank page) and the update server is not disabled. If, however, there's a temporary DNS or firewall issue with your server then Joomla! cannot access the update server, the update fetch returns immediately and the update site is disabled. Therefore this "protection" only disables update sites when it shouldn't and doesn't disable them when it should.
Please note that this comes from my 5+ years experience of providing Live Update in my components. If you want to disable update sites you need a simple, three step process:
The downside: more database queries
The upside: if a server is not responding for a long time and causes the update code to crash with a timeout error the update site remains disabled until the user manually enabled it again. Which is exactly what you want it to do.
Again, this is field-tested in dozens of thousands of sites over the course of 5+ years.
What you described is the common misconception of how Joomla! should behave. The reality is a bit different. If an update server is taking forever to respond you get a PHP timeout error (blank page) and the update server is not disabled. If, however, there's a temporary DNS or firewall issue with your server then Joomla! cannot access the update server, the update fetch returns immediately and the update site is disabled. Therefore this "protection" only disables update sites when it shouldn't and doesn't disable them when it should.
Hmm, the only time I had a disabled update server so far was when it was setup wrong: It pointed to a non-existant page which returned a 404 I think. Disabling it was perfectly fine for that case.
when the connection to an update site fails an error is shown to the users so they can disable them manually anytime.
Hmm true. That would change. Currently the message would be shown as well, but you rarely actually see it because of the update check done using AJAX from the cPanel Quick Icon. Once you go to the update view, the server is already disabled and you don't see that message anymore.
With your approach, one would see the message each time you go to the update site. That would indeed be an improvement.
I still don't like to remove the automatic disabling but it's less an issue given that.
I've replied to the PR with a detailed explanation of why the disable-fetch-enable actually solves your issues. Please remember that you have experience using Joomla! on VERY GOOD HOSTS. This is not your typical Joomla! user experience. I had to provide support to people on less-than-ideal hosts (to put it mildly). You wouldn't believe how many hosts require you to give them a complete list of the IP addresses of remote hosts you want your site to connect to. Lots of "fun" when you're using a CDN for your updates... It's also amazing how many hosts have intermittent DNS resolution and routing issues. I've gotten to the point where accessing the main admin page of my components forcibly enables the update site for the component to prevent 150+ support requests per month. It ain't fun and I can't keep telling people that Joomla!'s update system sucks. It's FOSS, it doesn't have to be broken when people know how to fix it!
@infograf768 You wrote above that you have changed this PR to use the name instead of the update site id. Should then not other parts of Joomla, which use the hard-coded extention ID 600 for the same purpose (identify which updates are language updates), be changed in the same way? One example which I mentioned in one of my comments above is administrator/components/com_installer/models/languages.php. If you want, I can make a PR for those I can find, and refer in the new PR this one here so we know where the idea for the change came from and why, but I would like to have your OK if this is the way to go, use the name everywhere we use harcoded ids to distingusih betwen normal updates and language updates (or normal update sites and language update sites). So what's your opinion?
Again @infograf768 : This is what I have found with a findstr on a local copy of staging (created just a few minutes ago) regarding hard-coded extension id 600 to check for language updates:
In administrator\components\com_installer\models\languages.php:
In installation\model\languages.php:
So we would have to change these 2 files for using names and not ids.
Shall I do this in a new PR?
Or shoud we change $updater->findUpdates ?
Beside this: Maybe the installation\model\languages.php is the one which creates in some cases the strange list?
@infograf768 Of course the best way would be to extend the updated by a function "findLanguageUpdate" or so, where then the database query is restricted by the name, and the 2 calls mentioned above are replaced by calls to this new function. The $updater->findUpdates should not be touched.
This comment was created with the J!Tracker Application at issues.joomla.org/joomla-cms/5284.
By the way, there is only one other point where I have seen a hard-coded extension id used with a call of updater->findUpdates. It is administrator\components\com_joomlaupdate\models\default.php, which calls it with a "700". Is this a reserved ID for the Joomla CMS updates, so we can leave it hard-coded? Couldn't we make then the 600 for the EN-GB language pack be a reserved ID, too, and change this PR again? Or shall we make com_joomlaupdate use the name, too? We could extend the updater by a function findUpdatesByExtensionName (or so) in this case, which also could be used for the languages instead of my pervious proposal for a findLanguageUpdate function. I think this depends on the basic idea for Joomla if for such cases the names or the ids shall be the constant thing.
Please remember that you have experience using Joomla! on VERY GOOD HOSTS. This is not your typical Joomla! user experience.
The server I use is actually the cheapest I found in Switzerland. Apparently itβs still better than some more expensive hosts out there
... then it's a very good host!
I've seen many $4/month hosts being better than some $20/month ones. It's all about the competence of the enginnering staff ;)
Beside this: Maybe the installation\model\languages.php is the one which creates in some cases the strange list? Maybe because called with cache time = zero?
I don't think so: in my "possessed" localhost #_updates
is absolutely empty when I first login and it is being populated by the weird list the exact instant I first access /administrator/index.php?option=com_installer&view=languages
Just a question: beside me, is there anybody else out there using "Uniform server"? Please raise his/her hand...
yeah... I understand, but I can't comment about that...
@smanzi But maybe you agree with me that it is inconsistent to use the hard-coded name here and the hard-coded extension id there?
And btw, what would you recommend to install on my PC for contributing and testing: Git for Windows, Tortiose or Eclipse? With Tortiose I have a bit experience, have used it once.
... but for the few I understand I think that en-GB having ID=600 is a pretty safe assumption, maybe unless if there are locally built versions for other languages. In this can be a case... I don't know
I'm pretty conservative and I use Git for Windows and Notepad++, but I'm sure there are better options...
another tool that complement my "dev stack" and I can't live without is "Beyond Compare", by Scooter Software, but that's commercial.
Regarding id=700, it's my fault. I was working under a very tight schedule to get com_joomlaupdate included in Joomla!. I took the same shortcut as the then-existing updater, using a hardcoded extension ID.
Please note that in 99% of cases using a hardcoded ID is not a problem. The problem only exists on sites which have been migrated from earlier Joomla! versions or transferred with tools which do not respect the extension IDs.
@nikosdion @infograf768 so we should leave things like they are? But then the pull request here is incosistent to all the rest because using the update site name and not the extension id = 600.
I'm in favour of using extension names AND types (and folder, for plugins) to retrieve updates. Or, at the very least, change all harcoded instances of 600 and 700 with a db query which fetches the actual extension ID.
@nikosdion Hmm, but then fetch the ID by what? By the name? I mean what if in Joomla 4.0 for example "Joomla!" will be changed to "Joomla?" (ok, silly example but you I think know what I mean.)
Ahem... You need more coffee / soda. SELECT extension_id FROM #__extensions WHERE type='file' AND element='joomla'
I am pretty sure that this won't change ;)
@nikosdion ok, this for the 700. And for the 600?
SELECT extension_id FROM #__extensions WHERE type='language' AND element='en-GB' AND client_id=0 ?
But where should we place these queries? In those scripts calling findUpdates (see list in my comment somewhere above)? Or should there be some function in the updater for this?
Yes and yes to both your questions.
@nikosdion yes on which of the alternatives for the 2nd quiestion? Calling scripts or updater? And can we rely on en-GB always being installed?
Everywhere where findUpdates is used with a hardcoded ID.
Regarding en-GB, you cannot uninstall it. It is a locked extension. If you remove it a lot of things will break in Joomla! and its extensions, that's why it's locked.
Status | Pending | ⇒ | Closed |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2014-12-03 16:57:26 |
OK, this thread is becoming a bit a "catchall" and I hope @infograf768 will not be upset... if this is the case, please just let me know:
I want to report that on another IDENTICAL (just copied, it is portable...) installation of Uniform Server, but running over Windows 7 Ultimate (instead of Windows 8 Pro)... I don't see "partially populated" languages list issue: just a neat clean empty list. At this point the cases can be:
What it isn't is case of hallucination: @brianteeman can confirm...
In any case I would say: "the heck with that!". We have all lost too much time on this: just let's remember that it might be a (small) issue that we will (very unlikely) see in the future...
Agreed?
Oops... thread closed in the meanwhile... probably better!
Best would imho be to add
$model->enableSites();
to the controller in the find method here:https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_installer/controllers/languages.php#L30
This would enable all updateSites automatically when someone presses the "Find Languages" button. There would be no need for an additional message then imho.
We could even adjust the enableSites method a bit to allow passing a parameter which would enable a specific update site only (eg the language one), since that's the only one we really want to have enabled for sure in this case.