User tests: Successful: Unsuccessful:
This PR introduces a command-line interface (CLI) extensions installer. This allows automation of extension installation (new installs and updates) using command line tools. It can be used on various scenarios including, but not limited to, virtual machine provisioning (see Vagrant), automated tests for extension integration, mass updates of hundreds of sites in a hosting environment and so on. The script in this PR offers an equivalent to all of the installation methods allowed by the Extensions Manager in the back-end of your Joomla! site.
For testing purposes we are going to install the same dummy extension in several ways. After each installation please check that the extension is installed on your site. Then remove it and test the next method. There are eight tests in total, including the short and long command versions. Important: Being familiar with the command line environment is necessary, given the nature of this PR.
Run this script as
php /path/to/site/cli/install-extension.php -w "http://akeebatest.s3.amazonaws.com/dummy/update.xml"
Also try the long command version
php /path/to/site/cli/install-extension.php --web="http://akeebatest.s3.amazonaws.com/dummy/update.xml"
Run this script as
php /path/to/site/cli/install-extension.php -u "http://akeebatest.s3.amazonaws.com/dummy/com_dummy-1.2.0.a1.zip"
Also try the long command version
php /path/to/site/cli/install-extension.php --url="http://akeebatest.s3.amazonaws.com/dummy/com_dummy-1.2.0.a1.zip"
Download the installation package into a directory, let's call it /foo/bar
. DO NOT extract it! Now run
php /path/to/site/cli/install-extension.php -p "/foo/bar/com_dummy-1.2.0.a1.zip"
Also try the long command version
php /path/to/site/cli/install-extension.php --package="/foo/bar/com_dummy-1.2.0.a1.zip"
Download the installation package AND extract it into a directory, let's call it /foo/bar
. Now run
php /path/to/site/cli/install-extension.php -f "/foo/bar/com_dummy-1.2.0.a1.zip"
Also try the long command version
php /path/to/site/cli/install-extension.php --folder="/foo/bar/com_dummy-1.2.0.a1.zip"
No b/c issues are present. This is a new CLI script which doesn't interfere with existing functionality.
One new translation file added (language/en-GB/en-GB.cli_install_extension.ini) with 22 language strings. Expected translation impact: about 8 minutes.
This PR is inspired by the CLI extension installation scripts written by Nicola Galgano and yours truly. It seems that many of us are reinventing the same small but very convenient wheel.
Labels |
Added:
?
|
I am testing with current staging branch, #5429 applied, php 5.4.16 (linux/windows), mysql/postgresql:
With linux/postgresql I get the following messages when executing
php install-extension.php -f "/test/com_dummmy-1.2.0.a1.zip":
Installing with method 'folder'
Extracting package file /test/com_dummy-1.2.0.a1.zip
Installing from /test/install_5491bcdcbe19f
Warning: get_html_translation_table() expects at most 2 parameters, 3 given in /var/www/html/joomla33/libraries/joomla/filter/input.php on line 698
Warning: Invalid argument supplied for foreach() in /var/www/html/joomla33/libraries/joomla/filter/input.php on line 700
Warning: strtr(): The second argument is not an array in /var/www/html/joomla33/libraries/joomla/filter/input.php on line 706
Warning: get_html_translation_table() expects at most 2 parameters, 3 given in /var/www/html/joomla33/libraries/joomla/filter/input.php on line 698
Warning: Invalid argument supplied for foreach() in /var/www/html/joomla33/libraries/joomla/filter/input.php on line 700
Warning: strtr(): The second argument is not an array in /var/www/html/joomla33/libraries/joomla/filter/input.php on line 706
Warning: get_html_translation_table() expects at most 2 parameters, 3 given in /var/www/html/joomla33/libraries/joomla/filter/input.php on line 698
Warning: Invalid argument supplied for foreach() in /var/www/html/joomla33/libraries/joomla/filter/input.php on line 700
Warning: strtr(): The second argument is not an array in /var/www/html/joomla33/libraries/joomla/filter/input.php on line 706
Extension successfully installed
The component is installed but gives a "404 component not found" when invoked.
Under windows it seems that the an absolute path to the package/folder is not supported, or am I doing something wrong when I execute
php install-extension.php -p "c:\test\common_dummy-1.2.0.a1.zip" ?
The dummy component isn't really designed to install on PostgreSQL. To be honest, apart from Akeeba Backup I am not aware of any other major component which can do as much as install on PostgreSQL. And since the PostgreSQL integration was FUBAR in all previous versions we couldn't even properly test our components against it. That said, I do believe that if you try using Akeeba Backup Core instead of the dummy component it will work. If not, we'll need to take a look at why JInstaller breaks under Postgres which is outside the scope of this PR.
Regarding Windows paths, I suspect the problem is the backslash. Try substituting backslashes with forward slashes, i.e. "C:\Somewhere\Foo.zip" becomes "C:/Somewhere/Foo.zip". If this doesn't work let me know. I will have access to my Windows machine tomorrow morning.
Chris: It seems to me that the other CLI scripts are not translated. If we need to provide translation strings someone will have to tell me where to put the language files. Back-end? Front-end? What will be the naming convention? Something like en-GB.cli_extension_install.ini? And how exactly does Joomla! know which language to use in the CLI when it does not detect the system language? Whatever is the default language of the site? Doesn't this create a whole lot of issues when an English-speaking-only engineer is installing extensions to a Spanish site?
It seems to me that the PLT needs to have a long discussion with the Translation team and decide what's best. My vote would be a new language directory under CLI and a new translation package for CLI apps. Also we should standardise on using –say– the back-end translations for the library and core components in the CLI apps. Basically, we need a translation strategy for the CLI apps because there's currently none that I know of.=
FWIW the Smart Search CLI (https://github.com/joomla/joomla-cms/blob/staging/cli/finder_indexer.php) is translated. en-GB.finder_cli in the site language folder.
@waader - If you're looking for an extension that will install (and theoretically work) on PostgreSQL or SQL Server, my own should work. Try https://github.com/BabDev/Podcast-Manager/releases/download/2.1.0/pkg_podcastmanager_2.1.0.zip
@nikosdion Your code already loads the administrator "lib_joomla" and "com_installer" language files, although I haven't checked to see if it really does load them or not.
But you raise some good questions. Looking through the existing CLI scripts, two do not appear to be translated at all (garbagecron.php and update_cron.php, although to be fair garbagecron.php doesn't appear to have any strings to be translated anyway). There are two that are translated: deletefiles.php loads the files_joomla.sys file from front-end and as Michael pointed out, finder_indexer.php loads finder_cli also from the front-end.
I'd prefer to keep all the language files in one or other of the two current locations rather than creating a new one just for CLI apps, but I'd be open to discussion on that point. The current CLI apps have set a precedent of using the front-end language folder, so I'd go with that for now. A naming convention would be useful and using "cli_" as a prefix makes sense, so I'd suggest cli_install_extension in this case (since the program is install-extension.php).
What would be nice is if we could agree on using --language as a command-line argument for all our CLI apps so that a different UI language can be loaded for the app. That would cover your English-speaking engineer installing on a Spanish site.
What do you think?
@mbabker and @chrisdavenport: Thank you for the feedback! Here is the CLI script with proper translation.
@chrisdavenport Regarding the --language parameter, I'd rather JApplicationCli detecting the current system locale instead. It's the POSIX way to communicate to CLI applications your language preference.
@nikosdion Okay, sounds fair. Thanks for working on this.
Category | ⇒ | CLI |
@nikosdion thanks for unify all the small "wheels" in a modern tire.
just one small consideration sometimes i've found usefull to group on a file all the extensions to be installed
so is possible to install a bounch of extensions with one command only
@alikon I saw that in your code and there's a reason why I did not implement it in PHP. You can install multiple extensions listed in a file without reinventing the wheel. Let's say you have a file with extension download URLs in /file/with/urls
. You just have to use this simple command line:
for e in `cat /file/with/urls`; do php /path/to/site/cli/install-extension.php -u $e; done
If you're on Windows and are about to complain that you don't have Bash to play with I'll give you two alternatives: a. Install cygwin or b. Install Git for Windows. They both install Bash. Make your life easier by also installing ConEmu to manage virtual consoles.
The beauty of this solution is that you can spin up a bash console and have access to both bash internal –also many GNU– commands and Windows executables. That's how I maintain my sanity when I have to work with Windows.
i can agree with you, but sometimes you simply are not allowed to install other "software" due to some restrictive policies on the target machine environment (ie. intranet)
Challenge accepted. The same thing using the plain old Windows command prompt available since the olden MS-DOS days:
FOR /F "tokens=*" %e IN (c:\your\file.txt) DO php C:\PATH\TO\SITE\cli\install-extension.php -u "%e"
No third party software or PHP code required, just mastering the use of the Force... er... command line.
I forgot to say that FOR /F has many more options. Take a look at http://ss64.com/nt/for_f.html It's easy to have a CSV file with lines like this:
;method,source
p,c:\somewhere\package.zip
u,http://www.example.com/package.zip
w,http://www.example.net/foobar_updates.xml
and write a single for loop to install the whole lot, e.g.
FOR /F "tokens=1,2 delims=, eol=;" %i IN (c:\your\file.txt) DO php C:\PATH\TO\SITE\cli\install-extension.php -%i "%j"
As I was asked by Chris my thought about this, here they are:
As Michael said, we already use: xx-XX.finder_cli.ini
Therefore I suggest
OR to use en-GB.install_extension_cli.ini
OR change en-GB.finder_cli.ini to en-GB.cli_finder.ini (+ related code + delete the older file in script.php) and keep en-GB.cli_install_extension.ini
NOTE:
Any new ini file added in core has incidences on com_localise component. It requires a bit more in the case of files which are not extension-dependent. It is not an issue as we can update com_localise.
I prefer having "cli" as a prefix instead of as a suffix. It makes it easier for humans to locate files. As for the Finder CLI translation, if you rename it for consistency you'll end up having to do a lot of unnecessary work including letting all TTs know they have to rename the file; write a new Joomla! upgrade script method to remove the obsolete file; create an installation script for each and every language to remove the obsolete file; change the finder CLI script; change the localisation component. I'm not convinced that this is a good way to expend scant resources.
com_localise will anyway have to be updated. Pity nobody thought a bit more ahead when creating finder_cli.ini. I guess that's life.
If only I had a penny for any architecture decision I've regretted two years down the line...
@nikosdion I struggled with this line
php install-extension.php -p "c:\test\common_dummy-1.2.0.a1.zip"
under windows. I found out that when applying the filter "path" the value could not be retrieved. So I changed it - for testing purpose - to "raw".
I also change in line 254 "file" to "package". After that changes package installation works under windows.
In line 254 there is a type - see "langauge". And I think that the comma in line 128 is not necessary.
@test success
should go with this one #2764 too
so you can setup all from CLI
Testing the installation of com_dummy-1.2.0.a1.zip and other components using the different methods went fine.
But I got one problem with the installation of acymailing (com_acymailing_starter_v4.8.1_2015-01-05_13-19-06.zip). Doing this
php install-extension.php -f "c:\com_acymailing_starter_v4.8.1_2015-01-05_13-19-06.zip"
gives the following fatal error: Call to undefined method JoomlaExtensionInstallerCli::isAdmin() in .... joomla34\administrator\components\com_acymailing\helper\helper.php on line 863.
On the other hand: the installation is successful in the backend.
Can somebody more knowledgeable assess this error?
It means they expect an instance of JApplicationCms
when you call JFactory::getApplication()
which contains that function (https://github.com/joomla/joomla-cms/blob/staging/libraries/cms/application/cms.php#L663). What we actually have is the cli application instance (https://github.com/joomla/joomla-cms/pull/5460/files#diff-f85f920df9966deeeae69694208c8acdR441) so it fatals. Nic this one is one for you to work out a solution for I guess.
Technically, this is an architectural issue. I can work around it by adding a dummy method in the CLI script’s class. Is that an acceptable solution?=
How far would this run though. I'm thinking logically the 3 methods that people would likely use in an install script are isSite
, isAdmin
and enqeueMessage
. I mean I know technically it could be anything. But I'd imagine these 3 would regularly appear
Meanwhile I tested with Kunena 3.0.6 which gives me another fatal error:
Call to undefined method JoomlaExtensionInstallerCli:getRouter() in ... \joomla34\libraries\joomla\application\route.php on line 47
I guess we have to deal with this issue architecturally instead of going through a patchwork of workarounds. Having an application interface which really includes all methods developers expect an application object to have and refactoring JApplicationCli / JApplicationWeb is the only way to realistically go forward :(=
I was just typing something similar; an interface will definitely help here.
I guess that this would have to wait for Joomla! 3.5 (I can see a probability for b/c issues there) and, as a result, the CLI extensions installer would also need to wait until then.=
@nikosdion
Shall we consider this PR as postponed to 3.5?
I need to be sure as I would have to upgrade com_localise for this new ini.
It would be a real shame if it was delayed
Yes, it has to be delayed. There’s no point shipping broken code just to make it through the 3.4 deadline. We need to deal with the JApplication* classes’ architecture before we can include the CLI extensions installer script.
Of course, if you want to use it on your sites you still can: just check out the file and put it in your site’s cli directory. The script is self-contained. I just can’t vouch it will work with any extension as you can read in this issue thread.=
:(
On 6 January 2015 at 10:08, Nicholas K. Dionysopoulos <
notifications@github.com> wrote:
Yes, it has to be delayed. There’s no point shipping broken code just to
make it through the 3.4 deadline. We need to deal with the JApplication*
classes’ architecture before we can include the CLI extensions installer
script.Of course, if you want to use it on your sites you still can: just check
out the file and put it in your site’s cli directory. The script is
self-contained. I just can’t vouch it will work with any extension as you
can read in this issue thread.=—
Reply to this email directly or view it on GitHub
#5460 (comment).
Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/
Milestone |
Added: |
Closing this PR as it requires a major core refactor which can't happen before Joomla! 4.0.
Status | Pending | ⇒ | Closed |
Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2015-06-22 13:21:19 |
Closed_By | ⇒ | nikosdion |
Labels |
Added:
?
|
@marcodings You should review for Joomla 4
Labels |
Added:
?
Removed: ? ? |
Labels |
Added:
?
?
|
I've added the label Re-evaluate for Joomla 4 as well
Milestone |
Added: |
Milestone |
Added: |
Looks good (haven't tested it yet), but I think translation strings will need to be added.