? ?
avatar coolbung
coolbung
2 Jul 2016

With Joomla being used for a lot of serious web applications, a key need is to be able to use a "build pipeline" to progress application code from dev > QA > Production. While GIT can be very easily used for code, data that is stored in database cannot be progressed. To aid this, it is proposed to build a import/export tool that allows import and export of the following

  • Joomla Global Configuration
  • Extension configuration (i.e. the one that is managed via the Options menu)
  • Plugin configuration

The export tool can be accessible via a new menu "Export Configuration" in com_installer. The export should generate a Joomla package containing a XML manifest, which can be installed via the extension installer. The structure of the manifest is documented here - https://github.com/coolbung/joomla-cms/wiki/Manifest-Configuration-Export

To aid automation, a new CLI script should also be created.

avatar coolbung coolbung - open - 2 Jul 2016
avatar brianteeman brianteeman - change - 2 Jul 2016
Title
RFC Allow import export for configuration options
Allow import export for configuration options
avatar brianteeman brianteeman - change - 2 Jul 2016
Title
Allow import export for configuration options
RFC Allow import export for configuration options
avatar brianteeman brianteeman - change - 2 Jul 2016
Title
Allow import export for configuration options
RFC Allow import export for configuration options
avatar brianteeman brianteeman - change - 2 Jul 2016
Title
Allow import export for configuration options
RFC Allow import export for configuration options
avatar brianteeman brianteeman - change - 2 Jul 2016
Category Feature Request
avatar brianteeman brianteeman - change - 2 Jul 2016
Labels Added: ?
avatar brianteeman brianteeman - change - 2 Jul 2016
Title
RFC Allow import export for configuration options
Allow import export for configuration options
avatar coolbung coolbung - change - 2 Jul 2016
The description was changed
Title
RFC Allow import export for configuration options
Allow import export for configuration options
avatar bertmert
bertmert - comment - 2 Jul 2016

Maybe I don't understand the approach correctly:

When I look at your example XML I think that's not very practicable to export/import all configuration settings. What happens when you import a file like this and an extension plg_system_invitex was uninstalled meanwhile or some configurations are dependend on other plugins that add further configuration settings via JForm that is not installed?
What are the benefits compared with a database table(s) export/import and just copying the configuration.php or just a backup of the site. Is it reliable to identify extensions just by name and not by id (I don't know)?
Couldn't an import like this not be a killer in another server environment under some circumstances?
ACL settings?
Secret datas, db password etc.?
Settings for items that are dependent on extension settings?

Exporting/importing settings for a single extension out of the extension itself could be very helpful sometimes (e.g. managed by a plugin) but at the moment I don't see the benefits for the "normal user" yet to make a global settings export/import from the backend. I see more possible conflicts than advantages "if you don't know what you do".

avatar mbabker
mbabker - comment - 2 Jul 2016

IMO if you need the full site's configuration you're looking at a backup of at least the extensions and modules database tables and configuration.php (global config, extension level configs). It gets more complex if you pull in the ACL configuration too. I don't think there is a way that you can export/import globally like this without additional validations that the environment is the same (same installed extensions). And on import unless you are doing an overwrite of the existing data versus merge you're likely going to break something, especially on the ACL segment.

Next this isn't an extension so ? on the notion that it should be installed the same way you install an extension. You would need dedicated API & UI to handle this.

avatar brianteeman brianteeman - change - 3 Jul 2016
Labels Added: ?
avatar coolbung
coolbung - comment - 5 Jul 2016

@mbabker @bertmert agree with a lot of your thoughts about ensuring the environments are same. The ACL settings should surely be left out. Also, my thought was to simply export the configuration as it's stored in the database, so ensuring that the target environment is right is the onus of the developer. If a plugin is not installed on the target system, the configuration import will simply fail since there won't be a row in the extensions table.

However the issue that I'm trying to solve is that in a build pipeline dev > QA > Prod
there are some issues that are changes to a plugin/component configuration. What do you think is a good way to propagate these changes through the pipeline ? Something like database migrations, but specifically for configuration changes.

avatar mbabker
mbabker - comment - 5 Jul 2016

For your own projects, use something like this Migrations class to help push changes since the JSchemaChange* API isn't designed to cope with non-schema changes.

I guess another option might be transaction logging in the different environments and finding a way to export/import that.

Note neither of these are core solutions, and to be honest I don't think you can come up with a good solution that could/should be implemented in core. Config import/export IMO is a heavily specialized process that is too dependent on too many variables to be done at a high level. I guess with #10991 you could potentially use the database API for data import/export but you'd need to programmatically handle things still.

avatar brianteeman brianteeman - change - 28 Jul 2016
Status New Discussion
Labels
avatar brianteeman
brianteeman - comment - 4 Aug 2016

I am closing this based on the comments above that this really isnt something for the core.

avatar brianteeman brianteeman - change - 4 Aug 2016
Status Discussion Closed
Closed_Date 0000-00-00 00:00:00 2016-08-04 11:07:03
Closed_By brianteeman
Labels
avatar brianteeman brianteeman - close - 4 Aug 2016
avatar parthlawate
parthlawate - comment - 3 Sep 2016

The simple reason why i think this should be offered as an 'infrastructure' solution by Joomla is that it would avoid a lot of duplicate code in extensions downstream to Joomla. Most extension developers use Options as a prescribed way of storing extension settings.

However since Joomla doesn't offer a way to backup these settings or restore them, end up building their own solution. Offering this, like you said excluding stuff like permissions which is not in the express control of the extension will ensure their is a standard way to import/export these settings.. Maybe exporting the full site configuration per say is too complicated .. but just providing a dump and restore option on a per extension configuration level would not be that complex ?

Does that make sense ? @mbabker

avatar mbabker
mbabker - comment - 3 Sep 2016

For devs it's easy:

SELECT `params` FROM #__extensions WHERE extension_id IN (1, 2, 3);

For an end user interface, IMO it gets too complex. I stand by my arguments before that this is not something that belongs to core.

Add a Comment

Login with GitHub to post a comment