The only viable solution I see atm
is to always include the extensions language strings in the full CMS. This means that any change in an extension language strings would only be done when we release a new version of the CMS.
This will let TTs include the strings whether they are used or not.

@infograf768 we can't have any files for these decoupled extensions in the CMS at all (that's what decoupled means). We'll need to work through the issues and see if we can come up with innovative and efficient solutions.

Is there any reason why we can't use transifex to manage our translations (we could even try and link the existing Joomla translating component into their API???).

Then posting latest versions of language packs becomes our responsibility and also gives us the ability to separate out any language packs that we don't want in the core system!

So I am not familiar with how translation files are maintained anymore, but
in terms of distribution, I expect all language files for all translations
will live in this repository. And this will be the case for each extension
suite. How that can work with how Transifex works, I'm not sure but I'd be
happy to talk to them about ways we can make this as efficient and painless
as possible.

A viable solution would be our own translation site, which I described earlier to other people in the community. We can't keep the translation files in the CMS and especially since com_weblinks wont be the last extension that we make standalone and hopefully since there will be new standalone, community maintained extensions, we have to implement our own translation site.

A viable solution would be our own translation site, which I described earlier to other people in the community.

When it's available, I'll be sure to use it :) But we need this all
done and dusted by May 28.

Transifex (more precisely OpenTranslators) works quite well for extensions. It's what many extension developers already use for quite some time. You can also automate the up- and download of the language files. I think Michael does that already with JIssues.

The real problem here is not so much technical but more about who will do it. Currently we have excellent language teams which make sure that the translations are up to date for the releases or shortly after. We have that for around 60 languages if I'm right.
That's only because we have dedicated teams and a single place where the files are located, using their preferred tools to translate the strings.
Moving to Transifex may work, but may mean that we lose on quantity or quality because the teams will concentrate on the remaining core files.
Also Transifex has some limitations as I understand it.

I think with the current translation structure, we may have to keep the language files managed using the core language packs. That will work the same it does currently for the webinstaller plugin.
It's not exactly nice but will work.
I think changing the translation process will be a big task which likely goes beyond the timeline for 3.4 😄

The TT coordination has already explained multiple times why we can't use Transifex:
The number of strings in a said language can change depending on the plural forms.
Also some strings are explained by comments and others may or may not be commented. Therefore the comments have to display.

Also has to be taken into account the fact that TTs may be working on 3 instances of a language at the same time. For example: 2.5.19, 3.2.3 and 3.3.0 dev as of today...
Then they have to make the packs (updates or new) and release.

@wilsonge
Could you explain "also gives us the ability to separate out any language packs that we don't want in the core system!" ?

@eddieajau
The way it works as of now is that core lang packs are posted by TTs on a specific repository on joomlacode and then a cron job makes them available to users in the admin interface via Update or Install languages.
Each Team choses the way they work together (Github, SVN, Text Editor, Webtranslateit, com_localise corrected manually,Transifex own project manually corrected, etc.).
Separating some "core supported" extensions lang files from the rest of core would mean that TTs have to produce their work differently. I am afraid of the confusion in the workflow and the ability by some to use github.

Separating some "core supported" extensions lang files from the rest of core
would mean that TTs have to produce their work differently. I am afraid of
the confusion in the workflow and the ability by some to use github.

Yep. I'm asking you to keep an open mind and embrace this as a problem
we should try to solve. Let's exhaust all possibilities before
throwing in the towel.

I'm not saying to use Github. I'm not saying that we should make it more complicated for them. What I am saying is, that they have to adapt to our translation workflow most likely. We simply can't make everybodys wish come true and implement every possible workflow out there.

Sure. I think it's a good idea but I would not want it to hold up this
particular feature (if you can call it that). And yes, I'd like everyone to
think out of the box on every aspect of how this extension gets to market.

Language strings are part of the b/c promise of the new development strategy, thus they can't be removed in J3.

@nicksavov, language strings for a component that is removed from core can also be removed. There is no BC issue here. The BC promise on language strings is that the keys won't change.

@infograf768 Wouldn't it make the lives of the core TT's a lot easier if they had a lot fewer extensions to translate? If we offload the translation of decoupled components to a 3rd party, such as Transifex or some other provider, wouldn't that make it the whole process simpler for everyone?

I have been asked by my collegues of OpenTranslators to clear up some misunderstandings about Transifex (which you can use without OpenTranslators' help, by the way)

1. OpenTranslators and Transifex are separate things. OT is a Joomla community project, USING Transifex as a translation tool.
2. Transifex does support comments on translations files. It automatically imports them from .PO files. For Joomla's .ini files you can add them manually or use the Source String API.
3. The multiple version story is irrelevant, since you're not translating JOOMLA but a single extension, where one version will suffice. Otherwise, you could add different sources per extension.
4. Using Transifex and CTransifex making language packs is extremely simple. Or you could use the Transifex client (CTransifex more for "end users", really easy but really powerful).

On top of that, Transifex offers a ton of advantages for both translators and developers, but you've got to keep an open mind to appreciate them.

Based on our three years of experience translating Joomla extensions with Transifex, and looking at the feedback of our 2500+ volunteers, we think ew can conclude that Transifex makes translating extensions easier, and Transifex has been very open to suggestions we've made in the past. Their tool keeps improving every day. No need to get stuck in the past.

Could you explain "also gives us the ability to separate out any language packs that we don't want in the core system!" ?

So transifex means we can separate out the language files a bit which should mean we have more flexibility in packaging them together. We can package them all together split them up into core + individual components etc. however we like :)

JM can you explain a bit more about The number of strings in a said language can change depending on the plural forms. because it's fairly clear that translations comments aren't a problem in transifex :)

I'm pretty sure what JM is referring to is the ability to quickly uncomment a line, then translate its string, e.g. checkout lines 37-49 of:
https://github.com/joomla/joomla-cms/blob/master/administrator/language/en-GB/en-GB.plg_captcha_recaptcha.ini#L37

Is that possible?

For the quote about plurals, check out:
http://docs.joomla.org/J3.2:Making_a_Language_Pack_for_Joomla#the_fr-FR.localise.php

Define specific plural functionality for some languages where the value of the string can change depending on the count (Russian for example).

@dongilbert
Would be much easier for TTs to not have to translate anything...
Seriously now:
It is the decision of the PLT, when decoupling anything or for new "core supported" extensions to go on trusting the TTs OR to choose a system where any volunteer, who may or may not translate stuff the best way and in sync with the rest of that language usage and grammar, will provide translations when they have time...
I have seen 3pd translations done on Transifex (OT) using Google Translate... Unreadable.

That proposal was already made a few years ago for the whole core and was turned down as we can't depend on single unknown volunteers for core stuff translating.
=> Language translation is NOT like code.
My opinion is that any "core supported" extension should be translated by the TTs to make sure we get the quality needed and that we get translations for ALL core packs.
To "offload", as you say, would therefore mean that all the TTs would have to use a specific platform only for these files (as I said, Transifex does not fit and we have been waiting for ages for major/senior Joomla devs to spare a bit of their time to help us concerning languages translations...

@toretto
The experience of a 3pd dev who does not use plurals and does not have to make sure that the indications in a commented line are read and understood or lines uncommented when translating are evidently very different from core.
And btw, forget .po. We use .ini.

@wilsonge
Please look at
https://github.com/joomla/joomla-cms/blob/staging/administrator/language/en-GB/en-GB.plg_captcha_recaptcha.ini
from line 37 on.
also a recent one:
https://github.com/joomla/joomla-cms/blob/staging/administrator/language/en-GB/en-GB.plg_twofactorauth_totp.ini
line 20 to 24
If one does not read the comments or be able to uncomment, they can't adapt their translations.

Concerning plurals, this is the Russian plural:

public static function getPluralSuffixes($count)
    {
        if ($count == 0) {
            $return = array('0');
        } else {
            $return = array(($count%10==1 && $count%100!=11 ? '1' : ($count%10>=2 && $count%10<=4 && ($count%100<10 || $count%100>=20) ? '2' : 'MORE')));
        }
        return $return;
    }

An this is the Scottish gaelic one

public static function getPluralSuffixes($count) {
        if ($count == 0 || $count > 19) {
            $return =  array('0');
        }
        elseif($count == 1 || $count == 11) {
               $return =  array('1');
        }
        elseif($count == 2 || $count == 12) {
               $return =  array('2');
        }
        elseif(($count > 2 && $count < 12) || ($count > 12 && $count < 19)) {
                $return =  array('FEW');
        }
        return $return;
     }

Now compare with en-GB:

public static function getPluralSuffixes($count) {
        if ($count == 0) {
            $return =  array('0');
        }
        elseif($count == 1) {
            $return =  array('1');
        }
        else {
            $return = array('MORE');
        }
        return $return;
    }

This means that when we have in en-GB:
COM_BANNERS_BANNERS_N_ITEMS_CHECKED_IN_0="No banner successfully checked in"
COM_BANNERS_BANNERS_N_ITEMS_CHECKED_IN_1="%d banner successfully checked in"
COM_BANNERS_BANNERS_N_ITEMS_CHECKED_IN_MORE="%d banners successfully checked in"

In Russian they need:
COM_BANNERS_BANNERS_N_ITEMS_CHECKED_IN_0="Ни один баннер не был разблокирован"
COM_BANNERS_BANNERS_N_ITEMS_CHECKED_IN_1="%d баннер успешно разблокирован"
COM_BANNERS_BANNERS_N_ITEMS_CHECKED_IN_2="%d баннера успешно разблокировано"
COM_BANNERS_BANNERS_N_ITEMS_CHECKED_IN_MORE="%d баннеров успешно разблокировано"

This means one string more in both cases for plurals.
These, at last I saw, can't be added in Transifex and comments can't be read or lines uncommented

We can't remove com_weblinks from the core. What we can do, is making it possible to uninstall com_weblinks and provide a seperate update package for com_weblinks. Simply said: com_weblinks will be installed by default, but you can uninstall it and it is guaranteed, that a Joomla update does not add in those files again.

"And btw, forget .po. We use .ini."

@jm - isn't the JTracker Joomla project using .pot files? on Transifex?

JTracker is not joomla...

I apologise JM - I used the name of the .pot file rather than the project

https://github.com/joomla/jissues/blob/master/templates/JTracker/g11n/templates/JTracker.pot

I have been collecting ideas about what a translation system that can be seen here: https://github.com/joomla-projects/translate-joomla

Anyway, after thinking and thinking I always hit a wall: I think there is not the perfect solution there is always challenges. So my suggestion is that we start solving one by one the problems.

For me the very first step would be:

• Move translation packages from joomlacode to github, see this example repository: https://github.com/joomla-projects/en-US_j3. This move has several benefits:
  • allow translators to use the Github UI online editor, that is much more easier than using Joomlacode SVN, given that many of our translators are no developers (see http://docs.joomla.org/Using_the_Github_UI_to_Make_Pull_Requests)
  • Github ACL can provide us the possibility to define who can accept translations, and also allow community translations
  • But also Github can just act as a repository, so we could add later support for Transifex, Crowdin or whoever service the Translation Team prefers. And if that service doesn't support plurals we can always do it by hand in Github
• One translation repository could contain all kind of files: joomla CMS, com_weblinks... it will be up to a tool that generate language packages to separate them

The challenge for this move will be to:

• define the structure of a translation repository
• build the server scripts to create the packages
• Others...

For the future I always see something like what we have done with JIssues, a Framework app that works with our Git repos in the back and have a nice UI in the front.

I can help to do that move but I would need help because I don't feel able to do all this alone. Is anyone willing to?

My proposal would be to do the en-US package https://github.com/joomla-projects/en-US_j3, and use it as a proof of concept.

We can't remove com_weblinks from the core.

Yes we can. The PTL has already decided. Removing com_weblinks from core is perfectly acceptable. If it remains in core, and you remove those files, then update, those files will be replaced. You can't have your cake and eat it too (sorry for the old expression).

Removing com_weblinks is approved by the PLT and will take place in version 3.4, provided we work out these issues. It won't be deleted for existing installs meaning it won't break BC if someone is using it. It will not, however, be provided in the default Joomla! CMS installation package for new installs.

@infograf768 In my few years being with the Joomla Project, I pretty consistently see posts about the troubles the TT's have due to time constraints and the problem devs have with having a code freeze 2 weeks in advance of a release so TT's can catch up. I was simply offering a partial solution to these seemingly constant annoyances. If you want to continue on in frustration as it has been for the past 3 years (since I've been around), then that's fine, but don't say no one tried to help.

@dongilbert
I do not see how putting everything on Transifex (or any online tool) could help concerning time constraints... These will remain, whatever the way we work. The problem devs have with code freeze is mostly unrelated to language. I have always managed to commit some language strings before a PR/patch was fully finished, when necessary.
We do most certainly need a Translation web app that fits our needs but also taking care of the com_localise component which needs a lot of love to be really useful. We would already save much time if that component was, at last, fully functionning.

@javigomez
It is a good idea, but, as I told you in the TT Coordination chat, we would need to create a full github project per language and per J version. New files should be added when necessary all over the place (as quite a few TTs will not get git-aware enough to make PRs/Commits). Then we would not be able to make github work as a translation tool anyway and TTs will still need to get .diff.
We would also need a way to build packs for a release ONLY when the Language coordinator thinks it is ready, but still be able to produce packs for testing in Joomla before releases.

To all:
Until a real workable solution is proposed where we would not lose any language for any "core supported" extension, I suggest we keep the weblinks inis in core so that the language packs contain them.

I do not see how putting everything on Transifex (or any online tool) could help concerning time constraints

It could help if the language strings are uploaded to the tool as soon as we commit them. Meaning you don't have to wait for language freeze to start translating. You would see them as soon as they are available in the repo.
Of course some TTs likely will only start translating when we freeze language, but they at least could start earlier if they wanted. To my understanding that isn't possible currently, except if they watch the repo themself.

@nicksavov https://github.com/joomla-extensions/weblinks/wiki should answer your b/c concerns

@ALL the point here is to ensure we have the language files arranged in the easiest way for translators to deal with them. Selecting a tool to use is not the issue. However, if there are existing tools, we should take them into account in working out how to arrange the files. Does that make sense?

I'd also ask people to be willing to compromise on everything except the goals (that is, no extension files can remain in the core CMS download, new 3.4 installs must still work and you can't break Weblinks on existing sites when they upgrade).

These are the goals you define. In this specific case I stand to my proposal that we should keep the weblinks ini in core until a working solution is effective.

Also, an aspect is apparently missing in this "strategy" for decoupled or "core supported" extensions, and that is the help. What will happen with it? Shall the "core supported" extensions get their help from the wiki as the rest of the CMS?

Forget the last comment. I see now there is a documentation discussion.

In this specific case I stand to my proposal that we should keep the weblinks ini in core until a working solution is effective.

JM, this is not an option. All files are being removed from the core. What is the issue you are concerned about that you need language files to stay in the core?

Also, an aspect is apparently missing in this "strategy" for decoupled or "core supported" extensions, and that is the help.

I'm ten steps ahead of you :) See #3

"JM, this is not an option. All files are being removed from the core. What is the issue you are concerned about that you need language files to stay in the core?"

If they stay in core, the diffs will show to TTs they can keep them in their packages.
But we indeed could do as we did for geshi in some lang packs and keep including the files although geshi is no more provided.
It would not solve updates to weblinks, if any, but better than nothing.

If they stay in core, the diffs will show to TTs they can keep them in their packages

I don't think the old "language package" is going to work once we decouple all these extensions. Also, we will have other extensions included under /joomla-extensions/ that were never added to the core anyway. So we are going to have to come up with another solution. Adding language files to the core for uninstalled extensions is not an option.

"I have seen 3pd translations done on Transifex (OT) using Google Translate... Unreadable."

JM, one of the benefits of Transifex is that you can change translations with problems or make a suggestion. It's an open process, really. We look forward to your feedback on those problematic translations.

Ok, let's try something different. Regardless of what tool is used, if all the translations are included in this repository, how am "I" going to know if it's ok to merge a Pull Request for a complete translation, or minor changes?

Just a note: this is probably the biggest problem with removing extensions from core. Until this is decided we cannot move.

Translations is one of the things that made Joomla to lead the multilanguage market and that's thanks to the current people/structure. So let's try to listen to those people who know about it and do as much as possible to make them happy.

We can think on it as the new issue tracker. If current solutions doesn't work for our specifications we need to build our own tool.

I'm joining @javigomez in the https://github.com/joomla-projects/translate-joomla project to try to help with this problem. I hope some of you do the same.

@eddieajau
You are not... The only person we can trust the best is a TT coordinator for the language. This is why we would need a totally different approach. Separating core lang files from "core supported extensions" lang files needs a different workflow.
Don't forget that we need updates as soon as possible from new releases. It is already hard to get in time for core, so separating core supported extensions would make it even harder

So actually you're not at all preocupied by translations, but trust. No tool can give you that. Method will.

@infograf768 I never said it would be easy. But everyone's comfort zone is going to be stretched, not just yours. Let's work the problem because if we can solve it, we have a process "template" for every other extensions developer out there and I think that would be a huge win. This is going to be a challenge but lets take the attitude that we can solve all these problems in new and innovative ways.

Start throwing around some ideas about how the process can change.

I don't see why we have to reinvent the wheel every time we have a need in the project.
There is a perfectly viable solution, Transifex, that solves the problem of "how do we manage the translations" problem

It works already for a lot of Joomla Extensions developers, who, arguably, have the same issues that com_weblink has in terms of language strings.

Just to point out a few things:

• Transifex deals with .ini files, not only .pot files. In fact, a lot of existing extensions already use transifex as their main repository and tool for language files
• Github is NOT the right tool for translations. It's code focused, not language focuses. Let's use the right tool for the right job.
• Transifex has both an excellent api and an excellent command line tool to manage translations. It can be easily integrated in any packacing / releasing workflow.
• It allows for "managers" to validate translations. This helps a lot in case of bad translations.

Just my 2c

Github is NOT the right tool for translations.

Of course it's not. But the completed language files (and you can use vi for all I care) need to be stored in Github to be able to make the build (the zip file that the user downloads). Given that, there are two choices:

1. We merge all available translations with the Weblinks package in this repository and Joomla handles the build and distribution process via the JED for you. This means we'll need to develop a new workflow to add translations via Pull Requests before the package is shipped.
2. Only the English files are shipped with the Weblinks package and each translator is responsible for managing and distributing their own language packs. All translation is done after the package is shipped and anyone can do whatever they like to produce them.

You will still do your core CMS language packs the same as always, but they will be a little lighter because they don't include the files for the Weblinks extensions.

I personally think option 1 is more convenient for the end-user, but it's less convenient for us because we have more work sorting out the new details (but this is a good problem to be solving).

I don't know if either of @eddieajau solutions include this, but transifex integrates nicely with github.

@Skullbock we don't care what flavour of tool is used (use a text editor if you want, it doesn't matter) - that is not what is being discussed. This is about what happens with the files when they are translated because the English files for Weblinks will not be shipped with the CMS from 3.4 onwards (and other components will be decoupled in 3.5). Making changes to this repository structure to make translation more convenient is also an option.

My personal opinion is the same I had when the transition to github was proposed: use the existing tools instead of building our own wheel. But if those who currently manage translations think that it doesn't fit for our needs we don't have another option that develop our own solution.

We have to rely on our experts for each task. We cannot bypass them each time we don't like their opinions. Frontenders have to decide about frontend, translators about translations....

So let's try to find/discuss a solution that works and if not there isn't another option than build the wheel.

@Skullbock JM already pointed out some issues with Transifex. Like it's hard to add more strings for plurals when needed. Let's just recognize Transifex isn't perfect (yet).

It allows for "managers" to validate translations. This helps a lot in case of bad translations.

This is true, however seldom used. But for core it could be a solution to cover the quality part.
The workflow here is that one person proposes a language change and a second one needs to review it.
It's also to note that in Transifex, not everyone can make changes. The person needs to be approved to the team first. OpenTranslators manages those teams, but you don't necessary need to use OpenTranslators for core extensions. You could as well build the TTs as they are today.
Thus I think the quality isn't really a problem of Transifex itself and could be handled with right setup.

"It is already hard to get in time for core, so separating core supported extensions would make it even harder"

Translating extensions is different from translating core. Yes, core translations relate to Joomla releases and are a 'one off' job. Extensions are not. It can be a continuous automated process completely independent from any Joomla release. The difficulty the 'official' Translation Teams have is the inability to review easily, no Translation Memory for consistency and often relying on one person for the translation and review process.

@ALL Joomla could rewrite an existing wheel (and thank you to @javigomez & @roberto for your work on this) but Transifex is a proven and constantly evolving automated but controllable system which even RedHat are using (and RedHat have also have written their own translation tool but consider Transifex to be superior). Transifex collects the feedback of thousands of translators across 10,000 projects, and continue to improve their product every day. They are also very open to suggestions for further improvements. They have around 400 languages enabled (OT currently has 89 teams with around 20 waiting due to being XX langs therefore no core translation, which uses xx-XX)

It doesn't matter to Transifex where the .ini files are stored in GitHub, they provide a Client to deal with that automatically and Daniel's cTransifex could automatically deal with them from there, if required.

@Bakual I Agree on the Quality issue.

As for the plurals, i really don't see an issue. What's stopping the translation file to add a new language string? Or from adding those strings directly in the base en-GB file, even if the translation is the same?

@Skullbock
Do you have all 51 languages that come with Joomla 3.x shipping with your extensions when using Transifex (OT or not)?
Do you use these in your extension?
#2 (comment)
Are you sure the syntax and consistency of the translations you have match with core packs?

@eddieajau
Please refrain from posting that kind of stuff:
"But everyone's comfort zone is going to be stretched, not just yours."
You should know better...
I have never cared about my "comfort".
I have always cared (understatement) that we get as many languages as possible for Joomla in the best conditions and quality possible, and get follow up/updates along the versions and this since Mambo times.
Let's say we have 60 core language packs. If we get only 30 for these "core supported extensions", I would consider it a failure for the project.

@Bakual "OpenTranslators manages those teams,"

This isn't factual. OT doesn't manage the teams or, except for a couple at the moment, approve translators. OT is a hub of independent translators and independent developers who gather together to increase translation resources for everyone who joins - like a cooperative.

What's stopping the translation file to add a new language string?

Transifex is stopping them 😄 To my knowledge you can't have more strings in the translated file than you have in the source language. You would have to manually add them after you pulled down the file. Each time you do that. That's not going to work.
But one could ask Transifex for a solution.

Or from adding those strings directly in the base en-GB file, even if the translation is the same?

You would have to cover each possible case in en-GB, which doesn't make much sense to me.

This isn't factual. OT doesn't manage the teams or, except for a couple at the moment, approve translators. OT is a hub of independent translators and independent developers who gather together to increase translation resources for everyone who joins - like a cooperative.

@Hils True. I meant it right but have worded it incorrectly 😄

@Bakual - no problem! Just wanted it right publicly :)

Do we agree that it is more convenient for the user if all the translations are shipped with the package they download from the JED (or use the Store UI)?

Personally I would say no. If one just wanted, say, Italian only, to download another unneeded load of translations (which could be nearly 100 or more) as well doesn't make sense. As I mentioned earlier, Daniel's cTransifex gets over this problem by enabling users to connect direct to github/transifex for the latest translations as and when they want.
https://compojoom.com/joomla-extensions/ctransifex-language-distribution-made-easy

(cTransifex is a Joomla component)

It also should be noted that not all extensions need translating fully. Some users might only need the front end translated whilst the backend remained in en=GB

Personally I would say no.

If that's the case, I can close this issue and marked it as "for others to solve" :)

For Weblinks, that's about 15k per language (component and module)

Including all languages in the extensions has two big issues:

• You need to release a new version just for updated/added languages. Especially for com_weblinks it could mean that we have several releases only for translation stuff.
• You can't fix typos in translations fast by just releasing a new language pack. It would need to wait for next release of the extension.

I currently do both in my own extension. I pull the translated languages from Transifex and include them with the package. And I use cTransifex to build the language packages and made them available with a single click in the backend of my component.
However I'm considering removing all languages except en-GB. They blow up my package and only complicate the release process (and I tend to forget pulling down the translations anyway).

On the other hand, it will become tedious if you have to install and update language packs for each core extensions individually.

You need to release a new version just for updated/added languages

Yes. That's typical of almost any other "device" application I can think of.

Especially for com_weblinks it could mean that we have several releases only for translation stuff.

I don't see that as a big drama.

You can't fix typos in translations fast by just releasing a new language pack. It would need to wait for next release of the extension.

While we aren't there yet, releasing should boil down to the push of a button (aka running a few phing commands). A typo would generate a new patch. A new language file would probably trigger a new minor. Ideally we can have the system so highly automated that we could release once a day if we so choose.

And I use cTransifex to build the language packages

I think that's ok for you to decide to lock into one vendor. Should core extensions lock into just one translation vendor?

Hrm /me scratches 3-day old stubble in deep thought.

If we can make cTransifex vendor agnostic, and write plugins for different types of translation sources, would that work?

I don't see that as a big drama.

Not saying it's a drama. Just not ideal.
I'm going from my experience with the german language pack where we already had a few times multiple versions for a given Joomla release. Like for example 3.2.3v1 and 3.2.3v2 because they found typos or whatever after their first release of the pack.
That's fine for a single pack. But you likely don't want that to happen for each extension.

that we could release once a day if we so choose.

Together with above, I'm not sure if admins will love seeing an update package for a core extension every other day. Only to fix a typo in a language file they don't even use. 😄

I think that's ok for you to decide to lock into one vendor. Should core extensions lock into just one translation vendor?

It doesn't matter which tool you use. cTransifex generates regular packages and lists them for download. You can do that with whatever tool you like, from whatever translation source.

I've been thinking about this a while. In my sleep and in the shower even! "How can we solve this issue?" Well it just occurred to me that how @eddieajau and I and the others at the code sprint in Feb discussed things, the optimal solution for this currently is for the strings to remain in core, as much as I dislike it.

Andrew, I'm sure you remember but I'll repost it here for others, but the team decided that removing com_weblinks wouldn't be a BC issue because we are leaving it in place for existing installations, and only removing it from new installations. If we remove the strings from core, then it's a BC issue with those who have existing installations. As much as I hate to say, it's my feeling that these do need to stay in core until 4.x because of this aspect.

Good point there Don about the upgrading sites.

Thats an issue in itself that I dont see being addressed anywhere here. How do we deal with upgraded sites and make sure that they still get the updated core xtensions (perhaps another issue not this one)

How do we deal with upgraded sites and make sure that they still get the updated core xtensions

That should actually be quite easy to do. On an update, we just enter the new updateserver for com_weblinks and they should see updates from there.

If we remove the strings from core, then it's a BC issue with those who have existing installations.

You wouldn't remove any Weblinks files when upgrading to 3.4+. My assumption is all the upgrade does is add the sql to register Weblinks as a standalone package. The one thing we will need to double check is if, today, you can fully uninstall Weblinks. We will need to do that before 3.4.

@dongilbert has exactly stated what I was trying to convey in my first post.
#2 (comment)

For your info, when we took off geshi from core, most TTs with existing packs kept their geshi ini files in their packs. The reason was simple: any older install would have geshi (and possibly could be in use), but no language file in case of deleting a language pack and later on reinstalling it.
The difference between geshi and weblinks is that it disappeared entirely from our releases and was not a "core supported" extension, therefore would never have to be updated.

Let's go further: this does not concern only decoupled extensions, but also added "core supported" extensions (For example the GSOC ones).
This without even considering a good way for ALL TTs to participate to the process easily as I explained above.

Adding all language files for an extension represents around 750k (for +-50 languages).
As of today we have no way to download/install a specific extension with only specific languages. Not even taking into account the fact that a said user may need another language later on in the site building process.

Postponing such a drastic change to all our processes to a time when we can really master all aspects of them is simple realism.

Ok, so are you saying that installing a language pack designed for 3.4 into an old Joomla 3.2 site is what you are worried about?

So is this the problem:
I have a Joomla 3.2 site with fr-FR.
I upgrade to 3.4.
I delete fr-FR (removes old weblinks language files).
For some reason I install the new 3.4 fr-FR language pack.

Result: Weblinks does not translation in fr-FR because it is missing from the core language pack.

Is that it?

That is one of the problems, yes.
But it goes further as I explained above as it also concerns "core supported" extensions.

What is the order of precedence for loading language files? Do we look in the component folders first or last?

@infograf768 it won't be a problem as long as you have installed the language pack for the decoupled Weblinks. The same will go for any other extensions. It will be your responsibility to install the appropriate language packs for the individual extensions.

Which extension language packs? How will they be proposed? How would a user know where to find it? That is not BC at all. We are losing B/C.
You are refusing to understand the obvious: just leave the inis in core... and sync them with the specific weblinks repo.
I leave this chat. I can't explain more.

Which extension language packs?

That's what I'm asking you to help solve. My initial reaction is we could just include all of them in the build, but you don't want to do that. So:

How will they be proposed?

That's another good question. What do you suggest?

How would a user know where to find it?

And another good question. If they aren't included in the build, how are we going to help the user find them? Can the JED help? Do we need to work on some new functionality in the CMS itself to be able to do this?

That is not BC at all. We are losing B/C.

I think that's a rather extreme point of view.

You are refusing to understand the obvious: just leave the inis in core... and sync them with the specific weblinks repo.

You need to take that up with the PLT. If that's the case, there is no point whatsoever in decoupling at all.

Andrew, in short, he's suggesting keeping things exactly the same as they are for TT teams today. This is the simplest solution, works well, and preserves backward compatibility.

If PLT is OK with it, I can post the Language Keys section of the new development strategy in here, which should help with this discussion. I'll double check.

@nicksavov I've posted some questions for the PLT on the mailing list. There is no point in doing any more work on this until those questions are answered. If the language strings need to be kept in the core repo, there is no point in continuing with this project (no point in decoupling it to 80%).

Good idea

I agree with JM, the best solutions for TTs is to always include the extensions language strings in the full CMS. It is the simplest way for our TTs, they already have to manage 3 language packages and will become 4 when 3.4 will be released in July, so please don't complicate the things and leave that few language files in the core, I don't think this will be a issue.

About Transifex, I don't think it's a good way to manage the language packs, I had a bad experience with it when I had to correct and add some italian language strings for Tinymce, and finally I decided to modify manually the language file of Tinymce for Joomla.

The best solution would be to create a special tool for joomla, a unique tool to give to translators, which overcomes all the problems that give the other instruments known. I mean to create it from scratch, as for the new jed that has been commissioned to an outside firm, or as the new tracker that has been made ​​by the team.

Stefania
Italian TT Coodinator and TT Coordination

Stefania

A translation tool for Joomla will take years to develop to the standard of Transifex. Transifex has now five years experience and has changed dramatically since you looked at it eight months ago. If it is relevant to this discussion, the old ideas about Transifex should really be rethought and properly evaluated.

Having spent the last year thinking about packaging and breaking up software, this thread is a serious head-scratcher for me.

First off: It's OK if things have a different structures for different steps of the process! A repository does not necessarily need to be in the same structure as the installation of the code later on. Similarly, a package does not need to be the same.

Second, I think a lot of the thought processes in this thread are tangled up in a similar way and I'm surprised that very few people understand it as what it is: a simple interface problem. Of course there is no silver bullet solution, which makes it easy for some here to argue that the easiest and cleanest way would be to simply keep doing it the same way it used to be done.

Because making Joomla more flexible and modular is a issue close to my heart, I urge you to be a little more imaginative.

What all this comes down to is a question of: How will this get packaged? In Joomla, we have separate packages for components and for languages. The disconnect comes into play because we also have packages that can have components AND languages. I suppose most people here care about having one canonical component install package that also serves en-GB with it.

Why not do all of those things?

I suggest we need one bare component package without any language files, separate packages for the languages and a kind of bootstrap package that installs the component and the en-GB translation.

Now, for the concrete structure, we can have one /language directory for each of the directories that will end up being a package. That way, you have an atomic structure and your translations are contained with the code they are related to.

/com_weblinks
/com_weblinks/language
/com_weblinks/language/en-GB
/mod_weblinks
/mod_weblinks/language
/mod_weblinks/language/en-GB

Then, when the repository is packaged, add a step that extracts the individual language directories so that you get the packages:

/com_weblinks/language/en-GB -> com_weblinks-language-en-GB.zip
/mod_weblinks/language/en-GB -> mod_weblinks-language-en-GB.zip

Remove the /language directories and zip up the functional bits:

/com_weblinks -> com_weblinks.zip
/mod_weblinks -> mod_weblinks.zip

And, if you want to, make another set of zip files like:

com_weblinks-language-en-GB.zip + com_weblinks.zip -> com_weblinks-en-GB.zip
mod_weblinks-language-en-GB.zip + mod_weblinks.zip -> mod_weblinks-en-GB.zip

Which would simply have both the component (or module) and the translation as individual installables.

The only thing left to do, then, is to link these language files in the repository to a service like Transifex, which provides a client to pipe through .ini files to their way of doing things.

As a final note: All of the problems that are being talked about here are well known and well solved in FOSS development. Please let us not pretend like we're the only ones having them and let's especially not pretend that there aren't proven and well working solutions that we can use (if with a little bit of creativity) for our particular situation. The thought process can not be: Oh this one way of doing it that I've seen is incompatible with how we do it, so of course we need to roll a solution that is 100% our own.

That's how wheels get reinvented and how good minds are wasted solving solved problems.

Realize that this is "the great experiment". We don't have a precedent for removing a component (and its supporting extensions) from the core distribution and supporting it separately. For years, we have focused solely on the monolithic package we distribute today. There's nothing wrong with that; the maintenance is overall simpler (everything in a centralized location), our workflows have been established for years with the current setup, and as some people have been undoubtedly thinking since the announcement of this goal, "if it ain't broke, why fix it?".

This is new territory. It's going to take time to establish something that works for all of us and trying to rush to a decision isn't going to help anyone. At the same time, being unwilling to explore change and new possibilities is going to inhibit us from one day reaching a goal of a slimmer core distribution and being able to dump the excess fluff. I look at the unused components on my sites the same way I do software on my personal systems; if I'm not using it I want it gone. I doubt everyone uses com_weblinks on every site they build, how many people want to update the full CMS package just because a security issue is discovered in that code (not implying there are any, please don't misconstrue that!)?

Regarding the language stuff, personally I think we set a bad precedent in allowing the Install from Web plugin's language file to be shipped with core even though the actual plugin is an optional install. That fact alone can be used to strengthen the argument to retain all core supported language files in the primary language (en-GB) in the main distribution. It isn't architecturally right. I'm pretty sure it's a JED rule that there should be no trace of an extension if it isn't installed on a site, why should core extensions be treated differently?

I think many of us watching this thread or offering opinions have a legitimate interest in seeing the CMS move forward and achieving the goals that we all have for it, but also recognize there are issues with the current workflow which are 100% unrelated to the software. Some issues are going to need to be addressed in working towards this particular goal, but we should do so with an open mind instead of "status quo wins unless you have a working solution already". Don't be afraid to experiment a little.

What is the order of precedence for loading language files? Do we look in the component folders first or last?

Joomla first looks in /language and if the file isn't there, it looks in the component folder.
Making an example with german:

1. /language/de-DE/com_weblinks.ini
2. /components/com_weblinks/language/de-DE/com_weblinks.ini

That means you can for example override a language file by placing a copy into the global language folder.

Best practice is that language packs go into the global language folder while languages bundled with a component reside in the component folder.

It's also to note that language files in the global folder are deleted if the main language pack of that language is uninstalled, while those files in the component folder will stay there.

@mbabker

This is new territory.

It is and then again it isn't. Keep in mind that a sizeable portion of the joomla community has been working in precisely that way for almost a decade now. So the precedent we should be looking for is all around and the solutions that have emerged in the 3PD community have been the result of exactly the kind of search that others in this thread propose still needs to happen. For instance - there are reasons why hardly any developer these days has her code on joomlaforge and there are reasons why so many developers use the services of Open Translators. A lot of feet have already cast a lot of votes.

I am getting a sense that there is some resistance to changes like the ones proposed here because it does blur the boundaries of what is the joomla project and what is development around the joomla project. But I think that's a very good thing.

One of the strongest points of joomla has always been the community of developers around it but unfortunately, there has been a certain antagonism in the dynamic between project and the developer community. There is no need to rehash those arguments, of course, but I would say that projects like this small one here will do wonders for calming the tides.

A monolithic package and a rather centralized project, as you've put it, may indeed be easier to manage, but it also creates an us-and-them and (still treading carefully around some open wounds) it has created a number of self-serving cycles that do little but use up peoples time and energy in maintaining the status quo.

"if it ain't broke, why fix it?"

Indeed some people think that, which is unfortunate, because most of the people I talk to say the exact opposite. My guess is that very few people actually believe it, let alone that they would be hard pressed to actually support their argument with facts. Centralization works until a certain degree and then the cost outweigh the benefits and it is my impression that joomla, unfortunately, is way past that mark. But that is the computation we should make: Do the benefits of the new way eventually outweigh the cost of change. In my opinion: A resounding "absolutely yes".

Splitting up the codebase into more managable parts that can be serviced by smaller teams that are more open to influence from the community - simply because the barrier of entry is no longer "this must be fit for the one big package" - will do absolute wonders for collaboration and bringing new people to the table.

Realize that this is "the great experiment".

On the one hand I agree with you - Whether or not joomla is able to decentralize and let go of control will determine its relevance in the market place in the future.

On the other hand: What's with the pathos? In my view, joomla needs to finally come to terms with the facts and make some obvious, even self-evident decisions that have been long overdue. Pitching it up as a gloom and doom event (you only hint at it, others go the whole nine yards) does not help. Let's focus on making new, wonderful things happen even if that means cutting ties with some old and comfortable, yet outdated and ultimately wasteful ways.

So I'm looking at the big picture here that the Core Joomla is going to get lighter and lighter and we are going to have more and more of these Extra extension packages. Let's even imagine that a project like Kunena says, stuff it, we want to be an officially community support forum solution for the project (I'm not suggesting they should, but just imagine we have all kinds of Extras, maybe even commerce solutions or CRM's).

On that basis, this problem is not going to go away and just stuffing more and more language files in Core Joomla is not going to work. Eventually the amount of ini files will exceed the PHP (well, maybe not but you hopefully get my point). So I encourage everyone to lay down their preconceived ideas and think outside of the current processes. We are going to need a different solution in the future. We need to start thinking about this now.

To that end my current thoughts are:

1. We can store all language files for all translations in this repository. I doesn't matter how they are created, but in order to make builds, we need them located in a single place.
2. It's not practical to ship all available language files in one build. We will need to think of ways of building the individual language packages for each translation. But this is new so we need to work out how people can install the language packs they need.

Brainstorming time. Thoughts?

Ok, looks like this is a dead end. If the language files remain in the CMS repository doesn't that mean Weblinks is going to have to wait for the CMS to release to make even the smallest change to the language files? That's going to be as frustrating as hell to deal with.

One possible solution (at this stage) could be that weblinks ships and contains only the en-GB inis.
When a change is made, it would have to be synced into the CMS whatever the change.
Information would be transmitted to the TTs (via the TT Coordination which would provide a lang .diff). We would make sure all TTs are informed, and they would be able to release an updated pack for the latest official version of Joomla. (As you know we now can do that easily as we now use a 4 digits version of the pkg and a cron between our specific repository and update.joomla.org).

Let's say you are ready to propose an updated weblinks package and decide of a language freeze, we would need around a week to get as many CMS lang packs updated as possible.
If a pack is not updated in time, the English values would at least display (instead of the constants, recent improvement in the 3.x and 2.5 series), and end-users would be able to create overrides in the meanwhile.

Obviously, getting the TTs in the process would only be necessary if new strings, not for English spelling mistakes.

One possible solution (at this stage) could be that weblinks ships and contains only the en-GB inis.

Ok - if the en-GB files ship with Weblinks that means they are the "master" copies, right? So translators would have to work off them. The language files in the CMS would be out of date, so is there any point in keeping them there?

Again, asking Translators to look all over the place for possible updates is not feasible at that time.
We would furnish the .diff, but some tools work FROM the files in the CMS and we also maintains the install.xml(s) there.
Therefore they have to also be in the CMS. As I suggested, just sync them...

As I suggested, just sync them...

The current workflow is you have to prepare a pull request and then get two testers. That can take anywhere from days to weeks to have that processed. That's not practical.

@infograf768 With the right tools, translators would not have to look for updates at all, that's the point. Also: "just sync them"? Tackling reliable sync can be one of the hardest tasks I can imagine and if it can be avoided, it should be. If you're worried about translators being confused keeping track of a handful of repositories (which I don't agree with, but the point is moot anyhow), just add Tags and Branches into the mix and a tough situation (that we can resolve entirely with the right tools) becomes an impossible one. Data and Code should be where they belong and pulled only to where they are needed.

@eddieajau I don't see many people left in this thread arguing to keep languages belonging to an extension repository in the main repository, so I think we agree that at the very least en-GB should be in /weblinks as the master reference. The only reason against having language files in that repository, so far, have been that it might be more in line with how translation work has been done previously, but in my opinion, it should be obvious that the cost outweigh the benefits. Whether or not the other translations are also added into the extension repo depends more on what would best suit the translators

The only thing that is left to discuss is: How would this be packaged if the files are indeed separate? I have made a number of suggestions, but would be curious to hear what ideas others have on that matter.

How would this be packaged if the files are indeed separate? I have made a number of suggestions, but would be curious to hear what ideas others have on that matter.

If you think it's worth battling on, go for it.

@eddieajau
Such changes for a core supported extension language files are taken care of in a day max. I would personally take care of it.

@daviddeutsch

With the right tools...

...You want to change the present working system. I have nothing against "progress".
As of now, we do NOT have a new system and keeping posting that Data and Code should be where they belong does not help to solve the present situation. It looks like you think like a developer, not a translator. It seems you are unaware of all what we have to do to get some languages for core. And I repeat: NO, the majority of TTs do NOT keep track of any repository, not even the CMS one.

The only thing that is left to discuss is:

The only thing? Where would the TT files be? How would that be organised? Do we want to pursue the necessity for any "core supported" extension to also have all language files corresponding to the Core lang packs and overlooked by TTs? Or just decide that TTs do not have to support them at all and it will from now on be a totally different game?

I don't see many people left in this thread arguing to keep languages belonging to an extension repository in the main repository

If by "main repository", you mean the CMS, I suggest you read better above.

In any case, research/experiments on new tools/processes are great, but please, do not confuse translators with coders and do not try to force a coder's solution in such a short term.
These matters are not framework type.

@infograf768 I have added plenty of suggestions on how to organize it and further suggested what kind of people we should ask for additional insight to balance or amend my viewpoint. Also: I may not be a translator, but a lot of translators and people who mainly work with translators have weighed in here and I have discussed this with a number of folks in private. I do not confuse translators with coders - the kind of interfaces that I and others have suggested here are precisely for bridging that gap.

It seems you are unaware of all what we have to do to get some languages for core.

Perhaps I could encourage you to entertain the possibililty that I spend a rather excessive amount of time considering the exact amount of effort that is going into maintaining the status quo.

Such changes for a core supported extension language files are taken care of in a day max. I would personally take care of it.

I don't think that's good enough. The system breaks if you aren't there and that means the system is brittle. I don't want to see contributors having to go to 20 different places just because 19 of those people don't want to work towards a common solution that helps everyone.

Perhaps I could encourage you to entertain the possibililty that I consider the exact amount that is going into maintaining the status quo with rather excessive depth.

That's an understatement!

Perhaps...

I quite like how it's now my original wording (in your comment) and my "attempt to make more sense" rewording out on display 😄

There has been a lot of talk about 'official translation teams' which is good. Perhaps it is now time to open up the official process and workgroup generally as most of their discussion is done in private: http://forum.joomla.org/memberlist.php?mode=group&g=47&sid=f2850518a55c9cd38858a95df5bb6d48 . Then everyone could have an informed discussion.

Hils, I think that's important, because I think this type of structure is the way forward for Joomla. That is, Core Joomla is very light and features are extensions. There are at least 4 extensions to follow in 3.5 and hopefully more to be added to the offering. Saying "let's just chuck the language files in the core" is not going to work, so we might as well start solving that problem now. Apart from that, I think the project does a good job at looking after itself but not the 3PD community, and this is something I'd like to solve as we look at Weblinks.

The current workflow is you have to prepare a pull request and then get two testers. That can take anywhere from days to weeks to have that processed. That's not practical.

The system breaks if you aren't there and that means the system is brittle.

Just a few comments. You are of course correct that this is not practical. But as JM said, it's not how changes in language files are handled. They are not required to have two testers (it's one of the exceptions to the rule, typos and codestyles being others) and can be merged fast. JM is also not the only one who can merge PRs. So I don't think that will really be an issue.
It's of course not perfect, I think all agree. But it would work.

First, the term "Official Translation Teams" does not exist, but only "Registered Translation Teams".
Second , all TTs do not form a Working Group. Only the Coordination is one.
We do have indeed a Private Forum for TTs where we post informations, share issues, and prepare new releases.

When a specific concrete proposal will be made here, we shall inform them and discuss. For now, we concentrate of 3.3.0 new lang packs.

[FYI, a member of OT (also TT) tried to force all TTs to get into a "kind of OT" project on Transifex by transferring there their packs "without their approval". It was strongly opposed as, among other issues at this time, the system on Transifex did not keep their copyright in the ini files.
That TT by the way has not released his core pack for a while and did not even update the installation lang file. Therefore, as his first pack was very obsolete (3.0.2v2), we had to take it off from the common repository]

The Coordination is aware of this discussion, as posted above by Stella.
For Marijke, she has explained at length for a long time now why we did not support the OT system in Transifex, as not good enough for Core languages. See also above some technical concerns we all have.

Therefore, no use to send all the OT lobby here. It may work for few strings for 3pd extensions but not for core for all the reasons we have explained multiple times. (It does not mean that some TTs should not use Transifex as an independant project and then correct the results to make their packs: it is their choice.)

Last, we have started a new project to correct and improve com_localise.
See https://github.com/joomla-projects/com_localise

Once this component has been completed to fulfill its original roadmap, my hope is that we can make it interact with Github/any_web_app, in order to fetch/push to any place appropriate the files (inis, .js, .css, zip and xmls) or packs.

Volunteers welcome. That is a concrete joomla project...

It may work for few strings for 3pd extensions but not for core...

Having individual components treated more like a 3PD project is precisely what this should lead to. It's a good development and an overdue one.

...for all the reasons we have explained multiple times.

The only reasons I have seen so far are excuses that it would be hard to change a running system or that with this or that particular bridge, it might not be simple. Doesn't mean it's not possible. And: that still says nothing about the cost-benefit analysis.

Once this component has been completed to fulfill its original roadmap

So what? There are tools working now, being used for what we're discussing here now. To restate my earlier point: You're asking to waste minds on solving solved problems. Why? Qui bono?

Volunteers welcome. That is a concrete joomla project...

The argument is being made that the project should be less centralized and more embracing of appreciating and coordinating outside efforts instead of reinventing the wheel trying to do everything as "a concrete joomla project".

To be brutally frank: I think joomla has lost it's bargaining position to ask in that manner. People are not coming to the table. That's the problem. Maintaining the status quo is not how you fix that.

Finally: The FYI anectode you mention is simply the kind of coordination fallout that can happen when things are decentralized. Nobody is seriously hurt because some parts of a community try something different for a while and fail. It's a price you pay, you just deal with it and move on.

The argument is being made that the project should be less centralized and more embracing of appreciating and coordinating outside efforts instead of reinventing the wheel trying to do everything as "a concrete joomla project".

Just for information what com_localise is, since I didn't know it myself before this discussion started.
It's an existing component translators can use to translate any Joomla language strings locally in their installation. You can select the source language you want from any of the installed languages and can translate to any other language or even simply create a new language pack. It creates the needed XML files from information filled out in an edit form.
You can translate the strings using an interface similar to Transifex or you can edit the files directly using an inbuilt editor. It takes care of double quotes (" are converted to QQ) and shows you syntax errors.
Imho, it goes quite beyond what regular translation tools do and is quite nice already.
However it has some nasty bugs and needs some love desperately.

So please don't shoot it down as "reinvent the wheel". It's not. It's also not a new tool.

It is a new tool because a large amount of it doesn't work in the desired
way. That's why we've forked it from it origins put it on github and are
working on it.

On 14 April 2014 14:08, Thomas Hunziker notifications@github.com wrote:

The argument is being made that the project should be less centralized and
more embracing of appreciating and coordinating outside efforts instead of
reinventing the wheel trying to do everything as "a concrete joomla
project".

Just for information what com_localise is, since I didn't know it myself
before this discussion started.
It's an existing component translators can use to translate any Joomla
language strings locally in their installation. You can select the source
language you want from any of the installed languages and can translate to
any other language or even simply create a new language pack. It creates
the needed XML files from information filled out in an edit form.
You can translate the strings using an interface similar to Transifex or
you can edit the files directly using an inbuilt editor. It takes care of
double quotes (" are converted to QQ) and shows you syntax errors.
Imho, it goes quite beyond what regular translation tools do and is quite
nice already.
However it has some nasty bugs and needs some love desperately.

So please don't shoot it down as "reinvent the wheel". It's not. It's also
not a new tool.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-40363181
.

So please don't shoot it down as "reinvent the wheel". It's not. It's also not a new tool.

I do understand what the tool does (or is supposed to do), but @infograf768 argued that it needs to be finished and extended to provide the functionality that is required for what is discussed in this thread. Since it would be possible to solve the problem with other, existing tools (and 3PDs already do that), I don't think it's unfair to say it would be reinventing the wheel.

No-one should be here to shoot ANYTHING down, ever. All I, personally, nothing to do with OT, wish is that the project could move on and consider embracing crowdsourcing translations, obviously with translation reviews (checks & balances), as a viable way forward. Translation Memory is invaluable for example.

Are we uninventing the wheel perhaps?

@infograf768
related to statements on your reply #2 (comment) that involves me directly I have to make some corrections. I don't want to get along in further conversations as this is not the issue here!

That TT by the way has not released his core pack for a while and did not even update the installation lang file. Therefore, as his first pack was very obsolete (3.0.2v2), we had to take it off from the common repository]»

infograf768 is talking about me, horus_68 the Portuguese (Portugal) language coordinator. Portuguese Joomla Community only supports the latest LTS versions, so no 1.5 and no 3.x (we do have a finished translation for J3.x available to all users, only not packed. So any early adopter for 3.x that "can work without support" can also create their packs. But when we create packs for our language we have other issues to consider as it will say to all governement portuguese sites: its time to use J3!
That's our option and you should respect it... or ask us why!
Also note, that after the recent Joomla release cycle changes, we will release a language for Joomla 3.4 and drop Joomla 2.5 support.
Note that we are translating Joomla core (site and administration) since Joomla 1.0 (using then a special distribution pack because there was no translation files for administrator).

tried to force all TTs to get into a "kind of OT" project on Transifex by transferring there their packs "without their approval".»

False: This is not a kind of OT: this is a single project to translate Joomla. As for many other extensions. I set things up for my language and open it for all others. On transifex a project can have as many languages he creates teams to!
It was a very detailed and documented move, each language team translation coordinators was informed on private translator forum and skype chat and invited into it. This would give any project on transifex also a translation memory form core files.
Several declined to use transifex and the language was removed from the project to translate so no double packs.
So some uses it and others don't use-it! No big deal!

Note: some asked to be removed because they already used other tools, others because already had their own project on transifex and other because you simple ask them not to or/and created panic on stolen files!!)
BTW: you are free to publish files from Joomla, keeping the copyrights. So...?

It was strongly opposed as, among other issues at this time, the system on Transifex did not keep their copyright in the ini files.

Not true. The system keeps the english copyright as its supposed to do. The copyright on any language files are already considered on XML files as you told me before. You also know that the copyrights from joomla core installation ini files was also removed on the same ground!
Maybe Joomla should have a field for copyrights on language files, but this is not the case.

@daviddeutsch
Existing tools do NOT fill our needs, and especially OT.
#2 (comment)
(BTW, I just checked the translation in French for one of Bakual's extensions on OT : it is real bad. Not only typos but values which directly come from Google Translation, some of which are totally wrong, full misinterpretation...)

To be brutally frank: I think joomla has lost it's bargaining position to ask in that manner. People are not coming to the table. That's the problem. Maintaining the status quo is not how you fix that.

I will not be brutal, just frank and no bargain: goodwill people come to the table, enjoy it and help. Otherwise I would have been alone to take care of languages since Mambo times. The project IS concrete and would help. You are free evidently to feel differently.

@Hils
We have not found a tool that fully filled our needs.
The best to date I am aware of is https://webtranslateit.com/

@infograf768

Existing tools do NOT fill our needs, and especially OT.

I am desperately trying to continue to be polite and to discuss this matter agnostically. Perhaps you would do the same and refrain from adding your 'opinions' please?

"(BTW, I just checked the translation in French for one of Bakual's extensions on OT : it is real bad. Not only typos but values which directly come from Google Translation, some of which are totally wrong, full misinterpretation...)"

So you saw a problem, and chose to ignore it? Good job, JM.

I am not an OT Translator. In case you do not know, I do already quite a lot for Joomla.
I told Bakual about it.

Let's stop this - it doesn't matter who the heck anyone is - WE SHOULD HAVE A COMMON GOAL - internationalisation for Joomla and peripheries

Existing tools do NOT fill our needs

And I'm still curious to hear why.

I will not be brutal, just frank and no bargain: goodwill people come to the table, enjoy it and help. Otherwise I would have been alone to take care of languages since Mambo times.

Times are changing, people go where things are interesting and moving forwards. How much has the translation setup changed 'since Mambo times'? How come you are so sure it's still a good, competitive setup and that com_localise would help? Again: Where is the cost-benefit analysis?

@Hils

I am desperately trying to continue to be polite and to discuss this matter agnostically.

Agreed, let's use "Transifex" from now on instead of OT. Which brings me to: @infograf768 - How is Webtranslateit more suitable than Tansifex? Can we please get a discussion going on the actual merits of existing solutions? I have no idea why you keep speaking in the abstract when we could be having a productive discussion.

"I am not an OT Translator. In case you do not know, I do already quite a lot for Joomla.
I told Bakual about it."
In that case, it would only be suitable if you stop mentioning OT in every 2nd comment you made. This isn't a debate about OT but you keep bringing OT up and wonder why we come here to comment...

I will stop mentioning anything as well as posting here. I made my case clearly and we are ready to test anything that fits PLT, lets get correctly translated strings and not make us lose languages for any "core supported" extensions.

I made my case clearly and we are ready to test anything that fits PLT

I'm sorry, but you have not done the former and the way you argue in this thread does not support the latter.

lets get correctly translated strings and not make us lose languages for any "core supported" extensions.

I don't know what that means.

@daviddeutsch I think you're being very provocative here. I don't know how much of the above you've actually bothered to read but JM has listed at least 3/4 things that are not right with transifex (for example the issues with pluralisations here: #2 (comment))

on @daviddeutsch #2 (comment)
There are several teams that really work with different systems: See:
As a component developer, what are my translation options? - http://forum.joomla.org/viewtopic.php?f=11&t=786377

Our pt-PT community used Webtranslate-it https://webtranslateit.com (and also several other platforms) since long time.
Since 2 years we opted only by Transifex https://www.transifex.com and moved all our projects into there. Even i'm not the one to make an updated compare on systems, why did we move?

• WTI looks more friendly to work but more difficult to manage. Download was easy using Zip but bulk management was a pain ruby on rails!
• Transifex has evolved in more user friendly and webware tools. It is a pain for non DOS users but we managed to have the TX client working on Windows systems for bulk management!
• It also brings us a shared translation memory from projects we manage in the same organization and was able to attract more translators from our community (even I think WTI is easier to them, go figure!).
• We are also able to create automated language packs from transifex translations, so less work managing zip files.
• Works with INI files (among other files)
• Can handle plurals. Transifex | Support for plurals - http://support.transifex.com/customer/portal/articles/1070366-support-for-plurals
• It can have comments and lines locked to not translate.
• You can also set up your own organization (even with separated teams), but using a shared translation memory so any term translated in one project is available into other projects. This can be filled automated from 100% identical translation on related project or by user confirmation, so no issues on "Google translations" either.

Note: OT is not Transifex... and there are many projects using the translators teams from OT and others using their own translation teams. You can set up a project using just one translator for each language, is up to the project administrator

It works for us, we have now more then 100 extensions translated with zip to be installed for pt-PT
see here: http://translations.joomlapt.com/languagepacks
There are issues? sure. But you can also be a bad translator using just notepad++

I believe that coders would prefer WTI but most translators, me included, are not coders, so we are the ones limited to use the things that are published.
See my tutorials on Transifex with joomla:
Keep Calm and Translate Joomla - https://sites.google.com/site/transjoomla/

@george
Paulo just dealt with plurals in his post

@wilsonge I have read everything in this thread since it started and the reason why I decided to participate was because it was frustrating to see it drifting into the abstract and speculative.

The arguments voiced against Transifex were cleared up, with the exception that you mention. For that, I have found this. So it's either still being worked on, or could at the very least be resolved with community input to Transifex. Also: If, for instance, Webtranslateit solves this issue, that would be interesting to hear.

I suppose, that is my problem with this discussion: Instead of stating issues and solving them, issues are being stated and used as argument stoppers. Why not, you know, work this out properly?

Finally: As I've said in my first comment in this thread: Why not put a little more trust in what developers around joomla (3PDs) have done for years and years. They have been working on these problems with the tools and/or the tool providers and have thus already done the work that is claimed to still be ahead of joomla.

I'm not saying there aren't compelling reasons to move to transifex (not even using OT) - I was the first one to bring up transifex in this discussion.

Also with the commented out lines - @toretto talked about having to use the API - that isn't ideal for all our TT's - some of them may well not be dev's (I know he also mentioned adding them manually - but that's not exactly the most ideal thing when they're sitting there as comments either!)

However I think we need to get in contact and fix the remaining issues for sure before we can start using it for these core extensions. We can't force the TT's to move to something that doesn't support everything we need. But I agree it's just something that needs smoothing out a bit! Maybe if we can smooth out these issues in advance then JM would more willing to consider it?

@wilsonge - What is the list of issues? It would be helpful to extract clear points into a list.

@infograf768 How does the Joomla project ensure quality translations?

Paulo just dealt with plurals in his post

@Hils @horus68 The post states that Transifex can manage them for PO and XML files. Can it do it for Joomlas INI files? I don't know. Does the portugese language use special plurals like the russian does? Or just the regular "zero / one / more" stuff like we have in german and english?

@ all:
Personally I think that the translation process sure needs some love. But I don't think that this is something which can be changed in 3 months. It is very likely out of scope for the decoupling of com_weblinks as it just produces to many issues within the community and workflows which need to be settled first. A deadline in July doesn't help for that process.
It's still something we need to find a solution for, and which needs brainstorming for. Let's take that as a separate topic for our roadmap which doesn't have to be solved for 3.4.

The decoupling of com_weblinks is meant as a project to detect those issues. PLT didn't expect that it works flawlessly. Let's do what is possible to do for 3.4. And think about ways to improve and solve the issues which are not that easy to solve. Those would be medium- (3.5, 3.6., 3.x) or even longtime (4.0) plans.

However I'm not sure if this platform is the best place to solve that puzzle. The picture is becoming much bigger than just com_weblinks.

@Bakual I have no doubt we can come up with a working solution in a short time and I did expect we could come up with a good plan to completely decouple an extension in the time allocated.

How about this:

1. We limit this trial only to Weblinks until it is proven it can work (which could be 3.6, 3.7, whatever).
2. We continue to brainstorm how language packaging can work in completely independently
3. We move forward to trial the best idea or ideas
4. We make a call about the readiness of those ideas come 28 May

Sound fair?

The reality is the code part is more or less finished. The only things left to do are to sort out wiring up the build process, sorting out documentation and this.

So let's get back to business. We can store all language files in the repo. It may be easier to collect all the language files in a central folder per language, it may not - I'm not sure (need input from all here). The major problem I think we need to solve is how to publish individual extension language packs since I think we all agree now that shipping all languages with the extension is not practical (although, it is only disk space). If we solve that problem we also solve it for the 3PD's and that is a good thing. We need a home for where translation packs for extensions can be listed per extension, and I think we should eventually extend that to any 3PD extension.

@daviddeutsch OK http://support.transifex.com/customer/portal/articles/1070366-support-for-plurals this is the issue for example - transifex only supports those features for .po not .ini files

@wilsonge Do we have an example? I believe @mbabker uses Transifex for JIssues. Let's try a plural and see how it works. I'm thinking that it is possible it works just fine -- but, maybe it doesn't. The only way to know for sure is to try it. Really good to have specific issues otherwise it's not clear where one would get started.

JIssues is also using .po files.

@horus68 @Hils @toretto - Do you have an example of Joomla ini file that uses a plural to share? Agree with @daviddeutsch that concrete examples of issues are best. If this already works, sharing an example of it working should help resolve concerns.

I just did a quick search on some of @horus68 's files and there are definitely translated plurals. It appears to me that Joomla's %s approach is the same as .PO, so, maybe when Transifex says it works for .PO -- it also works for Joomla's special .ini files?

Does that help resolve concerns? Or, am I missing the point? (I am not a language expert, that's for certain.)

You're missing the point. The plurals issue has been well described in this thread already

That's entirely likely, @brianteeman -- would you kindly link me to the comment that describes the issue in some detail? Thanks! =)

@AmyStephen #2 (comment) Read the bit JM addressed to me here. The issue is in non-latin dialetcs for this particular issue (rather than something like spanish/portuguese/french which is my straightforward translations).

@wilsonge - regarding lines 37-49 of: https://github.com/joomla/joomla-cms/blob/master/administrator/language/en-GB/en-GB.plg_captcha_recaptcha.ini#L37"

@toretto addressed that point a few comments above @infograf768 and @nicksavov

Again, in looking at @horus68's translations, he has indeed translated those commented out lines.

So, if I understand the point is that using the API or manually adding those lines is considered to be a significant problem? (Not mocking - asking.)

The second issue then on the plurals is a question as to whether or not languages that require another line for plurals can manually add them. Is that correct?

If so, are those the two issues we have identified for Transifex? (Or, are there others?) Just trying to get a good handle on the problems. When some are pointing at that post to say plurals work and others are saying not for Joomla's ini's -- it's clear -- not everyone is understanding things the same. At least we can iron that out.

Thank you.

Yes they are definitely two issues. A lot of the translators as I understand are not devs in any way. So adding the lines via an API is just bad. Manually adding is OK I guess as long as the translators can easily view the commented out lines (I genuinely don't know how easy that is - I know comments on lines are easy - but don't know about multiple commented lines).

And yes the second issue is definitely an issue - from what the API says it's not possible for ini files (and before people mention it I think many arguments have been had over this before so let's no bring up the .po vs. .ini argument :P)

Another issue JM raised with me was that translation teams are allowed to add their own copyrights to the top of their lang packs. But apparently transifex can't deal with this - it will only copy the copyright from the existing file.

By the way JM also said there's always manual testing of these packs before they are released so I think we should also have a transifex integration built into com_localise as well so that TT's can easily download these packs and test them without having issues of needing to use a CLI script to download the translated files.

I'm not a translator - that's JM's role :P But maybe he can raise if they are anymore issues. But those 4 are definitely a good place to start

If folks are willing to work together, almost every issue being highlighted
can be addressed. We can work with Transifex to request support on things
lacking that others could benefit from and use our own code to make things
simple. I already maintain a simple Transifex API wrapper (which is used
in the issue tracker) that could seemlessly hook into com_localise if we
would actually follow through on something like that. If we'd stop
nitpicking, finding reasons to argue, or shooting down any idea that isn't
the current status quo, all of our time would be better used.

On Tuesday, April 15, 2014, George Wilson notifications@github.com wrote:

Yes there definitely two issues. A lot of the translators as I understand
are not devs in any way. So adding the lines via an API is just bad.
Manually adding is OK I guess as long as the translators can easily view
the commented out lines (I genuinely don't know how easy that is - I know
comments on lines are easy - but don't know about multiple commented lines).

And yes the second issue is definitely an issue - from what the API says
it's not possible for ini files (and before people mention it I think many
arguments have been had over this before so let's no bring up the .po vs.
.ini argument :P)

Another issue JM raised with me was that translation teams are allowed to
add their own copyrights to the top of their lang packs. But apparently
transifex can't deal with this - it will only copy the copyright from the
existing file.

By the way JM also said there's always manual testing of these packs
before they are released so I think we should also have a transifex
integration built into com_localise as well so that TT's can easily
download these packs and test them without having issues of needing to use
a CLI script to download the translated files.

I'm not a translator - that's JM's role :P But maybe he can raise if they
are anymore issues. But those 4 are definitely a good place to start

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-40490515
.

A- On Plurals rules and Joomla:

• A file that uses it is: en-GB.com_banners.ini as in:
  COM_BANNERS_N_BANNERS_STUCK="%d banners successfully stickied."
  COM_BANNERS_N_BANNERS_STUCK_1="%d banner successfully stickied."
  See from line 35 on: https://github.com/joomla/joomla-cms/blob/staging/administrator/language/en-GB/en-GB.com_banners.ini
• For documentation, please see this doc (there can be other more updated, I don't know!):
  International Enhancements for Version 1.6 - Joomla! Documentation - http://docs.joomla.org/International_Enhancements_for_Version_1.6#Pluralization_Support_for_Translation_Files

B- On plurals and Transifex: Supported but not yet for INI files
Transifex do not have pluralization support for INI files. It works for other files so you need to ask them and be involved in order transifex recognize a pluralized entry on INI files.

• On other files type transifex automatically add the lines required by that language plurals rule. Transifex supports every language with a number of plural forms as defined in unicode.org the exact moment we add the language. See: http://support.transifex.com/customer/portal/articles/1070366-support-for-plurals

BTW: it works as a single field to translate but with more then one box for it, so you have to translate both boxes before you are able to save the line!

How to solve the issue:

• The issue here is the INI files and the way Joomla handles the pluralization. On the INI based system, each plural string needs it's own line.
• Usually this is 2 lines (or 3 lines for [ONE] [FEW] [MANY] ). But some languages requires 5 lines. Those languages have to add those extra lines by hand!
• So, unless someone finds a way to solve this for INI files, transifex or any other system will not work.
• For sure... you can do it by hand on the INI files. Thats the way people has been working... and we are now in the XXI century!

C- On comments and Transifex: Supported but not yet for INI files
Issue detailed here by JM: #2 (comment) regarding comments on files like this (line 20 to 24) https://github.com/joomla/joomla-cms/blob/staging/administrator/language/en-GB/en-GB.plg_twofactorauth_totp.ini

Transifex can recognize comments on language files and display them on the box where you will translate the line. But this is not implemented for INI files

• But if it works for others, I'm sure it will be possible to INI if some developer talks with Transifex (or other platform). See: Transifex now supports comments in Apple .strings i18n files - http://ratnadeepdebnath.wordpress.com/2012/01/13/transifex-now-supports-comments-in-apple-strings-i18n-files/

D- On copyrights and Transifex: There is no issue here
Actually some teams had been including extra lines (or removing the ones from en-GB) language files to include their copyrights (usually they are not for the person that actually translate each file but for the community that creates the language package)

• Copyrights for INI files should not be added by hand. Those are joomla code files and the headers must be the same as the en-GB files! Copyrights are in the XML files.
• But wait!! There is a proper way to do this, as some extension developers already do: create a new language string for copyrights for translators!
  So where is the issue?

E- On automatic translations and Transifex: There is no issue here

• Transifex do not translate. The automatic translations feature only suggests a translation: people has to hit a button to ask for a proposal on that particular sentence, than approve (or edit) than move to the next one and repeat! They can also ignore that button and translate as they do in any other world!!
• There is no issue at all regarding automatic translations and Transifex. This is done by real people!
• You can translate with or without google on any system. Even by hand. So where is the issue?

F- On quality translations and Transifex: There is no issue here
If you want that only the TT should translate the decoupled extensions, then you can also do that using Transifex or any platform!

• You can define each language coordinator.
• He/she will then accept others or not on that language team.
• Each language coordinator can decide to not using the platform for that language and simply lock that team to others... and use the system they choose.
  So where is the issue?

I'd be happy to help with any development work needed -- if help is needed. Just let me know who to talk to get started. I'll need direction and feedback.
@horus68 - thanks for going thru the list and summarizing!

RE: D
Copyrights for INI files should not be added by hand. Those are joomla code files and the headers must be the same as the en-GB files! Copyrights are in the XML files.

As you said the copyrights should not necessarily be the same - TT coordinators often add their own copyrights in addition to OSM and this is perfectly OK according to JM (obviously the rest must be the same). It's possible maybe to integrate this into the com_localise component?? But it's definitely an issue

So we need to solve point A through D (A-C with transifex) and work out whether D should be via transifex or the com_localise in development and as you say if we aren't going to be with OT (which I think is completely off the table according to JM) then as you say E and F aren't a problem

So we need to get in touch with transifex and also try and work through getting com_localise up and running.

P.S. @AmyStephen if you wanna help with dev work we're working on bringing back com_localise - it was dropped during the middle of 1.5 to 1.6 conversion work. So if you (or indeed anyone else) wants to help https://github.com/joomla-projects/com_localise

Thanks @wilsonge -- I'll fork it and take a look. I'll target you with questions (so don't hide!)

@nonumber has solved this for his translations - this is one of his french translations: https://opentranslators.transifex.com/projects/p/nonumber-advancedmodulemanager/viewstrings/#fr_FR/com_advancedmodules/1818995

@Hils do you know how this works back into the folder? Is this just a
frontend visable copyright rather than the copyright of the files?

On 15 April 2014 19:27, Hils notifications@github.com wrote:

@nonumber https://github.com/nonumber has solved this for his
translations - this is one of his french translations:
https://opentranslators.transifex.com/projects/p/nonumber-advancedmodulemanager/viewstrings/#fr_FR/com_advancedmodules/1818995

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-40516726
.

I have a custom script on my local machine that converts that language string to a comment line, like: https://gist.github.com/nonumber/10764726
So there is no one-on-one solution AFAIK. It needs some post-processing.

@ALL, the relative merits of individual editors and tools is off-topic. This is not the problem we are trying to solve.

The issue is to devise a new way to manage, build and publish language packs for individual extensions. Solving this problem will also greatly benefit the 3PD community.

In order to eliminate some variables for now, let' just assume that all language packs for Weblinks are stored in this repo and we can build them somehow. When someone installs Weblinks, how will they be able to find the language pack for it? Do we put that information in the XML manifest for the package so that the Joomla Extension Installer is aware of? Can we do something generic to simply suggest other extensions to install (cf https://getcomposer.org/doc/04-schema.md#suggest) and that could include any type of extension (that would actually be really cool)?

Thinking caps on please ...

The issue is to devise a new way to manage, build and publish language packs for individual extensions.

If I understand correctly, you want us to completely rethink translating for Joomla? If so, should Joomla translations be dependent upon a decision that must have been made nearly ten years ago, to use what are known as the restrictive "Joomla .ini" files rather than the universally acceptable .pot files for example? Could someone please explain the logic in that decision as it applies today?

If I understand correctly, you want us to completely rethink translating for Joomla?

Only in so much as 3PD's already have to find their own solutions,

Could someone please explain the logic in that decision as it applies today?

Yes. It's quite simple. You can't load more than one .pot file per page.

https://github.com/joomla-projects/translate-joomla/blob/master/README.md#why-we-use-ini-files-instead-of-po-mo There's a further list of reasons JM drafted up here. I don't think we could consider .pot files before J4 and even then there are definite issues.

Although out of interest how does WP work if you can't load more than 1 file a page? They have extensions like us?

If I understand correctly, you want us to completely rethink translating for Joomla?

I understand it more in distributing the translation, not translating itself.

With our current system, you either ship all available languages with your extension, or you make language packs.
First variant makes the extension bigger than needed and ties extension releases and language updates (or new languages) together.
Second variant is more flexible, but the user needs to find, install and update the language pack manually in a separate step after installing the extension.

We likely want a third - yet to invent - variant.

+1 to what @Bakual says. But I think creating the packs in a different way requires something different. Because current methods are geared towards producing everything in one massive pack

Although out of interest how does WP work if you can't load more than 1 file a page? They have extensions like us?

The only practical way for us to use .pot's is to assemble one huge file for each language. But you have to do that for each site and you'd have to rebuild the file each time there was a change in language string (similar to how LESS works for CSS). And if you did that, the format of the base language files is irrelevant because you just convert it to .pot anyway.

I think that would be a good experiment for someone to do, but I would not advocate changing the .ini standard for 3.x (actually, I don't see a case for changing it at all, but I wouldn't stop someone from trying it).

In the mean time, what if we simply used a post-install message to show a list of the language packs that are available? Click on the links to automatically install them?

Maybe just install the languages (and always update)?

The files will just be ignored unless Joomla assigns the language to the
user object.

On Wed, Apr 16, 2014 at 4:50 PM, Andrew Eddie notifications@github.comwrote:

Although out of interest how does WP work if you can't load more than 1
file a page? They have extensions like us?

The only practical way for us to use .pot's is to assemble one huge file
for each language. But you have to do that for each site and you'd have to
rebuild the file each time there was a change in language string (similar
to how LESS works for CSS). And if you did that, the format of the base
language files is irrelevant because you just convert it to .pot anyway.

I think that would be a good experiment for someone to do, but I would not
advocate changing the .ini standard for 3.x (actually, I don't see a case
for changing it at all).

In the mean time, what if we simply used a post-install message to show a
list of the language packs that are available? Click on the links to
automatically install them?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-40656931
.

If you auto-install languages then you still have the issue Bakual describes about having to update them separately to updating the lang pack separately. Also how you planning on automatically installing the language packs?

I think we are confusing terminology here. A language pack would be a separate language extension that you would install. To include all language as Amy suggests, you just include them in the language folders just as any 3PD would do. To be honest, the latter is the simplest solution available to get things done. But the disadvantage is you will be chewing up disk space. Not a huge issue for one extension, but when you have 5, 10, 15 times 60+ translations, it would add up I suspect so it's not even a good short term solution because 3.5 is going to decouple even more extensions.

On the other hand, managing 60x optional language packs is no easy feat either without a lot of automation. Aside from that, each Github release is going to have 60+ files attached to it with a full complement of translations.

Soooo, if we look at the problem from another side - should we centralise on the language, not the extension and have a repo for each language, and that repo supports languages for many extensions (maybe even the core translations)? Thinking out loud, something like: http://github.com/joomla-extensions/fr-FR

But then you are installing languages for extensions you don't need and we are back to the same problem we have of including the base en-GB files in core ....

Looks like the CMS is sitting at around 55 MB. Less than 2 MB is language
files. I can't see this getting to be a big problem disk space wise. These
are small files.

Admin:
12 KB en-GB.com_weblinks.ini
4 KB en-GB.com_weblinks.sys.ini

Site:
4 KB en-GB.com_weblinks.ini
4 KB en-GB.mod_weblinks.ini
4 KB en-GB.mod_weblinks.sys.ini

Personally, I would keep it simple and ship all the "extension things"
together. Install them together. Update them together. Remove them
together. Easiest for version management, too.

The goal is flexibility -- so, there will also be savings from not
installing extensions people do not use.

Easier for users to have the language in place when they add a core
language.

On Wed, Apr 16, 2014 at 5:25 PM, Andrew Eddie notifications@github.comwrote:

But then you are installing languages for extensions you don't need and we
are back to the same problem we have of including the base en-GB files in
core ....

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2?utm_campaign=website&utm_source=sendgrid.com&utm_medium=email#issuecomment-40660190
.

These are small files.

Remember to multiply that by, potentially, 60+. You'd be adding 2-3 meg of just-in-case language files per extension. But when you consider how much space image galleries chew up, probably not a huge issue.

The typical weight of .po files I have seen in wordpress installs so far (albeit I have only dealt with a handful) was a couple hundred kbs, so I don't think the concept is flawed to start out with. I also did a quick check on a Joomla 3 install and the number was similar, so I don't think 2-3megs is realistic.

Furthermore: While loading more "just-in-case" language files is of course questionable in terms of taxing your memory, keep in mind that the overall goal is to end up with less extensions to begin with.

It would be interesting to see what is the bigger problem - compositing translations at installation/removal of extensions (creates larger overal language file = potentially higher memory usage) or dynamic loading and parsing of .ini files (higher processor and disc load during page rendering).

My hunch would be - memory is only getting cheaper and language files indeed don't take up as much space. Running the same compilation process on each page load on the other hand does seem more readily like a waste of resources overall, even if individual calls might be more efficient in itself.

Not trying to make any suggestions here, just thinking out loud.

@brianteeman -

https://twitter.com/brianteeman/status/456568278622547968

It's not an issue of the number of files but rather what users must do to
use an extension and version management processes.

If someone wants to use the extensions that are to be uncoupled -- they
would install the core language and then locate and install the language
pack for each extension they added?

Seems less than user friendly to me.

From a version management standpoint, if there are 60 language packs to
maintain for each extension, that means 60 git tags, 60 zip files, 60
downloadable links, 60 units whose versions must be managed.

Maybe this entire exercise seems silly but if extensions can be bundled,
that paves the way for distro's.

These questions are tedious and while it all seems silly, it's important to
think through.

On Wed, Apr 16, 2014 at 6:09 PM, David Deutsch notifications@github.comwrote:

In the typical weight of .po files I have seen in wordpress installs so
far (albeit I have only dealt with a handful) was a couple hundred kbs, so
I don't think the concept is flawed to start out with. I also did a quick
check on a Joomla 3 install and the number was similar, so I don't think
2-3megs is realistic.

Furthermore: While loading more "just-in-case" language files is of course
questionable in terms of taxing your memory, keep in mind that the overall
goal is to end up with less extensions to begin with.

It would be interesting to see what is the bigger problem - compositing
translations at installation/removal of extensions (creates larger overal
language file = potentially higher memory usage) or dynamic loading and
parsing of .ini files (higher processor and disc load during page
rendering).

My hunch would be - memory is only getting cheaper and language files
indeed don't take up as much space. Running the same compilation process on
each page load on the other hand does seem more readily like a waste of
resources overall, even if individual calls might be more efficient in
itself.

Not trying to make any suggestions here, just thinking out loud.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2#issuecomment-40663459
.

so I don't think 2-3megs is realistic.

About 30kb of language files per language times 60+ available translations. If you include all the translations in the Weblinks payload, you are looking at about 2 megs.

Just giving the basis for my calculation is all. That is, if we include all translations in pkg_weblinks.zip, that's what you probably add to the disk usage of the web site. Equivalent to about 10 optimised photo's maybe?

https://twitter.com/brianteeman/status/456568278622547968

Brian is never wrong - just ask him.

I think 2 meg is a reasonable guess.

Adding 60 files per extension does seem silly.
But, managing 60 language packs for each extension is downright insane.
I suppose you could just remove weblinks. I mean, how many people really want it?
But, what about those who do?

It's always easier to stick with the status quot -- no one is going to blame you if you don't take risk or stick your neck out or try to move forward.

In the end, I guess it's snarky comments or embrace mediocrity! =)

Just giving the basis for my calculation is all. That is, if we include all translations in pkg_weblinks.zip

Sorry, that's the part that I missed. I was talking about what the final install on a typical site would weigh if we summed up the language files that are actually in use.

As I've said earlier, I don't see any other way than delivering separate packages for translations.

As I've said earlier, I don't see any other way than delivering separate packages for translations.

I think that's the right answer, I'm just not sure on the logistics of how they are published. It almost feels like we need something like a Packagist.org/Bower/NPM for translations ...

As I've said earlier, I don't see any other way than delivering separate packages for translations.

If we go that route, we need a good way to tell the user where the language pack is available for his language. Best would be a link he can click to install it directly. And once installed, it should have it's own update server obviously.
For my own extension, I wrote a view in the backend which lists the available packages with links that will trigger the installer. The list comes from a custom XML view I wrote for cTransifex.
We also have such a list in the language installer view. We could probably also work out something from there.

Best would be of course if Joomla could somehow detect missing extension language packs for the installed languages and notifiy the user of the existance.
That would likely need some entry in the manifest XML file which points to the corresponding language pack update stream (the collection one).

Lets stop for a minute! Before asking how will it be done lets see the big picture and make a public statement. Then pass this discussion to the main Joomla forum as this is not just a developers issue!

A- What is the goal here by decoupling actual core extensions?

• 1- Take Joomla to a minimum code, almost a strip down software? What will not be removed, what can be removed? What is the final goal?
• 2- Creating decoupled extensions as an official package? As in install Joomla: Core pack (Joomla software) + Official pack A (extensions forms, editor) + Official pack B (weblinks, Feed reader) + Official pack C (votes, internal messages) and so on?
• 3- Removing extensions "as time goes by" and let them have their own life (as in release cycle, and so on... and probably let them go away with no official support). And what extensions Joomla core will not drop as "official" even not released in a core pack?

After this been settle we can move to the next questions.

B- Until then, lets keep language files in the main language pack as a temporary solution:

• If you choose to have option 2 then you will end up with a separated language pack for each pack.
• If you go for option 3 then you will end up with a single language pack for each decoupled extension.

C- You should consider for future options:

• Decoupled extensions will have their development teams, release cycle and... independent language updates!
• The language files will be placed in the same actual languages folders or in a different folder? I see some extensions are just moving their files from core language folders (see Virtuemart).

I had a chance to talk with Gabor Hojtsy about Drupal's approach on this. This isn't something doable for the weblinks but you might find some of the thinking helpful in terms of long term planning. It's pretty cool what they do.

This discussion is core + modules the community provides. Of course, a big benefit they have is that the community modules are on drupal.org.

When a project is tagged for release, a script runs to parse the module for translatable strings and adds those language strings to the database on Localize Drupal.

Translators either work on Localize do or they download their language files, work somewhere else, and upload the results.

Translations are exported into .po files for each tagged release on a regular basis since the teams are always working and not on the same schedule.

Site builders install the l10n_update module for Drupal 7 -- Drupal 8 Locale module has this in core -- and that software goes out to Localize to find which tagged releases are in use -- constructs the download URLs, import those translation .po files for the sites use.

It's all done in such a way that developers and translators both get what they need without having to find one another and share information or files or wait on each other.

Since translations are continuously made available to sites -- it's very fluid and asynchronous and automatic. Site builders don't have to locate and install what they need -- sites get all the translations for the languages they require automatically.

He said the biggest downside is that they built this environment for themselves, so they aren't as easily able to take advantage of well known translations services, like Launchpad or Transifex. But, the automated work flow makes it easier for everyone.

Gabor shared this link to an article he wrote a few years ago that might also be helpful.

Be good to see all the fine folks who have been working on core and extension language support lock arms and help bring something like this in -- truly remarkable.

I'm going to close this issue. If someone has a better idea than what is proposed in the existing code, they can make a pull request.

@AmyStephen & @eddieajau In fact, this has been available for nearly two years now for 3rd party Joomla extensions.

https://compojoom.com/joomla-extensions/ctransifex-language-distribution-made-easy

Continuous localization & automation - makes sense to me.

Hils

Awesome @Hils

@Hils, yes that's a good solution for indie developers. However, is it good to say the project should officially lock into that component and Transifex as a vendor? I'm not so sure, but if you want to send a pull request in to do that, I'm sure you'll get some comments. On principle I think you should be able to choose the translation tool that suits you.

@eddieajau

that's a good solution for indie developers

That sounds a bit dismissive - a good number of seasoned professionals use that setup that way. Brings me back to my earlier point - Don't pretend that the project has special technical problems that 3PDs have not encountered yet. It's more likely that the opposite is true.

(It may have a number of other problems that are indeed special, but they are the kind of hard problems you can't solve through technology - they need to be solved through people.)

However, is it good to say the project should officially lock into that component and Transifex as a vendor? I'm not so sure, but if you want to send a pull request in to do that, I'm sure you'll get some comments.

Nobody said the project should lock into Transifex. But - Transifex would certainly be better than sticking to the duct-taped setup that has been showing its cracks and wrinkles for years.

By stalling to even try out any new solution, the project misses the chance to grow and learn and thus never gets to "know what it doesn't know". Which is why people can still argue in favor of keeping the boat steady. Until everything is perfectly still... stiff and cold.

That sounds a bit dismissive

Not at all. It means that I think the choices indie developers have are more flexible than when considering what the project as a whole should standardise on.

By stalling to even try out any new solution

I have no problem trying it. Make a pull request and see what comment you get.

@eddieajau quoted @daviddeutsch
" As I've said earlier, I don't see any other way than delivering separate packages for translations".

then replied: "I think that's the right answer, I'm just not sure on the logistics of how they are published. It almost feels like we need something like a Packagist.org/Bower/NPM for translations ..."

and

@AmyStephen re Gabor Drupal "He said the biggest downside is that they built this environment for themselves, so they aren't as easily able to take advantage of well known translations services, like Launchpad or Transifex. But, the automated work flow makes it easier for everyone."

I was replying to both of these to point out that this already exists and has done for some years!

A pull request? It wouldn't add anything - this solution has been known and generally been ignored for years. Daniel's solution with CTransifex? That is his own project and he should be contacted direct if it is of interest.

Translators should choose their own method of translating? Of course! They would anyway.

I would have thought the automation & packaging is the key point of interest to developers.