? ? Pending

User tests: Successful: Unsuccessful:

avatar nibra
nibra
18 Jun 2020

Summary of Changes

All copyright statements have been changed to the standard format
© YEAR OWNER <CONTACT>
as discussed and decided in the Production Department meeting at June 2nd, 2020:
"The 3.10 and 4.x, framework repos. Introduce into 3.10 and merge into 4.0 and then do missing files for 4 beta2. Niels in charge."

Testing Instructions

No functionality has been touched

Expected result

Everything works like before

Actual result

Everything works like before

Documentation Changes Required

No. However, samples showing the copyright statement SHOULD be updated accordingly.

Scripts used to process the copyright statement

(Also see nibra/fix-copyright for progress on this)

fix-copyright.sh

#!/usr/bin/env zsh

ROOT=${PWD}
GREP_PATTERN="(Copyright )?\(C\) .* Open Source Matters.*All rights reserved\.?"
SED_PATTERN="\(Copyright \)\?(C) .* Open Source Matters.*All rights reserved\.\?"
OWNER="Open Source Matters, Inc."
CONTACT="https://www.joomla.org"

main() {
	echo "Processing files in ${ROOT}"

	COUNT=0

	for FILE in $(grep -rilP "${GREP_PATTERN}" --exclude-dir="vendor" "${ROOT}"); do
		YEAR=$(php fix-copyright.php ${FILE})

		if [[ ${FILE} == *.xml ]]; then
			REPLACEMENT="(C) ${YEAR:-2005} ${OWNER}"
		else
			REPLACEMENT="(C) ${YEAR:-2005} ${OWNER} <${CONTACT}>"
		fi

		COUNT=$((COUNT + 1))

		printf '%5.5s| %7.7s => %s\n' "${COUNT}" "${YEAR:-default}" "${FILE}"

		sed -i -e "s|${SED_PATTERN}|${REPLACEMENT}|" "${FILE}"

	done

	echo "Fixed ${COUNT} copyright notices."
}

main

fix-copyright.php

<?php

class GitHistory
{
	/**
	 * An implementation of file history algorithm with renames detection.
	 * (This comment block is from the IntelliJ source code at https://github.com/JetBrains/intellij-community/blob/ea20241265f9fb956c5a99b1023765aa0e941979/plugins/git4idea/src/git4idea/history/GitFileHistory.java)
	 *
	 * 'git log --follow' does detect renames, but it has a bug - merge commits aren't handled properly: they just disappear from the history.
	 * See http://kerneltrap.org/mailarchive/git/2009/1/30/4861054 and the whole thread about that: --follow is buggy, but maybe it won't be fixed.
	 * To get the whole history through renames we do the following:
	 * 1. 'git log <file>' - and we get the history since the first rename, if there was one.
	 * 2. 'git show -M --follow --name-status <first_commit_id> -- <file>'
	 * where <first_commit_id> is the hash of the first commit in the history we got in #1.
	 * With this command we get the rename-detection-friendly information about the first commit of the given file history.
	 * (by specifying the <file> we filter out other changes in that commit; but in that case rename detection requires '--follow' to work,
	 * that's safe for one commit though)
	 * If the first commit was ADDING the file, then there were no renames with this file, we have the full history.
	 * But if the first commit was RENAMING the file, we are going to query for the history before rename.
	 * Now we have the previous name of the file:
	 *
	 * ~/sandbox/git # git show --oneline --name-status -M 4185b97
	 * 4185b97 renamed a to b
	 * R100    a       b
	 *
	 * 3. 'git log <rename_commit_id> -- <previous_file_name>' - get the history of a before the given commit.
	 * We need to specify <rename_commit_id> here, because <previous_file_name> could have some new history, which has nothing common with our <file>.
	 * Then we repeat 2 and 3 until the first commit is ADDING the file, not RENAMING it.
	 *
	 * @param  string       $file
	 * @param  string|null  $hash
	 *
	 * @return string|null
	 */
	public function findCreationDate(string $file, ?string $hash = null): ?string
	{
		$firstCommitId = $this->getFirstCommitId($file, $hash);

		if (empty($firstCommitId)) {
			// Not under version control
			return null;
		}

		$commitType = $this->getCommitType($firstCommitId, $file);

		if (!preg_match('~([ACR])\d*\s+(\S+)(?:\s+(\S+))?~', $commitType, $match)) {
			throw new RuntimeException("Unexpected commit type for $file: $commitType");
		}

		if ($match[1] === 'R') {
			// File was renamed, follow old file
			return $this->findCreationDate($match[2], $firstCommitId);
		}

		// File was created or copied
		return $this->getCommitDate('%Y', $firstCommitId, $file);
	}

	/**
	 * @param  string       $file
	 * @param  string|null  $hash
	 *
	 * @return string
	 */
	protected function getFirstCommitId(string $file, ?string $hash): string
	{
		if ($hash === null) {
			$command = "git log --pretty=format:%H \"{$file}\" | tail -n 1";
		} else {
			$command = "git log --pretty=format:%H \"{$hash}\" -- \"{$file}\" | tail -n 1";
		}

		return $this->runCommand($command);
	}

	/**
	 * @param  string  $firstCommitId
	 * @param  string  $file
	 *
	 * @return string
	 */
	protected function getCommitType(string $firstCommitId, string $file): string
	{
		$command = "git show -M --follow --name-status \"{$firstCommitId}\" -- \"{$file}\" | tail -n 1";

		return $this->runCommand($command);
	}

	/**
	 * @param  string  $format
	 * @param  string  $firstCommitId
	 * @param  string  $file
	 *
	 * @return string|null
	 */
	protected function getCommitDate(string $format, string $firstCommitId, string $file): ?string
	{
		$command = "git log --date=format:\"{$format}\" --pretty=format:\"%cd\" \"{$firstCommitId}\" -- \"{$file}\" | tail -n 1";

		return $this->runCommand($command);
	}

	/**
	 * @param  string  $command
	 *
	 * @return string
	 */
	protected function runCommand(string $command): string
	{
		return trim(shell_exec($command));
	}
}

$history = new GitHistory();

if ($argc < 2) {
	throw new RuntimeException('Specify a file');
}
if ($argc > 2) {
	throw new RuntimeException('Specify just one file');
}

$file = $argv[1];
echo $history->findCreationDate($file);
avatar nibra nibra - open - 18 Jun 2020
avatar nibra nibra - change - 18 Jun 2020
Status New Pending
avatar joomla-cms-bot joomla-cms-bot - change - 18 Jun 2020
Category Repository Unit Tests Administration com_admin
avatar brianteeman
brianteeman - comment - 18 Jun 2020

How have you calculated the year to be used?

I thought perhaps it was the year the file was added to joomla but that can't be the case as for example with com_actionlogs which was a brand new component I would expect all those files to have the same year. But they are a mix of 2005, 2016, 2018,

avatar brianteeman
brianteeman - comment - 18 Jun 2020

Another example is joomla-cms/administrator/components/com_admin/postinstall/htaccess.php

This file was introduced in joomla 3.4 and the first git commit was Oct 17, 2014 but you are setting the copyright year to 2005

avatar nibra
nibra - comment - 18 Jun 2020

It is the first commit date according to git, following moves and renames, retrieved with the command

git log --follow --date=format:%Y --pretty=format:"%cd" --diff-filter=A --find-renames=40% "${FILE}" | tail -n 1

avatar brianteeman brianteeman - test_item - 18 Jun 2020 - Tested unsuccessfully
avatar brianteeman
brianteeman - comment - 18 Jun 2020

I have tested this item ? unsuccessfully on 573b075


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/29689.

avatar brianteeman
brianteeman - comment - 18 Jun 2020

There is no way that these are correct. Either your query is wrong or I dont know but I only checked the first few and they are not the date of creation for the code - therefore the copyright is wrong.

I am not against this pointless change but if its going to be done then it should be correct.


This comment was created with the J!Tracker Application at issues.joomla.org/tracker/joomla-cms/29689.

avatar nibra
nibra - comment - 18 Jun 2020

Apparently joomla-cms/administrator/components/com_admin/postinstall/htaccess.php has been copied and renamed from a file from 2005; removing the --find-renames option results in 2014.

avatar brianteeman
brianteeman - comment - 18 Jun 2020

I am sure that the same will be true for the other files that I saw were wrong

avatar nibra
nibra - comment - 18 Jun 2020

Likely... I'm re-doing the change without that option. Will take a while.

avatar brianteeman
brianteeman - comment - 18 Jun 2020

seems a lot of work for no benefit

avatar infograf768
infograf768 - comment - 18 Jun 2020

Also, adding « Inc. » means all sites using Joomla in their site name would have to modify their disclaimer. Plus headers in lang packs, 3rd party extensions, etc.

Is that decision a requirement by our lawyers?

Imho, the Production Department should better concentrate on other more important matters.

avatar mbabker
mbabker - comment - 18 Jun 2020

Also, adding « Inc. » means all sites using Joomla in their site name would have to modify their disclaimer. Plus headers in lang packs, 3rd party extensions, etc.

The legal entity name is "Open Source Matters, Inc.", so I would suggest that the copyright statement not using the full legal name is incorrect.

As for the date change in copyright statements...

This might come across as a pointless change, but the other pointless change is modifying every file in every Joomla owned repository in January to amend the ending date of the copyright claim. Additionally, a copyright claim is being made on every file in most every Joomla owned repository of a copyright dating back to 2005, which is clearly not factual.

During my term on the BoD, my question (which was never answered regardless of number of requests) was "why does every file in the repository need a year range in the copyright statement", because my desire for "pointless change" was to remove the year range from everywhere except the LICENSE file (making this procedural change allows for the generated patch packages to stop bloating when their diff includes the commit making that change). I would still advocate for that to be the ultimate end result, but if that is not allowable for some reason, then the changes Niels is making which removes the ending date is a move in the right direction.

avatar alikon
alikon - comment - 18 Jun 2020

Imho, the Production Department should better concentrate on other more important matters.

fully agree

avatar brianteeman
brianteeman - comment - 18 Jun 2020

from the osm report

[Copyright Statement] Following Hugh’s request, Luca asked the Open Source Initiative for advice and they suggested this page that includes the clarifications on how to write an effective copyright statement: https://matija.suklje.name/how-and-why-to-properly-write-copyright-statements-in-your-code . Summarizing the copyright statement should be formulated as follows:
© {$year_of_file_creation} {$name_of_copyright_holder} <{$contact}>

avatar nibra nibra - change - 18 Jun 2020
Labels Added: ? ?
avatar brianteeman
brianteeman - comment - 18 Jun 2020

@nibra I can see some improvements but there are still some very obvious errors. I really don't think this is something you can automate accurately

For example here we have a creation date 3 years before the copyright date

image

avatar brianteeman
brianteeman - comment - 18 Jun 2020

And in some of the files the changes break the code style rules - no idea why drone doesnt pick that up

image

avatar mbabker
mbabker - comment - 18 Jun 2020

@nibra you're probably going to have to do this in two runs, one without rename history and one with rename history.

With the namespacing work in 3.8, hundreds of files were renamed. So the copyright claim on the bulk of the Joomla\CMS namespace is now 2017 even though a fair share of the library API is going to date back to 3.0 (2012) or 1.6 (mostly likely committed in 2009 or 2010). So for things in libraries/(cms|joomla|legacy|src) there's a pretty good chance that doing whatever you're doing with rename history will be pretty accurate. Extensions are definitely pretty hard to get right though with an automated pass because of the murky commit history in places (especially when you get back to the "import the new trunk" commit somewhere between 1.5 and 1.6 since IIRC that essentially blanked and reimported the old SVN trunk) and the fact that a lot of files are copied from somewhere else in the repo with minor adjustments.

Screen Shot 2020-06-18 at 5 01 34 PM

avatar nibra
nibra - comment - 18 Jun 2020

And in some of the files the changes break the code style rules

Which file?

avatar nibra
nibra - comment - 18 Jun 2020

you're probably going to have to do this in two runs, one without rename history and one with rename history

I'll give it a try...

avatar brianteeman
brianteeman - comment - 18 Jun 2020

#29689 (comment)

It was in the screenshot. The ones I saw were all css

avatar infograf768
infograf768 - comment - 19 Jun 2020

The legal entity name is "Open Source Matters, Inc.", so I would suggest that the copyright statement not using the full legal name is incorrect.

This is totally stupid and typical US picky legalese. We have been using "Open Source Matters" for 15 years and it never has been an issue for establishing our trademark (we had other issues that were unrelated to that naming).
It is again one these useless changes which, if they were proposed by a politician, could be understood as a way to distract the attention of the voters from a real problem.

Now, as some want to be picky, let's be fully too and do the right thing:

The Berne Convention: http://www.copyright.fr/hypertext/berne1.htm (in French) https://wipolex.wipo.int/en/text/283698 (in English), signed by all countries including the USA, defines the copyright laws.

The display of the copyright was already defined by the http://portal.unesco.org/en/ev.php-URL_ID=15381&URL_DO=DO_TOPIC&URL_SECTION=201.html (Convention Universelle sur le Droit d’Auteur (Universal Copyright Convention), 1952, revised 1971)

Article III

  1. Any Contracting 'State which, under its domestic law, requires as a condition of copyright, compliance with formalities such as deposit, registration, notice, notarial certificates, payment of fees or manufacture or publication in that Contracting State, shall regard these requirements as satisfied with respect to all works protected in accordance with this Convention and first published outside its territory and the author of which is not one of its nationals, if from the time of the first publication all the copies of the work published with the authority of the author or other copyright proprietor bear the symbol © accompanied by the name of the copyright proprietor and the year of first publication placed in such manner and location as to give reasonable notice of claim of copyright.

i.e.
© [Name of company] [year-of-first-publication]
in that order.

Any other way of display it is just incorrect.

Note: Indicating the copyright has no legal interest whatsoever since the US joined the Bern Convention, contrary to the Trademark. It is just an indication of the authorship.

avatar nibra
nibra - comment - 19 Jun 2020

Also, adding « Inc. »

The 'Inc.' has always been there:
"Copyright (C) 2005 - 2020 Open Source Matters, Inc. All rights reserved."

avatar nibra
nibra - comment - 19 Jun 2020

© [Name of company] [year-of-first-publication]
in that order.

"accompanied by" does not imply an order.

avatar nibra
nibra - comment - 19 Jun 2020

you're probably going to have to do this in two runs, one without rename history and one with rename history.

I just wrote my own 'rename-tracing' algorithm according to IntelliJ's 'Show History' implementation to circumvent a bug in the --follow option of git log. Results look promising so far...

avatar nibra
nibra - comment - 19 Jun 2020

And in some of the files the changes break the code style rules - no idea why drone doesnt pick that up

Actually not - the CSS files (in Hathor) have tab characters between tag and message (which is wrong, but not caused by this change), so the indentation depends on your individual tab width settings.

avatar infograf768
infograf768 - comment - 19 Jun 2020

The 'Inc.' has always been there

not for language files and readme, certainly others.

concerning the order, it is quite obvious: the copyright is given TO an entity, the date being a simple supplementary info.

In any case, do whatever you want. Even discussing it is a loss of time.

avatar nibra nibra - change - 20 Jun 2020
Labels Added: ?
Removed: ?
avatar nibra
nibra - comment - 20 Jun 2020

This version was created using a different algorithm to follow renames, since the --follow option of git log seems to be buggy.
README files and SQL files were tweaked manually.
Also, the contact link was changed to point to joomla.org directly.

avatar brianteeman
brianteeman - comment - 20 Jun 2020
avatar nibra
nibra - comment - 20 Jun 2020

It was discussed in PD, considering what people expect to find when following the link. Thus the choice fell on joomla.org.

avatar brianteeman
brianteeman - comment - 20 Jun 2020

Obviously I was not aware of that decision.

avatar brianteeman
brianteeman - comment - 20 Jun 2020

You also need to update the bump script or your changes will be replaced when the next version is released

avatar nibra nibra - change - 20 Jun 2020
Labels Added: ?
Removed: ?
avatar nibra nibra - change - 3 Jul 2020
Labels Added: ?
Removed: ?
avatar nibra
nibra - comment - 3 Jul 2020

@laoneo @rdeutz @wilsonge @zero-24 Please review, so there's a chance to propagate it to 4.0

avatar zero-24
zero-24 - comment - 3 Jul 2020

Why do we require an test to be changed when we just update the copyright here?

avatar brianteeman
brianteeman - comment - 3 Jul 2020

Surely changing tests to match your code can not be correct. What's the point in tests if thats what happens

/me confused

avatar nibra
nibra - comment - 3 Jul 2020

Because otherwise Travis failes...

avatar brianteeman
brianteeman - comment - 3 Jul 2020

surely that means that the code is wrong. tests are supposed to make stuff fail if they are wrong.

avatar nibra
nibra - comment - 3 Jul 2020

Surely changing tests to match your code can not be correct. What's the point in tests if thats what happens

There's a convention to have a newline character at the end of a file. The tests were expecting layouts/libraries/cms/html/bootstrap/endtabset.php and layouts/libraries/cms/html/bootstrap/endtab.php to not follow that convention without any reason. So I chose to fix the test instead of the files.

avatar brianteeman
brianteeman - comment - 3 Jul 2020

That is all kinds of wrong. If anything the changes to the test should be in a completely separate PR and not hidden inside a pr that changed 4000 files

avatar zero-24
zero-24 - comment - 3 Jul 2020

There's a convention to have a newline character at the end of a file. The tests were expecting layouts/libraries/cms/html/bootstrap/endtabset.php and layouts/libraries/cms/html/bootstrap/endtab.php to not follow that convention without any reason. So I chose to fix the test instead of the files.

Can we move that CS changes out of that PR that would make review easier and the test can be fixed in a seperat PR too.

avatar zero-24
zero-24 - comment - 3 Jul 2020

Additionally to the things mention above:

  • All files from the build folder still have the old copyright
  • /instation/language/sw-KE/sw-KW.ini is missing
  • installation\language\eo-XX\eo-XX.xml not using the correct copy right
  • some files are missing the changes see:
    image
avatar brianteeman
brianteeman - comment - 3 Jul 2020

I just tested about 70% of the xml files. I am not saying the date you have used is wrong or right but it doesn't make sense to have a copyright date that is different to a created date. Of the files that I checked the majority of them have conflicting dates.

As I said before you can't automate this sort of thing correctly. If there was ever a copyright issue then these conflicting dates would be a problem. Certainly more of a problem than this PR is supposed to resolve

avatar nibra nibra - change - 4 Jul 2020
Labels Added: ?
Removed: ?
avatar zero-24
zero-24 - comment - 4 Jul 2020

@nibra

Exchanged copyright sign with '(C)' in SQL files

This has to be done for the XML files than too.

avatar nibra
nibra - comment - 4 Jul 2020

Can we move that CS changes out of that PR that would make review easier

Yes, of course, reverted (hopefully all) unrelated changes introduced by the IDE.

avatar zero-24
zero-24 - comment - 4 Jul 2020

Yes, of course, reverted (hopefully all) unrelated changes introduced by the IDE.

Awesome.

The build folder still seems to be missing the changes.

avatar nibra
nibra - comment - 4 Jul 2020

All files from the build folder still have the old copyright

That directory was skipped intentionally, but I can include it, if it makes sense. IIRC, it is not part of the distribution...

/instation/language/sw-KE/sw-KW.ini is missing

Fixed manually (upcoming commit)

installation\language\eo-XX\eo-XX.xml not using the correct copy right

Yes, 'Kopirajto (C) 2005 - 2019 Open Source Matters. Ĉiuj rajtoj rezervitaj.' was not recognised by the pattern. Fixed manually.

some files are missing the changes see:

Yes, libraries/joomla is considered to be external; the changes should (will) be made in their respective repository.

avatar zero-24
zero-24 - comment - 4 Jul 2020

That directory was skipped intentionally, but I can include it, if it makes sense. IIRC, it is not part of the distribution...

well it is part of the repo and not external so i would include it

Fixed manually (upcoming commit)

?

Yes, 'Kopirajto (C) 2005 - 2019 Open Source Matters. Ĉiuj rajtoj rezervitaj.' was not recognised by the pattern. Fixed manually.

Sure i just wanted to point that out thanks for fixing ;)

Yes, libraries/joomla is considered to be external; the changes should (will) be made in their respective repository.

libraries/joomla is external? only libraries/vendor/joomla are the framework packages right or I'm missing something?

avatar nibra
nibra - comment - 4 Jul 2020

Ok, will process build/ and libraries/joomla/ as well... may take a while

avatar nibra nibra - change - 4 Jul 2020
Labels Added: ?
Removed: ?
avatar nibra nibra - change - 4 Jul 2020
Labels Added: ?
Removed: ?
avatar brianteeman
brianteeman - comment - 4 Jul 2020

I feel sorry for @wilsonge who is going to have to review everyone of these again when he merges up to joomla 4 and then repeat this futile exercise for all the new files

avatar nibra nibra - change - 4 Jul 2020
Labels Added: ?
Removed: ?
avatar nibra
nibra - comment - 4 Jul 2020

@zero-24 Fixed the language files ('Español'); the files had a wrong encoding

avatar nibra
nibra - comment - 4 Jul 2020

I feel sorry for @wilsonge who is going to have to review everyone of these again when he merges up to joomla 4 and then repeat this futile exercise for all the new files

@brianteeman You may feel sorry for me as well - I was 'volunteered' to do this (also for J4, once this is merged into 3.10 and then into 4.0.

avatar zero-24
zero-24 - comment - 5 Jul 2020

@zero-24 Fixed the language files ('Español'); the files had a wrong encoding

Awesome thanks! Will give this another review soon.

avatar zero-24
zero-24 - comment - 5 Jul 2020

I was 'volunteered' to do this (also for J4, once this is merged into 3.10 and then into 4.0.

I would suggest to do a staging -> 3.10 -> 4.0 branch upgrade upfront. So all the "simple" stuff is fixed already right?

Also can you please post the final command used to extract that copyright updates? This can be added to the first post so everyone can find it easily :)

avatar zero-24
zero-24 - comment - 5 Jul 2020

@infograf768 @imanickam Can you please coordinate the changed things about the not changing of the creation date and the updated copyright statment with the TT Teams?

avatar infograf768
infograf768 - comment - 5 Jul 2020

I am totally confused about what has been done for language files...
Here for example why has the date 2011 been chosen? By who?

The first Estonian installation was
Screen Shot 2020-07-05 at 08 51 05

diff --git a/installation/language/et-EE/et-EE.ini b/installation/language/et-EE/et-EE.ini
index 67a98b1a7d1d..2d2a24fe535a 100644
--- a/installation/language/et-EE/et-EE.ini
+++ b/installation/language/et-EE/et-EE.ini
@@ -1,5 +1,5 @@
 ; Joomla! Project
-; Copyright (C) 2005 - 2020 Open Source Matters. All rights reserved.
+; © 2011 Open Source Matters, Inc. <https://www.joomla.org>
 ; License GNU General Public License version 2 or later; see LICENSE.txt
 ; Note : All ini files need to be saved as UTF-8
 
diff --git a/installation/language/et-EE/et-EE.xml b/installation/language/et-EE/et-EE.xml
index 3672909ad002..07c46cab10d4 100644
--- a/installation/language/et-EE/et-EE.xml
+++ b/installation/language/et-EE/et-EE.xml
@@ -4,9 +4,9 @@
 	client="installation">
 	<name>Estonian</name>
 	<version>3.6.3</version>
-	<creationDate>2016-09-16</creationDate>
+	<creationDate>Jan 2011</creationDate>
 	<author>Joomla! Project</author>
-	<copyright>Copyright (C) 2005 - 2020 Open Source Matters. All rights reserved.</copyright>
+	<copyright>(C) 2011 Open Source Matters, Inc.</copyright>
 	<license>GNU General Public License version 2 or later; see LICENSE.txt</license>
 	<files>
 		<filename>et-EE.ini</filename>

Why is the copyright format different in ini and xml?
; © 2011 Open Source Matters, Inc. <https://www.joomla.org> vs <copyright>(C) 2011 Open Source Matters, Inc.</copyright>

Also, which date should be chosen for the 3.10 language packs ?

avatar infograf768
infograf768 - comment - 5 Jul 2020

Also, while we are at it:
It was decided to change the date format to xxxx-xx-xx (which has not been done yet for many languages)
This PR formats it month year like <creationDate>Jan 2011</creationDate>

Also this <creationDate> (for the lang packs as it is never displayed for the installation files except in com_localise) is only displayed in the Installed languages and Extensions Manager interfaces and is useless as Creation. It should rather be considered as an updatedDate to serve a purpose. Which means it should not be a fixed date.

avatar zero-24
zero-24 - comment - 5 Jul 2020

Why is the copyright format different in ini and xml?

As that value gets pushed to the database and there could be instances running that break over that symbol.

Here for example why has the date 2011 been chosen? By who?

By that script based on the git history: https://github.com/joomla/joomla-cms/commits/staging/installation/language/et-EE/et-EE.ini

avatar infograf768
infograf768 - comment - 5 Jul 2020
  1. The script is wrong as I demonstrated. Use a fixed copyright date for all installation and xml ini file is the thing to do. Use 2005 is the simplest as they are all based on en-GB.

  2. Then chose (C) all over instead of © to normalize stuff. Why adding an url in one case and not the other? Why make it complicated when it can be simple?

  3. Still remains the Date format and using a fixed date for what should be understood as an updatedDate

avatar zero-24
zero-24 - comment - 5 Jul 2020

The script is wrong as I demonstrated. Use a fixed copyright date for all installation and xml ini file is the thing to do. Use 2005 is the simplest as they are all based on en-GB.

The srcript is right the file history is wrong but i get your point :D

Then chose (C) all over instead of © to normalize stuff. Why adding an url in one case and not the other? Why make it complicated when it can be simple?

Fine for me.

Also this (for the lang packs as it is never displayed for the installation files except in com_localise) is only displayed in the Installed languages and Extensions Manager interfaces and is useless as Creation. It should rather be considered as an updatedDate to serve a purpose. Which means it should not be a fixed date.
Still remains the Date format and using a fixed date for what should be understood as an updatedDate

Ok makes sense.

It was decided to change the date format to xxxx-xx-xx (which has not been done yet for many languages)
This PR formats it month year like Jan 2011

Ok so this can be updated with the date of the latest change to that file for now with that format and will be updated by the TT's right?

avatar infograf768
infograf768 - comment - 5 Jul 2020

Ok so this can be updated with the date of the latest change to that file for now with that format and will be updated by the TT's right?

Yes, that would make sense. When we ask for new installation .ini in 3.10.x (and therefore the .xml), xx-XX.xml —plus, if changes to language packs install.xml and pkg_xx-XX.xml—, the <creationDate> should be updated by TTs at the same time as the <version>. [in J4 it will be langmetadata.xml instead of xx-XX.xml].

We will have to be extra careful about this before merging new installation lang files.
I have not checked yet if we have new installation lang strings for 3.10 vs 3.9.20-dev

avatar zero-24
zero-24 - comment - 5 Jul 2020

I have not checked yet if we have new installation lang strings for 3.10 vs 3.9.20-dev

Not that I'm aware of.

avatar brianteeman
brianteeman - comment - 5 Jul 2020

That means that creation date is used to mean something different in language files to elsewhere

avatar nibra
nibra - comment - 5 Jul 2020

I am totally confused about what has been done for language files...
Here for example why has the date 2011 been chosen? By who?

History for et-EE.xml says

Jean-Marie Simonet 07.01.2011, 12:53 $ Adding sk-SK, be-BY, lv-LV, et-EE, ro-Ro, ml-BE, lt-LT, aa-AA, no-NO in installation

avatar nibra nibra - change - 5 Jul 2020
Labels Added: ?
Removed: ?
avatar nibra
nibra - comment - 5 Jul 2020

It should rather be considered as an updatedDate to serve a purpose. Which means it should not be a fixed date.

Then either the field should be renamed to 'updateDate' or an 'updateDate' field should be added. It is confusing if semantics are 'misused'.

avatar nibra
nibra - comment - 5 Jul 2020
Then chose (C) all over instead of © to normalize stuff.

Done

avatar nibra nibra - change - 5 Jul 2020
The description was changed
avatar nibra nibra - edited - 5 Jul 2020
avatar nibra
nibra - comment - 5 Jul 2020

can you please post the final command used to extract that copyright updates? This can be added to the first post so everyone can find it easily

Done. The pair of scripts does the heavy lifting, the result might need some manual polishing, as
in some files (README, SQL data), the copyright year does not necessarily mean the creation year of that specific file.

avatar nibra
nibra - comment - 5 Jul 2020

Why is the copyright format different in ini and xml?
; © 2011 Open Source Matters, Inc. <https://www.joomla.org> vs <copyright>(C) 2011 Open Source Matters, Inc.</copyright>

The '©' sign has been replaced with '(C)' in all copyright statements now. The contact information <https://www.joomla.org> is not used in XML files, because a) it would break the file syntactically and b) the contact information is contained in the XML file already.

avatar infograf768
infograf768 - comment - 5 Jul 2020

@nibra

Heard about fighting against wind-mills?

History for et-EE.xml says
Jean-Marie Simonet 07.01.2011, 12:53 $ Adding sk-SK, be-BY, lv-LV, et-EE, ro-Ro, ml-BE, lt-LT, aa-AA, no-NO in installation

The real history, once more, if not in git!! See again
Screen Shot 2020-07-05 at 08 51 05

It makes no sense to define a different copyright year for these language files. Please use 2005.

Also, I repeat: it was asked to TTs to use the xxxx-xx-xx (year-month-day as figures for the date) and you still keep a different format.

avatar nibra
nibra - comment - 5 Jul 2020

@infograf768

The real history, once more, if not in git!! See again

If you provide the data, I'm happy to integrate them manually.

avatar nibra nibra - change - 6 Jul 2020
The description was changed
avatar nibra nibra - edited - 6 Jul 2020
avatar infograf768
infograf768 - comment - 6 Jul 2020

As proposed, just use 2005 for all. It is unecessary to use any other date for the copyright.

avatar nibra nibra - change - 6 Jul 2020
Labels Added: ?
Removed: ?
avatar nibra
nibra - comment - 6 Jul 2020

@infograf768

As proposed, just use 2005 for all. It is unecessary to use any other date for the copyright.

That would be 'more wrong' than the git history's first commit date, as the date documents when the file was introduced to the codebase. However, I went through all installation language files and reconstructed the creation date where possible using JoomlaCode and the original content of the file. The commit date is now only used where neither JoomlaCode nor the file content provided sufficient data. It's IMO not possible to get closer to the truth.

avatar nibra nibra - change - 7 Jul 2020
Labels Added: ?
Removed: ?
avatar nibra
nibra - comment - 7 Jul 2020

PD decisions about installation language files:

Copyright: Follow the legal advice and state the year of creation/initial commit of the file (package), so the copyright years stay as they are now.

creationDate: It is ok to use it as creation date of "this specific version" to accommodate Translation Team's needs.

avatar nibra nibra - change - 7 Jul 2020
The description was changed
avatar nibra nibra - edited - 7 Jul 2020
avatar Bakual
Bakual - comment - 8 Jul 2020

Honestly, just take a year and be done with it. It doesn't matter at all as nobody will ever claim copyright for language files anyway.

avatar nibra
nibra - comment - 8 Jul 2020

Honestly, just take a year and be done with it. It doesn't matter at all as nobody will ever claim copyright for language files anyway.

The years are as correct as determinable now.

avatar infograf768
infograf768 - comment - 8 Jul 2020

Language packs are not PRs. What remains of the TT coordination is not going to check each language pack for the « correct » copyright date for each ini file.
The PD, as it is so much concerned, will now take care of it.

avatar Llewellynvdm
Llewellynvdm - comment - 8 Jul 2020

@nibra looking at the following file: installation/language/ckb-IQ/ckb-IQ.ini
image

The scripts force the whole file to update instead of just the Copyright notice:
image

Is there away to avoid this? or is this just git being unable to tell the difference.

Then I see many database copyright updates and I am not able to validate those changes, it is hard to make out what has changed, since git is again unable to isolate the lines that were changed but just highlights the whole block.
image

Yet for the most part the changes look correct, and consistent @nibra great Job!!
image

quick_over_view
When looking at the large volume of changes being made, and knowing PHP and BASH well enough I know you must have worked on this for days. So I just want to thank you for this tremendous effort! Even though not everyone is to existed about this update, since it does not directly befit the code, @nibra you have saved the community many many hours. Thank you!

avatar nibra
nibra - comment - 11 Jul 2020

The scripts force the whole file to update instead of just the Copyright notice:
Is there away to avoid this? or is this just git being unable to tell the difference.

It is your diff viewer; PhpStorm will tell you very precisely, as will the diff view here on GitHub (although GitHub is not very user friendly when diffing ~4000 files). Only apperances of the copyright pattern were touched, wherever they were found in a file.

Then I see many database copyright updates and I am not able to validate those changes,

Those dates are hard to validate. In SQL statements, the year was changed to be consistent, as those entries just are cached results from the manifest files and have no legal implications.

avatar nibra nibra - change - 22 Jul 2020
Labels Added: ?
Removed: ?
avatar zero-24
zero-24 - comment - 11 Aug 2020

@nibra please fix the conflicts than I will take the green button to merge this here in. When I got the initial post correct you are doing a PR against 4.x after that here is merged too?

avatar nibra nibra - change - 11 Aug 2020
Labels Added: ?
Removed: ?
avatar nibra
nibra - comment - 11 Aug 2020

When I got the initial post correct you are doing a PR against 4.x after that here is merged too?

Yes, that's the plan...

avatar nibra nibra - change - 11 Aug 2020
Labels Added: ?
Removed: ?
avatar zero-24 zero-24 - change - 12 Aug 2020
Status Pending Fixed in Code Base
Closed_Date 0000-00-00 00:00:00 2020-08-12 15:57:21
Closed_By zero-24
Labels Added: ?
Removed: ?
avatar zero-24 zero-24 - close - 12 Aug 2020
avatar zero-24 zero-24 - merge - 12 Aug 2020
avatar zero-24
zero-24 - comment - 12 Aug 2020

Just merged here. Thanks @nibra and all others involved.

avatar nibra
nibra - comment - 12 Aug 2020

Thank you! How do I know, when 3.10-dev is pulled into 4.0 (without polling all the day)?

avatar zero-24
zero-24 - comment - 12 Aug 2020

Up to @wilsonge i guess.

avatar richard67
richard67 - comment - 14 Aug 2020

This PR has broken updating from 3.10-dev to 4.0-dev by breaking the closing tag of the copyright element here:

https://github.com/joomla/joomla-cms/blob/3.10-dev/administrator/manifests/files/joomla.xml#L7

The update component uses that file to get the version from.

I am preparing a PR to fix that.

Add a Comment

Login with GitHub to post a comment