@mbabker regarding your previous comments in another PR:

Few thoughts since this has been hijacked, and I'm just going to end up repeating everything I've said before here (sorry if you've heard it before):

• Not everything requires a query string version, don't mandate that everything has to have it (don't think that's ever been suggested but getting ahead of the curve here)

Of course, i don't see the need to add media version to all joomla js/css files. But i see the need in some files.

• If media loaded through $doc->add***Version() isn't correctly getting version strings changed post update, there's a bug in that process somewhere that needs to be addressed

I think it's ok, but i think is not being used in some js/css.

• JHtml already has a version string mechanism with MD5SUM files in directories, don't proxy it to the doc's version methods unless you intend on deprecating the MD5SUM functionality (this is also the only way right now to do version strings with JHtml loaded assets unless you're hijacking the JDocument head and manipulating it before it's rendered)

Yes i think joomla api has already version/hash mechanisms, but i think is not being used in some js/css.

• Asset files like this aren't loaded through Joomla at all, so unless there's a global "tell browser to bust cache for everything on this domain" header, not sure what can be done at that level (especially since I'm fairly sure things like last modified time are sent with the response headers and browsers should be smart enough to bust cache if it detects that changed)

I was only refering to joomla core files.
Many servers, for performance reasons, force a long expiry time (1 month, 1 year or so) to js/css files and i think that normally overwrites other HTTP headers. The browser will check the last modified + plus expiry time (defined by the server).

Also, besides Last-modified, there could be other headers at play Etag, Cache-control, Expires, Pragma, etc

In those cases, i think the only form to make sure the browser is not useing cached content is load the js/css from a new URI (e.g., a new version or hash in front in the js/file query string).

Of course, i don't see the need to add media version to all joomla js/css files. But i see the need in some files.

snip

In those cases, i think the only form to make sure the browser is not useing cached content is load the js/css from a new URI (e.g., a new version or hash in front in the js/file query string).

The only way to do that is to either force every call in Joomla core to use $doc->add***Version() or ship MD5SUM files in all the core media directories, which forces query strings on all core media. Extensions have the same issue if they ship updated media and don't have some kind of version string in the file path or a query string like either of the above methods would add.

So it's kind of an all or nothing approach here. Either all Joomla core media has to be versioned (because once you place a MD5SUM file in one of the directories to use with the JHtml API everything in that directory uses it), you refactor everything that's "critical to refresh" to load $doc->add***Version() directly bypassing the JHtml API, or you do nothing.

There are a few issues:

• You can't say "these files need to be force refreshed after an update but these other ones shouldn't", especially since it's the web server serving the files and not Joomla (maybe the .htaccess and web.config files can be better configured for this?)
• If there's no way for Joomla to send a "bust all the browser cache" directive (pretty sure there isn't), then the only way to effectively force a refresh is to query string all the media
• Then the question becomes how do you make sure those MD5SUM files get generated and are actually updated whenever something in a directory is changed
• Now people who like to do everything to mask the fact they're running Joomla will argue that this is just another way to fingerprint a specific Joomla version someone's running

There's ways to fix it, the question is how much effort do we really want to put in?

There's ways to fix it, the question is how much effort do we really want to put in?

Yes, that is the real question.

An less time consuming not so good alternative, is have a message after update to use Ctrl+F5 to clear the browser cache.

What about using
libraries/cms/version/version.php

and creating a single query hash for all media

• the hash will be different for every website, because of using creation/modification time of version.php file, which will depend on the time that the web-site is updated to next Joomla version

public function getQueryHash()
{
    return md5(
            filemtime(__FILE__) .
            filectime(__FILE__) .
            $this->RELEASE . '.' . $this->DEV_LEVEL
    );
}

[EDIT]
version.php file is changed on every joomla update to next version

That's exactly what JVersion::generateMediaVersion() already does but it relies on you using $doc->add***Version() which JHtml is not designed to do.

it does the same ?
it looks like the date is changing ?:

        $date   = new JDate;

so the string does not remain the same, thus the browser can not use caching

You suggested a new endpoint which creates a hash. That method already does just that. JVersion::getMediaVersion() handles retrieving that generated hash from the database, because that's where it gets stored. JVersion::setMediaVersion() lets you create your own hash, and JVersion::refreshMediaVersion() will trigger that generate method to recreate the hash.

All of this is already wired into JDocument::add***Version() if you are loading your media assets through that endpoint, which Isis does.

JHtml::script() nor JHtml::stylesheet() support the use of JDocument::add***Version() because its internal API already has a method to generate query strings using MD5SUM files which is processed in JHtml::includeRelativeFiles(). Using the JDocument::add***Version() endpoints instead of the MD5SUM files is a massive break (if you have MD5SUM files now you're getting 2 query strings if you add a parameter to use JDocument::add***Version() instead of the non-versioned endpoints, or you break B/C by dropping support in full for the MD5SUM files, not to mention then you either add the parameters to JHtml::script() and JHtml::stylesheet() then have to go through core to turn on query strings for every core loaded asset or you default the API to use query strings which forces everyone to go through all of their assets to decide whether they are needed or wanted).

I will study it
I am not sure it is same as with what i suggested, i guess you mean

• it is created automatically
• and only once when joomla is updated, and stored in DB

I will look at it more

The methods in JVersion let you create a single hash to use for all media and it is stored as a parameter of the Joomla Platform library extension. If said parameter doesn't exist when it is fetched, a value is generated and stored into the database for use until the next instruction to regenerate it happens (if you have debug mode enabled and on site updates for normal use). So yes it's more expensive in terms of CPU cycles than your method of relying on only the filesystem, but my point is that a unique hash for a site can already be generated, AND set on a global basis not using this generate method. You get all of that for free if you use JDocument::addScriptVersion() and JDocument::addStyleSheetVersion() and don't inject your own version string into those methods.

The JHtml API is not compatible with this though. Long before JDocument had API endpoints for setting a query string, it supported the MD5SUM files and changing its internals to use the JDocument endpoints is just going to break B/C in a very annoying way (you either drop the MD5SUM support or you tack parameters onto the already long declarations for these methods and force everyone to refactor their code to either opt-in or opt-out of auto query stringing).

We already have 2 incompatible ways of deal with query stringing things. Use them. Don't invent a third way. Please. I've said that every single time this discussion happens, and yet every single time a "let's add another API endpoint" suggestion is made and frankly it's starting to get annoying.

I dont care about 3rd party scripts, but I think the core Joomla JS/CSS should have their exact version number (of Joomla) appended to them - this would fix 99% of the issues reported after a version upgrade.

I dont vote for a whole media version handling hash checking system, its not needed, all that is needed is the Joomla version number appended to the url of CORE JOOMLA SCRIPTS

@PhilETaylor a method is already working, for instance, in template.js
See https://github.com/joomla/joomla-cms/blob/staging/administrator/templates/isis/index.php#L31

the question, for sendtestmail.js and others (ex: permissions.js) is that they don't use the script/style version, because they are loaded through JHtml.

See, for instance, https://github.com/joomla/joomla-cms/blob/staging/administrator/components/com_config/view/application/tmpl/default_mail.php#L12

then why not just ALWAYS append the Joomla version number to ALL output scripts, unless one is specified... seems a quick win to me... maybe 2 lines of code to save hours of repeatedly telling people they need to clear their ***** cache!

because JHtml does not support it

because JHtml does not support it

but could...

Read my above rant and any other place this has come up. It's not a simple 2 line fix.

all is possible, we "just" need a volunteer

Sounds like a job for those that rant the loudest

To quote the master @mbabker :

There's ways to fix it, the question is how much effort do we really want to put in?

:)

I'm about to go on 9 weeks leave... paternity leave ;-) I'm sure the cache would have cleared by the time I return.

congratulations them! And keep doing Ctrl+F5 ;-)

@PhilETaylor

The hash (as mbabker informed ...) already exists inside JVersion class, it does not need to be joomla version

        $date   = new JDate;
        $config = JFactory::getConfig();
        return md5($this->getLongVersion() . $config->get('secret') . $date->toSql());

The above is the hash that is created if it does not exist and stored in DB and can be used by automatically by:
JDocument::addScriptVersion() and JDocument::addStyleSheetVersion()

or manually by

$ver = new  JVersion();
$uhash =$ver->getMediaVersion();
JHtml::script('system/sendtestmail.js?'.$uhash, false, true);

See the function JVersion::getMediaVersion() below

    /**
     * Gets a media version which is used to append to Joomla core media files.
     *
     * This media version is used to append to Joomla core media in order to trick browsers into
     * reloading the CSS and JavaScript, because they think the files are renewed.
     * The media version is renewed after Joomla core update, install, discover_install and uninstallation.
     *
     * @return  string  The media version.
     *
     * @since   3.2
     */
    public function getMediaVersion()
    {
....
        return $mediaVersion;
    }

I know its there - but what is the point of it being there if even the Joomla core doesnt implement it right - here is the head from the Global Config page in Joaoml 3.6.0 where only ONE script is appended!

As @mbabker rightly says (often, on many channels), we cannot shout at developers until the Joomla core implements the Joomla features correctly itself...

here is the head from the Global Config page in Joaoml 3.6.0 where only ONE script is appended!

Because, that i know of, all others use JHtml ...

@PhilETaylor

After i made use of

JDocument::addScriptVersion()
JDocument::addStyleSheetVersion()

• with my compontent based hash (not Joomla hash getMediaVersion() because this is appropriate for Joomla files only)

in all my CSS / JS
i never had an issue with users issues related to browser caching of CSS /JS

• and because of loading CSS/JS files via a "loadFramework('framework') method, did not have issue with double loading CSS / JS files either

It is safe to add to things like:

JHtml::_('formbehavior.chosen', '...
JHtml::_('bootstrap.tooltip', '...
JHtml::_('jquery.framework');

but not always safe for cases like:

JHtml::script('system/sendtestmail.js
$document->addScript(

it may lead to double loading of files if these appear inside templates,
because old / custom templates overrides and 3rd party extensions maybe loading them without the hash,

to overcome the above issue maybe we would need a small "fix" inside

$document->addScript()
$document->addScriptVersion()
$document->addStyleSheet()
$document->addStyleSheetVersion()
JHtml::script()

to detect URLs with / without the built-in Joomla hash, and treat them as the same, but it is do-able

@ggppdk @PhilETaylor please test #11289

That PR will "open the door" to add the script version in

$document->addScript()
$document->addScriptVersion()
JHtml::script()

If that PR is merged a similiar PR can be done for the stylesheet part

PR for the js part #12373

(for the css part we still need a lot of work)

for solving the CSS part we first need this one merged #12375 and them do a similiar PR to #12373 for stylesheets, so please test this one also

Closed as we have pr for testing now

@brianteeman we only have a PR for the js part yet

No PR yet for the css part.

OH - whoops - reopened

For JS #11289 and #12373 merged.

For CSS #12375 and #12720 waiting for tests.

Since we finnaly have PR for all of this ... i will close this