? ?
avatar brianteeman
brianteeman
5 May 2016

This is a placeholder issue for multiple existing issues and pull reports (see below)

Aim

To be able to load Javascript just before .

Reasons Why

The async and defer attributes for the <script> element have great support to pause and defer the execution of the script. But still... they are downloaded in the head.

Clear explain of async and defer can be found on: http://www.growingwiththeweb.com/2014/02/async-vs-defer-attributes.html

Comment by Paul Irish in Modernizr github repo why not to load modernizr in the head... and therefor move scripts to the end of body.
Modernizr/Modernizr#878 (comment)

  • If you need html5shiv, split it out from modernizr and conditionally include it for IE8 (better as a conditionally included inline script)
  • Determine if your use of Modernizr requires it to run in the <head>. If it doesn't, defer loading it till the bottom of <body>
  • If you do need it in the head, make the build as small as possible and drop it into an inline script.
  • Do not continue to use a <script src=modernizr.js> from within the <head>

Issues

Unfortunately it still isn't just as simple as this. You still should deal with at least duplicates now, and integration with JHtml::script() (otherwise you possibly have people trying to load jQuery in both the head through the JHtml function and the body through direct including of that file and nothing using JHtml::script() can make use of this right now meaning everything included into the body has no override possibility). IMO that's why things like this keep stalling out.

Previous Discussions

#10223 #2730 #2688 #8271

avatar brianteeman brianteeman - open - 5 May 2016
avatar brianteeman brianteeman - change - 5 May 2016
The description was changed
avatar DavidBoggitt
DavidBoggitt - comment - 8 May 2016

Have never posted before so prepared to be shot down... but... it is worth talking to Michael Richey about his plugin:

http://extensions.joomla.org/extension/scriptsdown

avatar mbabker
mbabker - comment - 8 May 2016

At a quick glance, it's not a proper solution. It looks to be arbitrarily moving everything after the page gets rendered out. Which works for a quick fix I guess, but doesn't really address any of the API related issues that block this from going anywhere.

avatar PhilETaylor
PhilETaylor - comment - 19 May 2016

Would a good start be to implement a JSpecific API like addScript() (

public function addScript($url, $type = "text/javascript", $defer = false, $async = false)
) but for footer included scripts, maybe a addFooterScript() or addDeferedScript() method (and its associated rendering) - then developers can start using this new API while the "blockers" and "stallers" of a full solution are dealt with?

avatar Bakual
Bakual - comment - 19 May 2016

I think the existing addScript could be expanded with another parameter that allows the file to be loaded in the footer. It already supports defer and async so no need to add something specific for those two options.
The big task is how to render it just before the closing body tag. It probably would need changes in the templates (a new jdoc call) to be really safe, but then it would only work on templates that are updated. Since only a few will be updated, devs will not be able to reliable use it and it will be useless.
The other option would be to search for the body tag and add the files there. That would work for all templates but may have side effects and possible bugs, not sure about how reliable that would work..

avatar mbabker
mbabker - comment - 19 May 2016

You really need a full blown asset management API to make it work. And something that can hook into content (if someone's got inline JavaScript reliant on jQuery and it's being loaded at the end of the body versus the head, bam, site's busted).

avatar Bakual
Bakual - comment - 20 May 2016

I'd say jQuery is something which never can be loaded in footer reliably in a CMS context.
Loading a file in footer is something an extension developer has to actively enable. Any "automatic" moving of files from head into footer is bound to fail.

avatar mbabker
mbabker - comment - 20 May 2016

Never say never. Most of the joomla.org properties could load it there successfully. The problem with our inline content is there's no way to mark dependencies if you are inputting inline JavaScript elements, so you either have to have a really smart text parser to figure out if inline content has a dependency and move jQuery if in the API it's been instructed to load in the body or the WYSIWYG editor will need something like RegularLabs Sourcerer where users can freely put the relevant scripts into that space. It's why the API is going to have to get a lot smarter.

avatar Bakual
Bakual - comment - 20 May 2016

Imho it is the job of the extension developer to say if the JS file can be loaded in footer or not, that includes jQuery. Maybe add an argument to the JHtmlJquery::framework method where the dev can specify that it can be loaded in footer. The logic then has to check that all instances where it has been requested allow to load it in footer, if not it's loaded in head instead.

I don't think a parser can be smart enough to figure out the dependencies itself. I would rather adjust our API to support it. Of course a dependency manager for JS files would be even better :)

avatar mbabker
mbabker - comment - 20 May 2016

At the API level you're spot on but you're still excluding the use case of inline JavaScript in any input through a WYSIWYG editor.

Yes, anyone using the JHtml API should specify where the file should load if they need (like they should load their dependencies and not just assume they'll always be available). But that API isn't accessible in user submitted content without something like Sourcerer or a new plugin that can take a {javascript jquery head} tag (or something similar) and parse it to know that jQuery needs to be loaded in the head even if every other extension called up to that point has specified that it can load in the body.

avatar Bakual
Bakual - comment - 20 May 2016

you're still excluding the use case of inline JavaScript in any input through a WYSIWYG editor.

I know. I don't think there is an easy solution there.
But then, if the user JS has a dependency on jQuery, chances are high that jQuery is either loaded in the template or using a plugin. If we introduce this feature then it's likely that those plugins and templates either always load jQuery into head (hopefully using the API) or provide a parameter where the user can choose to load into head or footer.

avatar mbabker
mbabker - comment - 20 May 2016

Either way it's a use case to keep in mind if such an API is ever
implemented. If too many users report issues with load order it may make
sense to pull one of those plugins changing where core libraries load into
the base distro.

On Friday, May 20, 2016, Thomas Hunziker notifications@github.com wrote:

you're still excluding the use case of inline JavaScript in any input
through a WYSIWYG editor.

I know. I don't think there is an easy solution there.
But then, if the user JS has a dependency on jQuery, chances are high that
jQuery is either loaded in the template or using a plugin. If we introduce
this feature then it's likely that those plugins and templates either
always load jQuery into head (hopefully using the API) or provide a
parameter where the user can choose to load into head or footer.


You are receiving this because you commented.
Reply to this email directly or view it on GitHub
#10250 (comment)

avatar stutteringp0et
stutteringp0et - comment - 21 May 2016

I appreciate the mention DavidBoggitt, but my ScriptsDown plugin is a very fragile solution that depends on very well structured xhtml or html5. Any deviation can cause the parsing to fail. It works great if you have the time to fine-tune the output of every extension you install. Mr Babker is right, it's more of a quick fix. The real solution is going to be much more complex.

I'd personally welcome a method to control this within the framework. How that's done is really tough to say. The biggest problem I run into using ScriptsDown is that of dependency. So often, inline javascript depends on jQuery or other frameworks being present on the page - I end up excluding most (if not all) scripts from moving to the bottom of the page. Once more than half must remain in the head, I uninstall the plugin because it's not worth running anymore.

I have, on occasion, written overrides that avoid inline javascript just so I can move all of the scripts to the bottom - but it's so much work...

If there is anything I can do to help, I'll gladly offer.

avatar stutteringp0et
stutteringp0et - comment - 21 May 2016

So, to add my 2 cents here - because I have some experience moving JS to the page bottom, I can confidently say that the problem is dependency for inline scripts. Personally, I hate inline JS - it's ugly, unnecessary, and unfortunately legal....

If I had to design a solution, it would involve requiring 2 new parameters to addScript and addScriptDeclaration, $depends and $provides. If a script depends on another script or scripts, and if a script provides something. This could also lead to automatic dependency injection for system registered scripts. I would also alter the method to have 2 parameters: $url and $options, as many other methods in J use.

For example:

$options=array(
    "type"=>"text/javascript",
    "defer"=>true, 
    "async"=>false,
    "depends"=>array("jquery","jquery-ui"),
    "provides"=>"jquery-animation"
);
JFactory::getDocument()->addScript($url, $options);

By the same method, scripts inserted into content would need to have some sort of process to indicate their dependencies. Once all script files and inline scripts are known, before render can meet undeclared dependencies, sort them, and determine which can be safely moved to the bottom.

There would also need to be a template modification. As there is a jdoc:include for the head, something similar would need to be present for the bottom moved scripts.

Big changes, this wouldn't be possible to include until a major revision. Start planning boys!

avatar Fedik
Fedik - comment - 21 May 2016

@stutteringp0et there exist suggestion about JavaScript/StyleSheet dependencies #8913, but for now that suggestion does not offer anything about placing script in head/footer

avatar stutteringp0et
stutteringp0et - comment - 21 May 2016

it addresses the problem of inline scripts mentioned by mbabker and bakual just 3 comments ago. With known dependencies, it can be determined which can be safely moved to the bottom.

avatar brianteeman
brianteeman - comment - 9 Dec 2016

So long and farewell

avatar brianteeman brianteeman - change - 9 Dec 2016
The description was changed
Status New Closed
Closed_Date 0000-00-00 00:00:00 2016-12-09 15:35:05
Closed_By brianteeman
avatar brianteeman brianteeman - close - 9 Dec 2016
avatar leecollings
leecollings - comment - 17 May 2018

Why was this issue closed @brianteeman? Has it been implemented and merged into a more recent release?

This is a vital feature that it seems Joomla lacks compared to Wordpress. Is there anything in place to be able to load a JS file in the footer of the page source?

avatar dgrammatiko
dgrammatiko - comment - 17 May 2018

Is there anything in place to be able to load a JS file in the footer of the page source?

No the project doesn't care about performance/current best practices and of course PWA...
You need to do it yourself...

avatar mbabker
mbabker - comment - 17 May 2018

No the project doesn't care about performance/current best practices and of course PWA...

That's a BS response and you know it.

There is a lot more to consider than just blindly moving JavaScript from head to body. To be frank, Joomla's API doesn't have the features necessary to even make this an option (even with the split head renderers in 4.0) unless you run sites where you can absolutely guarantee there is no chance in hell of a script being inserted in the body through content or modules that relies on something being loaded before it.

avatar stutteringp0et
stutteringp0et - comment - 17 May 2018

This is possible with a plugin called "ScriptsDown" - but it's not a set-and-forget plugin, it takes a lot of testing and you'll never get all of the scripts to the bottom. Like @mbabker said, sometimes a script needs to be in the head for something to function correctly.

It's a super pain, but if you're willing to put in the time, you can make it happen.

avatar laoneo
laoneo - comment - 17 May 2018

Just wondering, what for real use cases should force you to put a script in the head of the page.

avatar PhilETaylor PhilETaylor - comment - 17 May 2018
avatar PhilETaylor PhilETaylor - comment - 17 May 2018
avatar mbabker
mbabker - comment - 17 May 2018

Just wondering, what for real use cases should force you to put a script in the head of the page.

The real world of mass-market CMS distribution.

^^ That. End users commonly paste JavaScript snippets inline in their content (articles, custom HTML modules, etc.), if they are pasting jQuery scripts they need jQuery loaded before their body content.

We don't have a manner to "defer" scripts to the footer, so even "well coded" modules like the stats charts module on the Developer Network puts scripts inline to the body with dependencies to scripts loaded anywhere before those, unless you're going to tell me the solution to that is to refactor the module to be a plugin so it can append the scripts to the end of the body.

Forcing Joomla to only output JavaScript at the end of the HTML document essentially restricts the ability to place JavaScript in a site to someone who is comfortable manipulating template mixed HTML/PHP files or writing extensions hooking the API. This is why you can't just blindly move the scripts, it MUST be configurable and MUST be something that an end user can fix.

avatar dgrammatiko
dgrammatiko - comment - 17 May 2018

Couple of years ago I did some code for proper script placement at the bottom IF there was no other script tag in the body (there's no safe way to say when some inline script has a dependency).
BUT for J4 the head is separated to metas-styles and script so you can place the scripts at the bottom of the page if you are sure that there is no inline scripts in the body, btw J4 will not have inline scripts, or even if some are still used those are loaded properly through JHtml.
The point is that the scripts to the bottom is a common advise for performance, but the web moved on and nowadays this is not enough we need to actually lazy load scripts...

avatar mbabker
mbabker - comment - 17 May 2018

To be honest half the things Google wants you to do (because it is predominantly them pushing a lot of these things) does not account for mass market general purpose platforms like our's. Unless they do the engineering work themselves.

avatar dgrammatiko
dgrammatiko - comment - 17 May 2018

TBH it's not Google, it's javascript itself that dictates most of these changes. Mainly because they want to go from the callback hell to the async world [lately there are a gazillion of repos that split the workload between the main thread, only painting and the logic loaded as a worker]. Same thing as streams.
But I will agree that many of these changes require some hard B/C breaks, eg disallowing jQuery in Joomla will be an interesting thing [although it will eventually happen, as it did for mootools].
We are at the verge of a major transformation of the web and the coming years will be very interesting (probably the full stack dev will be a think of the past, if it's not already)

avatar brianteeman
brianteeman - comment - 17 May 2018

As Michael says it is a big difference between developing a single web site from scratch and developing a mass market application like joomla.

avatar mbabker
mbabker - comment - 17 May 2018

probably the full stack dev will be a think of the past

One can only hope (says the guy who today has done work in PHP, JavaScript, and SysAdmin realms of responsibility) ?

avatar leecollings
leecollings - comment - 17 May 2018

Hmm this has turned into a big discussion with no real purpose as far as I can see.

Surely, wouldn't it just make sense to support an option that can be used to decide to place it before or not, and by default, to keep it in the header?

That's how Wordpress uses their function to enqueue script files. Surely it makes sense to do the same thing? I don't personally understand why this is worthy of a huge discussion/debate?

A colleague of mine were discussing this earlier today as I need to place a JS file at the end of the source code, and he said, and I quote, 'This is why Joomla sucks'! Which I kinda agree.

It seems miles behind Wordpress.

So anyway my question is, is anything like this currently under development?

avatar mbabker
mbabker - comment - 17 May 2018

No there is not anything currently in development which exposes an API to say "this script should load in the head and this script should load in the body". Because with the existing document API, this is nowhere near as simple as adding a boolean flag.

avatar mbabker
mbabker - comment - 17 May 2018

Anyone is free to work on a solution to this. But I am telling you right now that it is much more complex than adding a boolean flag to JDocument::addScript(). As I have said about a thousand times (because we have had this conversation on and off since, what, late 2014 or early 2015?), to do this right requires at an absolute minimum an API which allows tracking dependencies (so giving an instruction to load Bootstrap through the core API needs to know that jQuery needs to be loaded before it). That is important because you also need to know if your script's dependencies are already in the queue, and here's the important part, where in the queue they're loaded (head/body). If I say to load Bootstrap in the head and jQuery is already enqueued in the body, something needs to be smart enough to say "oh crap, this script needs to be moved because a dependency needs it here".

If this API isn't built correctly, it is going to do nothing but create problems and the end result is going to be the extension ecosystem completely bypasses the core API for enqueuing resources. Clearly it's not an easy problem to solve, because if it were something would've been proposed and merged already.

Joomla isn't the responsibility of the typically active couple dozen contributors. Anyone is free to propose a change at any time. If you're seriously interested in the topic, contribute to a solution (it would definitely take pressure off my shoulders, I already have enough on my plate without people coming in requesting more features and not offering to support that development in any way than saying "yes we want this"). Without people working on a solution, nothing's going to be done.

avatar stutteringp0et
stutteringp0et - comment - 18 May 2018

@leecollings The plugin to do this has been in the JED and free since January 2011 - Joomla has never been behind wordpress in anything but vulnerabilities.

The plugin is ScriptsDown - and it's every bit the pain in the ass @mbabker says it is, I almost wish I had never written it because end-users can never manage to wrap their heads around javascript dependencies.

Add a Comment

Login with GitHub to post a comment