By default all JavaScript assets are rendered at the bottom, but you can easily specify which must be rendered in the head instead.

I think that will fail because developers would especially need to specify that their script has to be loaded in the head, which is the default location at present. Thus this default value would be a backward compatibility break.
If you change it to default in head and let the developers specify which scripts can be loaded at bottom (similar to the current deferred and async options) then it will be fully BC.

By default nothing changes so in that respect it is fully BC. If the site builder opts in to using it then they would modify the template to use these new jdoc statements and that gives them full control over whether JS renders at the top or the bottom regardless of what the extension developer specifies. I have now changed the default so that If the only change that the site builder makes is to add the "top" and "bottom" statements then the default is that everything is rendered in the head. The load order might still be slightly affected because the new API treats all assets equally so internal and external scripts may now be inter-mingled, so in that sense it may break a site, but the solution is to add the relevant dependencies to force whatever load order you need. If an extension developer chooses to adopt the new API then regardless of whether they specify loading in top or bottom the template can still override that and move it to wherever makes sense for that site.

What happens if the extension developer adopts the "new" methods but the
template developer doesnt?

On 26 December 2013 12:03, Chris Davenport notifications@github.com wrote:

By default nothing changes so in that respect it is fully BC. If the site
builder opts in to using it then they would modify the template to use
these new jdoc statements and that gives them full control over whether JS
renders at the top or the bottom regardless of what the extension developer
specifies. I have now changed the default so that If the only change that
the site builder makes is to add the "top" and "bottom" statements then the
default is that everything is rendered in the head. The load order might
still be slightly affected because the new API treats all assets equally so
internal and external scripts may now be inter-mingled, so in that sense it
may break a site, but the solution is to add the relevant dependencies to
force whatever load order you need. If an extension developer chooses to
adopt the new API then regardless of whether they specify loading in top or
bottom the template can still override that and move it to wherever makes
sense for that site.

—
Reply to this email directly or view it on GitHub#2730 (comment)
.

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

Good question. I wouldn't expect extension developers to adopt the new methods exclusively, precisely because it would break existing templates. What they can do is continue using the existing methods but then add some "hints" about dependencies, whether to load in the head or not, etc. An old-style template would ignore the hints so no change there, but a new-style template (or rather the site builder) would have less work to do because it would honour those hints and get the correct load order regardless of the execution sequence of the extensions.

Thats not going to work. A good Extension developer would quite rightly
want to use the new features and it would break a non upgraded template. It
has to be done in such a way that if and when an extension developer adopts
this it doesnt break existing templates. Otherwise no extension developer
will ever adopt it. In which case it would be a waste of time implementing
it.

On 26 December 2013 13:47, Chris Davenport notifications@github.com wrote:

Good question. I wouldn't expect extension developers to adopt the new
methods exclusively, precisely because it would break existing templates.
What they can do is continue using the existing methods but then add some
"hints" about dependencies, whether to load in the head or not, etc. An
old-style template would ignore the hints so no change there, but a
new-style template (or rather the site builder) would have less work to do
because it would honour those hints and get the correct load order
regardless of the execution sequence of the extensions.

—
Reply to this email directly or view it on GitHub#2730 (comment)
.

Brian Teeman
Co-founder Joomla! and OpenSourceMatters Inc.
http://brian.teeman.net/

This is great

@brianteeman brings up a good point though. As a solution, in 3.x, would it be possible to inspect the template for these new template placeholders, and if they don't exist, append all data to <jdoc:include type=”head” />, or prepend deferred scripts to </body>. We could then add an entry to a log, or error, and also make these tags required for 4.x templates.

@brianteeman. Since this is aimed at empowering site builders I would argue that it's not a waste of time. I believe it solves a real problem that many site builders face on almost every site they build (I know I do) and it does so in a way that does not depend on extension developers making any changes to their code.

In my opinion an extension should only ever be hinting about matters such as load order, load position (head/tail), etc. The last word should always go to the site builder. At present there is no mechanism for that to happen (except that JHtml::script/stylesheet has some override capabilities that have yet to be universally adopted by extension developers anyway). This proposal allows extension developers to continue using the existing API but extends it so they can hint about load order, etc. If template/site builders don't use the new API/jdocs (perhaps I should call it an "extended" API rather than a "new" API) then the extension developers are no worse off than they are now.

Completely replacing the current API probably can't be done before 5.0 (mark as deprecated in 4.x, then remove in 5.0). That's a long time to wait for improvements. This proposal can deliver real improvements today.

@betweenbrain. If I understand you correctly then you are suggesting that existing templates could be transparently "upgraded" to use the new system. Hmm. That's pretty ambitious and is beyond what I was expecting to achieve, but having given it some thought, I wonder...

I might not have thought of all the ways in which this could fail but I think that top.php could be modified to deliver exactly the same results as head.php provided the template and all extensions are unchanged (ie. they only use the current API). It might be a few microseconds slower because of all the extra work it's doing to achieve the same result, but I think we can live with that. However, I would not be in favour of automatically adjusting the template to prepend deferred scripts, add log entries, or anything like that. I think those are all potential points of failure. I'll look at the code again later and see what I can come up with.

@chrisdavenport I agree that falling back to the existing behavior is likely the most robust way of implementing this. I agree that adjusting the template does carry some risk with it, and likely best to avoid.

Just want to add that there was recently a discussio in the Google Groups about this: https://groups.google.com/forum/#!topic/joomla-dev-cms/qlyWQEUj63g.
A PR was made here: #2688

Please read the discussion and especially the proposal of Don to add a new $placement parameter. The current JDoc would then behave as always but there are more "specialised" JDocs which can be used optional (opt-in).

@chrisdavenport That's an interesting approach but I might say that Don's proposal with the parameter $placement and the 4 different types: meta, styles, scriptstop and scriptsbottom, is quite easier (it's closer to the current code) and doesn't break anything right now as the default jdoc:include type="head" still works as it used to. I was supposed to bring some code but then xmas get in the way...
If the PLT gives ok to this one I am more than happy as it brings the same needed functionality.
By the way thanks for taking the time to code this.

@Bakual, @dgt41. Don's proposal as I understand it would break up the current head functionality so it's a little more granular. So you would have a group for head scripts (with all external scripts coming before all internal scripts), a group for all tail scripts (all externals before all internal again) and a group for all stylesheets (all externals before all internals again). But this granularity, whilst undoubtedly an improvement, is still not enough to solve some of the problems that are encountered. Even going further and breaking it into 6 groups (internal head, external head, internal tail, external tail, internal stylesheets, external stylesheets) would not be enough.

My proposal allows extension developers, template developers and site builders to "draw" the dependency graph amongst the individual assets, regardless of their type, and have that dependency graph resolved at runtime. For example, suppose an extension adds a rule that says script A must load before script B. Another extension states that script A must load before script C. If both extensions are installed on the same site the actual load order could be ABC or ACB. Suppose the site builder discovers that one works (ACB, say), but the other (ABC) doesn't. Currently, and with Don's proposal (or even the more granular version), you'd have to write some PHP code in the template or a plugin to manipulate the head arrays to switch the load order (or use a plugin that parses the final rendered output and tries to manipulate the HTML (probably not reliable). With my proposal you would just add a single statement to the template to switch the load order:

<jdoc:include type="javascript" id="B" dependson="C" />

Now suppose that C is actually an internal (inline) script. That can't be done with Don's proposal at all (since it needs to be sandwiched between two external scripts). You'd have to drop B and C from the JDocument arrays (with some PHP code) and load them in the required order by manually adding HTML <script> tags to the template. In other words, you'd have to stop using the API because the API couldn't cope with that situation.

The example is a bit contrived (although it is indicative of many real situations that I've had to deal with), but I hope it gives you an idea of how flexible this proposal is. Each extension draws its local part of the dependency graph and the template pulls them all together into a single graph and resolves it at runtime.

I am have big doubt in current suggestion, especially in dependency management...
In two word: too complicated and not to much profit.

In more than two word:
I not liked additional layer JDocumentAsset, just for add script/style.
I see no sense in <jdoc:include type=”javascript” url=”/js/some.js” />
because in the template I can use <script src="/js/some.js" type="text/javascript"></script> that will work faster , and I can add it in the ordering that I need.

All problem starts when the extension developer tried put the own scripts everywhere, without count that there can be the same script from other extension.
It more complex problem, than just "where to render the script".

as @chrisdavenport said: need automatic dependency resolving.
there already was suggestion and pull to use Require.js as one of possible solution #1227 , but also not perfect (not enough a couple backed gears) ... and discussion about it https://groups.google.com/forum/?fromgroups#!topic/joomla-dev-cms/s2Wqo7eNKfo

Idea is that each extension can register js library, and Joomla! will count the dependency automatically when some extension add script.js .
I tried paint some example for Require.js (can be without of course)

Also not prefect but...

I think this topic require a more complex discussion about JS back-end API refactoring.

I would like to point some things here:

1. The only js that is mandatory to go in head is core.js, everything else can go to bottom (some work is needed in front end [finder] and back end [toolbar buttons]) even in 3.x. Especially if there are some easy to copy paste snippets for dev's to opt in immediately. And it's only few lines of code e.g.

/** replace a js that CAN be placed in bottom
if(version_compare(JVERSION,'3.5.0','ge')) {
    $document->addScriptToBottom(JURI::root(true).'/path/to/script.js');
} else {
    $document->addScript(JURI::root(true).'/path/to/script.js');
}

/** replace a js that CAN NOT be placed in bottom and requires Jquery in head
if(version_compare(JVERSION,'3.5.0','ge')) {
$document->addScriptToHead(JURI::root(true).'/path/to/jquery.js');
$document->addScriptToHead(JURI::root(true).'/path/to/noconflict.js');
    $document->addScriptToHead(JURI::root(true).'/path/to/script.js');
} else {
    $document->addScript(JURI::root(true).'/path/to/script.js');
}

1. CDN can be a global thing for all assets and one plugin can rewrite all the paths (many available in the extensions directory)

2. The dependency and hierarchy of the plugins I think is out of the scope of the core. Let me explain: joomla still uses two js frameworks although sortly it will use only jquery. So basically out of the box all the scripts have to be jquery or vanilla. If vanilla no problems for dependency, if jquery that is taken care of the joomla. Now in case a developer needs something else I guess he/she has to find the way around.

Dimitris

The only js that is mandatory to go in head is core.js, everything else can go to bottom (some work is needed in front end [finder] and back end [toolbar buttons]) even in 3.x.

This may be true for core. But it's not true for 3rd party extensions. As soon as you have inline scrips depending on for example jQuery, then jQuery needs to be loaded in head.
That's why by default all scripts need to be loaded in head, except if the extension (or site builder) tells otherwise. Head is always a safe place, bottom (and defer and async) may fail depending on the script.

@Bakual I believe that even for the next version (e.g. 3.5) this can be done in a way that is fully B/C. Since we have the array of scripts we can check if a module, component or plugin js is loaded with the new logic and if not put everything in head, else follow the new preferred way and load it where is specified. It should be fairly easy to check if the parameter $placement is set and do some basic array manipulation.

@chrisdavenport I think you're trying to solve two things here. One thing is the ordering. You made a good example why it should be possible for a site builder to change the order of script loading. And I agree that this is an area Joomla needs to be improved. There are too many hackish plugins around which try to solve that. And given that those plugins exist, there is a need for it
I'm not sure if it needs to be a jdoc command though. I think the main issue here is that all scripts are just loaded into the document in the order in which they were added with no way to alter it afterwards. It's not so much about dependancy but about scripts blocking each other. The dependancy itself isn't a problem imho, because when you add the scripts in the correct order, they will also be rendered in that order. So dependancy itself shouldn't fail.
If we could add an API to change the order of the JDocument->_scripts array somehow, that would be all that is needed. It could then be done easily in a plugin instead of the template and would work without the template actually supporting it.

The other thing you try to solve is the ability to load scripts at the bottom of the body. Here I would prefer the suggestion from Don.

Personally I wouldn't mix the two things as they don't necessary need to be tied together.

I think if we have an API to easily manipulate the JDocument->_scripts array in a plugin, you could also adjust the defer/async/placement parameters there, and even add or remove a particular script.

@Fedik :

I not liked additional layer JDocumentAsset, just for add script/style.
I see no sense in

You only need to add a jdoc:include type="javascript" or jdoc:include type="css" if you want to override something that has been specified by the extension (eg. load position (head/tail), load order, CDN url, etc.).

because in the template I can use that will work >faster , and I can add it in the ordering that I need.

Yes, you can. And currently that is usually the only way to solve these sorts of problems. On many occasions I have just abandoned Joomla's script and stylesheet rendering and replaced it all with hard-wired statements like that. But that's basically an admission that Joomla is just not up to the job. What I'd like to do is fix Joomla so it is up to the job.

there already was suggestion and pull to use Require.js as one of possible solution

Looks interesting. I'm not a JS developer and I've never used require.js, but from looking through the PR and the comments you refer to I don't see any reason why these two proposals should not complement one another.

Note that my proposal will manage dependencies for stylesheets as well as JavaScript too.

It should be fairly easy to check if the parameter $placement is set and do some basic array manipulation.

Of course. Just make the $placement defaulting to head and everything will work. That's what I mean
If it's not specified, it will act as always has been. If it's specified, it will depend on the template to support it.

I see no sense in because in the template I can use that will work faster , and I can add it in the ordering that I need.

@Fedik If you use <script> to load for example jQuery, you may end up with jQuery loaded twice on your site

Why movin JS to bottom at all ? Since modern browsers should know defer and async, i see now value in putting scripts to the bottom. So the only thing we have to deal with is a dependency management.

@Bakual

If we could add an API to change the order of the JDocument->_scripts array somehow, that would be all that is needed. It could then be done easily in a plugin instead of the template and would work without the template actually supporting it.

That already exists. You can manipulate the order within any of those arrays in a plugin with the onBeforeCompileHead event.

What you can't do is intermingle internal and external scripts (eg. that one-line no conflict js that needs to site between jQuery and Mootools).

My proposal was also aimed very much at non-developers (well, non-PHP-developers anyway) who wouldn't have a clue how to write a site-specific plugin to manipulate the head arrays.

@sybrek

Why movin JS to bottom at all ? Since modern browsers should know defer and async, i see now value in putting scripts to the bottom. So the only thing we have to deal with is a dependency management.

As I understand it, defer and async are not supported across all browsers.

@Bakual We are on the same page just one reminder: we will have also the new jdoc:include so what I really suggest is default the placement to head for {jdoc:include type "head"} ( B/C) and the {jdoc:include type "scriptshead"} only if placement is null for some js.

@chrisdavenport only IE <= 9 is a bit buggy. http://caniuse.com/script-defer

@sybrek Unfortunately you cannot defer and async everything and still javascript is loaded in series (due to nature of code and dependancies), so there is a real need to put javascripts at the end of the body.

@chrisdavenport

That already exists. You can manipulate the order within any of those arrays in a plugin with the onBeforeCompileHead event.

Yes, I know. But as far as I understand it, it's not an easy task (I may be wrong) to code and also to the settings of the plugin.

What you can't do is intermingle internal and external scripts (eg. that one-line no conflict js that needs to site between jQuery and Mootools).

Yeah, that would be helpful in some cases.

My proposal was also aimed very much at non-developers (well, non-PHP-developers anyway) who wouldn't have a clue how to write a site-specific plugin to manipulate the head arrays.

My idea would be a simple to configure plugin which manipulates the arrays. Then it would be even simpler than jdoc commands since the non-dev wouldn't have to edit any files at all. And it would work with all templates.
I think that should be the right approach. Make it possible to create (and share on JED) a simple to configure plugin which can achieve that sort of stuff.

@chrisdavenport && @Bakual Should we be looking into a WG for this feature? If so, we can pick it up there and close this issue.

What will be really interesting is to get require.js in joomla. And there is a pr #1227

I'm going to close this in the meantime as it doesn't seem feasible from the comments to get this into Joomla 3.x and it hasn't had any proper activity in a year and a quarter