This PR proposes a cleaned up structure for the application router. This is a work in progress and submitted as a PR to get feedback from the project community.
In the past, the application router was pretty messy with lots of hardcoded behavior and sometimes did some backtracking. This cleanup tries to make this easier to understand and easier to modify.

Workflow of parseing and building a URL

Parseing and building a URL applies the respective rules to the URI object given. To give the rules a bit of order, there are 3 steps per process, the BEFORE, DURING and AFTER step. For parseing, the URL would be cleaned up (decoding URL encoding, removing unnecessary index.php parts, etc.) in the BEFORE step, parse the URL into query elements in the DURING step and rename for example the pagination query element in the AFTER step.
For building a URL, finding the right Itemid would be a rule for the BEFORE step, creating the SEF URL would be done in DURING and cleaning up would be done in the AFTER step.

There are no SEF and RAW modes anymore, since the difference between the two is not as binary as the current system makes us want to believe. All URLs need the component preprocessor, enabling SEF URLs does not enable all features, etc. Instead, the behavior is read directly from the application object and should mainly be setup in the constructor of the class. That way, we are not checking a few thousand times if SEF is enabled for building a thousand URLs.

Instead of working with a JURI object, a $vars array and a variable storage in the router object itself, the routing in 4.0 should only work with a JURI object. There is still a variable storage in the router object, but the usage of that should be greatly reduced and the only reason to even use it anymore will be shown in the section below. In general, I'm trying to think of the router as something like a production line in a factory (or more mathematical: a bijection from one form to another) You put in a JURI object on one end and on each step of the process, that object gets transformed and molded into the final product.

URL inputs and their behavior

Joomla right now supports three different types of input URLs with differing behaviors.

1. URLs with an Itemid. These basically are processed as they come in.
2. URLs without an Itemid, but an option. The router takes the current request and merges these 2 URLs, giving each URL at least the current Itemid if the option of the URL and the Itemid match.
3. URLs without an Itemid and without an option. Here again, the current request is merged with this URL. This allows to simply do echo JRoute::_('?format=rss') to get the RSS feed for the current request.

The new router should only have different behavior for 2 different types of input URLs.

1. URLs with an Itemid and/or an option should be processed as they are given to the router.
2. URLs without an Itemid and without an option should get the current request merged in.

This would mean that component routers would have to do the correct lookups for the Itemid, instead of simply relying on the current Itemid being injected. The reason is, that we simply don't know if the Itemid that a component router sees is the one being forcefully injected from the router because there was no Itemid or if that Itemid was intentionally given to us. Yes, this requires a little bit more work for component developers, but on the other hand we do provide a simplified view based component router which does this for you already.

Performance implications

This change should improve performance a little bit. By removing lots of old methods and cleaning up the code, we should see an overal improvement. At the same time, only checking for the enabled features upon creating the router object should also help a little bit. Doing some simple tests showed an increase in the parseing performance from 30ms to less than 20ms on the default testing data. ?

Backwards compatibility implications

This change means 2 breaks in backwards compatibility:

1. Existing routing rules that use JRouter::attachBuildRule() will most likely not work anymore or at least need to be slightly modified.
2. Component routers need to properly lookup the Itemid.

Progress

JRouter and JRouterSite have been finished, as well as the unittests. With that, this PR is ready for primetime.