[#1023] - Replace Bower in build pipeline
- Fixed in Code Base
- 25 Jun 2018
- Medium
- Build: master
- # 1023
- Diff
- joomla:webpack
- Failure continuous-integration/travis-ci/pr The Travis CI build failed Details
- Failure continuous-integration/travis-ci/push The Travis CI build failed Details
User tests: Successful: Unsuccessful:
Pull Request for Issue #973
Summary of Changes
Bower is deprecated, this moves us off of relying on it in favor of straight up NPM. For compiling, this uses Webpack with the Laravel Mix wrapper (because I can barely figure that out let alone "plain" Webpack but feel free if you want to work that out).
Testing Instructions
All the assets load. npm run prod and npm run dev (which have no practical differences right now other than minifying files) both work.
Environment Info
At this moment I've got Node 8.9.3 and NPM 6.1.0 installed. For this to work well the requirements need to be fairly recent versions of both (at least new enough to support the lock file, though I also personally use npm ci pretty heavily and the npm audit feature is pretty useful.
Current Configuration
Right now we've got a lot of outdated UI dependencies. Some of them are updated in order to be able to get things to install correctly, but there are still a lot needing updates. Add to that we have "hard" dependencies to jQuery 1.x and Bootstrap 2.3.2 by way of our template, the end result is there are some things that aren't really Webpack friendly right now. So instead of trying to compile a monolithic vendor file for each UI segment, I've combined the resources of each vendor and those get loaded/used as needed instead. With an environment supporting HTTP/2, the increased number of resources shouldn't be a problem, plus we can serve smaller responses by not throwing the entire UI stack into 2 files loaded on every page.
There are a few dependencies that aren't put into NPM right now, predominantly because they aren't on the NPM registry, or our jQuery UI build being somewhat customized.
More work can be done on optimizing how things are built, the main goal here is to be able to break the Bower dependency and keep things working, while laying infrastructure to keep moving forward.
So after testing all the JS functionality we have I'm afraid but I cant find any error
Visuals also align with the live site ;)
A small glitch with the markdown editor; there seems to be some custom CSS missing?
Some observations:
- When running
npm installthepackage.lock.jsonfile gets changed.
The tag"optional": trueis added to some packages. - When running
npm run prodall the files under/www/media/markitupare changed (including images). At least they appear as modified ingit status..
System: Some Linux, npm 6.1.0, node 8.11.3
Can we get rid of
- The grunt stuff
- The
bower.jsonfile?
Question:
Would it make sense if we take the generated files out of the repo?
I'd
RTC
Oh and the CI errors are not related to this PR, I believe the line length check is a great PITA and we should exclude this check in our repo.
Oh... and what if we move all of our custom CSS and JS to /assets and completely .gitignore the /www/media directory, or maybe even renaming it to /www/build and change the refs?
A small glitch with the markdown editor; there seems to be some custom CSS missing?
I don't know if it's custom CSS or a different version or what the case is right now. Still something to play with I guess.
When running npm install the package.lock.json file gets changed.
The tag "optional": true is added to some packages.
Looks to be related to npm/npm#17722.
When running npm run prod all the files under /www/media/markitup are changed (including images). At least they appear as modified in git status..
I forget which now but some of the files copied from node_modules come over with 755 permissions and I had to chmod a bunch of stuff. Maybe it was that.
Would it make sense if we take the generated files out of the repo?
Eventually, yes. But that means changing the deploy workflow from what it is now, which is basically SSH into server and do a git pull and composer install --no-dev -o -a. NPM isn't on the production server and I'm not planning on installing it there, but I can make use of our Jenkins system to build and deploy (just need to get set up to do that).
Oh... and what if we move all of our custom CSS and JS to /assets and completely .gitignore the /www/media directory, or maybe even renaming it to /www/build and change the refs?
Eventually, that should be the goal. Still need to shuffle more files around to reach that point though (i.e. the main template CSS files move to assets and be copied with webpack, all the jqPlot and jQuery UI stuff the same).
Oh and the CI errors are not related to this PR, I believe the line length check is a great PITA and we should exclude this check in our repo.
I made it 250 characters on another repo because I wasn't going to concatenate a long SQL query string
A small glitch with the markdown editor; there seems to be some custom CSS missing?
I don't know if it's custom CSS or a different version or what the case is right now. Still something to play with I guess.
Found it. We modified the skin from the upstream source files. So, we just need to make a new skin with our custom CSS to get the display back to what it's supposed to be.
We modified the skin from the upstream source files
Uh, bad practice... leave it for another round..
Looks to be related to npm/npm#17722.
It is.
but I can make use of our Jenkins system to build and deploy (just need to get set up to do that).
Yeah I guess that this would add some workload / another thing to look at to your list of duties which must be huge - just leave it as is ;)
Editor skin is fixed.
Well... seems that the file upload thingy is not doing its thing :(
Not sure if it's something local or what but I guess you might find out what's going on quicker then me (feeling stupid)..
And while trying on the live site I get
which, of course, is completely unrelated to the PR here ...
File upload is fixed.
Looks good now.
Awesome job @mbabker. Code is looking good, but testing failed on my test env that is currently on webpack branch.
You can try it by yourself http://projects.j4devs.ru/jtracker/tracker/jissues-test/add
Console is showing a bunch of 404 errors on "Add issue" screen.
The compiler transforms things into a full path relative to the domain root, so declarations like src:url(/media/fonts/vendor/octicons/build/octicons.eot?d038ccbc4a99be24f33a54b482b2422e); won't work in a subdirectory (that accounts for most of the 404 errors). Trying to upload something has a similar problem (tries to POST to http://projects.j4devs.ru/upload/put/ but it should be http://projects.j4devs.ru/jtracker/upload/put/). TBH I'm not sure if there is a way to fix this one without locally editing this line in the webpack config if you're pushing to a subdirectory.
I do not think there is a need to think about subdirectory in scope of this PR as we are not planning to move to subdirectory :)
| Status | New | ⇒ | Fixed in Code Base |
| Closed_Date | 0000-00-00 00:00:00 | ⇒ | 2018-06-25 11:37:15 |
| Closed_By | ⇒ | b2z |
It should, yes, since the same files are now always being loaded.
Just a small reminder:
We have some docs about asset management at https://github.com/joomla/jissues/blob/master/Documentation/Development/Asset-Management.md
so, if somebody finds some time on a rainy day... that would be awesome
I have tested this item✅ successfully on 2554f7b
?
Cool @mbabker
This comment was created with the J!Tracker Application at jissues.local/tracker/jtracker/1023.