?
avatar brianteeman
brianteeman
15 Feb 2021

Trying to debug an issue on the one test site I have that gets updated from the nightly update and I am seeing both beta1 and beta2 files in the media/vendor/bootstrap/js folder

Is there a problem with the build scripts?

image

avatar brianteeman brianteeman - open - 15 Feb 2021
avatar joomla-cms-bot joomla-cms-bot - change - 15 Feb 2021
Labels Added: ?
avatar joomla-cms-bot joomla-cms-bot - labeled - 15 Feb 2021
avatar richard67
richard67 - comment - 16 Feb 2021

@brianteeman Do I understand right that it is an updated installation? In this case it might be that the list of files and folders to be deleted hasn't been updated yet in script.php. But the strange names of the files marked with the red arrows make me think that it might not be this easy, and there might be really something wrong with the build scripts.

@dgrammatiko Do you have an idea what could go wrong here?

avatar dgrammatiko
dgrammatiko - comment - 16 Feb 2021

Ok, there's a problem here but before I propose any solution let me explain what these dom- and popper- are. The bootstrap components share some common code. All of them share some basic helpers for DOM, events that are stored in the dom- file while dropdown, tooltips and popovers also share another common dependency the popper- file. These files have a hash in their filename so that new versions load the appropriate new files (common dependencies). We cannot remove the hash because these files get loaded using import and if we did so then for sites with far future expire the bootstrap would be broken (keep running the old code).

There are a couple of ways that we could tackle this:

  • create a json file every time there's a release that the script.php reads as well and adds the files to the uninstall loop
  • at the start of the update rename the vendor/bootstrap to vendor/bootstrap.bak and if the update is successful then delete this folder and if it's unsuccessful restore it
avatar richard67
richard67 - comment - 16 Feb 2021

So at least I could help with pinging the right man to explain what happens ;-)

Regarding what's the better way to deal with it, the 1st or the 2nd way: To me the 2nd way seems to be easier, but I'd like to have @wilsonge 's opinion on this since he is the man for the script.php's file and folder deletion procedure.

avatar Fedik
Fedik - comment - 16 Feb 2021

@dgrammatiko can the compiler be configured to use fixed "hash name"? that would be easiest

avatar dgrammatiko
dgrammatiko - comment - 16 Feb 2021

can the compiler be configured to use fixed "hash name"? that would be easiest

Yes, it can but then Boostrap will never get the updated common dependencies on any update (these are loaded with import so they need a unique hash or sites with far future expires will be broken). The problem is not why these have a unique hash but rather that the update should rm -rf the vendor/bootstrap/js folder. That said let me check if I can use some weird ?uniqueId in the imports

avatar brianteeman
brianteeman - comment - 16 Feb 2021

I guess it all depends if we are bothered abut file name changes during nightly packages

avatar richard67
richard67 - comment - 16 Feb 2021

We will have the same problem between beta or RC versions, where we promise updates to work, not only between nightlies where we don't promise that.

avatar Fedik
Fedik - comment - 16 Feb 2021

That said let me check if I can use some weird ?uniqueId in the imports

Yes, I just wanted to suggest it. It would help to solve caching issue.

avatar dgrammatiko
dgrammatiko - comment - 16 Feb 2021

@Fedik doable:

const {
  readdir, readFile, rename, writeFile, unlink,
} = require('fs').promises;
const { resolve } = require('path');
const { minify } = require('terser');
const rimraf = require('rimraf');
const rollup = require('rollup');
const { nodeResolve } = require('@rollup/plugin-node-resolve');
const replace = require('@rollup/plugin-replace');
const { babel } = require('@rollup/plugin-babel');

const tasks = [];
const inputFolder = 'build/media_source/vendor/bootstrap/js';
const outputFolder = 'media/vendor/bootstrap/js';

const getCurrentUnixTime = Math.round((new Date()).getTime() / 1000);

const createMinified = async (file) => {
  const initial = await readFile(resolve(outputFolder, file), { encoding: 'utf8' });
  const code = initial.replace('./dom.js', `./dom.js?${getCurrentUnixTime}`).replace('./popper.js', `./popper.js?${getCurrentUnixTime}`);
  const mini = await minify(code, { sourceMap: false, format: { comments: false } });
  await writeFile(resolve(outputFolder, file), code, { encoding: 'utf8' });
  await writeFile(resolve(outputFolder, file.replace('.js', '.min.js')), mini.code, { encoding: 'utf8' });
};

const build = async () => {
  // eslint-disable-next-line no-console
  console.log('Building ES6 Components...');

  const bundle = await rollup.rollup({
    input: resolve(inputFolder, 'index.es6.js'),
    plugins: [
      nodeResolve(),
      replace({
        'process.env.NODE_ENV': '\'production\'',
      }),
    ],
    external: [
      './base-component.js',
      './dom/data.js',
      './event-handler.js',
      './dom/manipulator.js',
      './selector-engine.js',
      './util/index.js',
    ],
    manualChunks: {
      alert: ['build/media_source/vendor/bootstrap/js/alert.es6.js'],
      button: ['build/media_source/vendor/bootstrap/js/button.es6.js'],
      carousel: ['build/media_source/vendor/bootstrap/js/carousel.es6.js'],
      collapse: ['build/media_source/vendor/bootstrap/js/collapse.es6.js'],
      dropdown: ['build/media_source/vendor/bootstrap/js/dropdown.es6.js'],
      modal: ['build/media_source/vendor/bootstrap/js/modal.es6.js'],
      popover: ['build/media_source/vendor/bootstrap/js/popover.es6.js'],
      scrollspy: ['build/media_source/vendor/bootstrap/js/scrollspy.es6.js'],
      tab: ['build/media_source/vendor/bootstrap/js/tab.es6.js'],
      toast: ['build/media_source/vendor/bootstrap/js/toast.es6.js'],
      popper: ['@popperjs/core'],
      dom: [
        'node_modules/bootstrap/js/src/base-component.js',
        'node_modules/bootstrap/js/src/dom/data.js',
        'node_modules/bootstrap/js/src/dom/event-handler.js',
        'node_modules/bootstrap/js/src/dom/manipulator.js',
        'node_modules/bootstrap/js/src/dom/selector-engine.js',
        'node_modules/bootstrap/js/src/util/index.js',
      ],
    },
  });

  await bundle.write({
    format: 'es',
    sourcemap: false,
    dir: outputFolder,
    chunkFileNames: '[name].js',
  });

  // closes the bundle
  await bundle.close();
};

const buildLegacy = async () => {
  // eslint-disable-next-line no-console
  console.log('Building Legacy...');

  const bundle = await rollup.rollup({
    input: resolve(inputFolder, 'index.es6.js'),
    plugins: [
      nodeResolve(),
      replace({
        'process.env.NODE_ENV': '\'production\'',
      }),
      babel({
        exclude: 'node_modules/core-js/**',
        babelHelpers: 'bundled',
        babelrc: false,
        presets: [
          [
            '@babel/preset-env',
            {
              corejs: '3.8',
              useBuiltIns: 'usage',
              targets: {
                chrome: '58',
                ie: '11',
              },
              loose: true,
              bugfixes: true,
              modules: false,
            },
          ],
        ],
      }),
    ],
    external: [],
  });

  await bundle.write({
    format: 'iife',
    sourcemap: false,
    name: 'Bootstrap',
    file: resolve(outputFolder, 'bootstrap.es5.js'),
  });

  // closes the bundle
  await bundle.close();
};

module.exports.bootstrapJs = async () => {
  rimraf.sync(resolve(outputFolder));

  try {
    await build(resolve(inputFolder, 'index.es6.js'));
    await unlink(resolve(outputFolder, 'index.es6.js'));
  } catch (error) {
    // eslint-disable-next-line no-console
    console.error(error);
    process.exit(1);
  }

  (await readdir(outputFolder)).forEach((file) => {
    if (!(file.startsWith('dom-') || file.startsWith('popper-'))) {
      tasks.push(createMinified(file));
    }
  });

  await Promise.all(tasks).catch((er) => {
    // eslint-disable-next-line no-console
    console.log(er);
    process.exit(1);
  });
  // eslint-disable-next-line no-console
  console.log('ES6 components ready ✅');

  try {
    await buildLegacy(inputFolder, 'index.es6.js');
    const es5File = await readFile(resolve(outputFolder, 'bootstrap.es5.js'), { encoding: 'utf8' });
    const mini = await minify(es5File, { sourceMap: false, format: { comments: false } });
    await writeFile(resolve(outputFolder, 'bootstrap.es5.min.js'), mini.code, { encoding: 'utf8' });
    // eslint-disable-next-line no-console
    console.log('Legacy done! ✅');
  } catch (error) {
    // eslint-disable-next-line no-console
    console.error(error);
    process.exit(1);
  }
};

basically 3 lines:

  • const getCurrentUnixTime = Math.round((new Date()).getTime() / 1000);
  • const code = initial.replace('./dom.js',./dom.js?${getCurrentUnixTime}).replace('./popper.js', ./popper.js?${getCurrentUnixTime});
  • chunkFileNames: '[name].js',
avatar dgrammatiko
dgrammatiko - comment - 16 Feb 2021

Ok I pushed the code on #32300 with 76559fa

avatar brianteeman
brianteeman - comment - 16 Feb 2021

Might as well close this then - thanks

avatar richard67
richard67 - comment - 16 Feb 2021

@dgrammatiko So #32300 will solve this issue here, too, and there won't be those hashes in the filenames anymore?

avatar dgrammatiko
dgrammatiko - comment - 16 Feb 2021

@richard67 basically in 2 images, the content of the components becomes:
Screenshot 2021-02-16 at 11 16 46

And the filenames:
Screenshot 2021-02-16 at 11 47 16

avatar richard67
richard67 - comment - 16 Feb 2021

Perfect ?

avatar richard67
richard67 - comment - 16 Feb 2021

So what would remain to be done is to remove files from 4.0 Beta 7 which have such hashes in their name with the next Beta 8 (or RC 1 or whatever it will be), right? If so, then we maybe should leave this issue open as reminder to add them to the script.php procedure before the next Beta or RC.

avatar dgrammatiko
dgrammatiko - comment - 16 Feb 2021

@richard67 should be only 2 files dom-blabla.js and popper-blabla.js

avatar richard67
richard67 - comment - 16 Feb 2021

@dgrammatiko I think if George doesn't forget to run his script for generating the list of files to be deleted against the last Beta 7 when making the next release, then it should be fine.

@brianteeman So we can close this issue?

avatar dgrammatiko
dgrammatiko - comment - 16 Feb 2021

@brianteeman So we can close this issue?

Keep it open, till we get the other PR merged. This is kinda essential to be patched

avatar richard67
richard67 - comment - 16 Feb 2021

OK. Will test again your PR later on Linux and Windows.

avatar dgrammatiko
dgrammatiko - comment - 16 Feb 2021

@richard67 if you do check with and without debug the backend (already tested but checking again is better)

avatar dgrammatiko
dgrammatiko - comment - 19 Feb 2021

@brianteeman I think this can be closed now

avatar brianteeman brianteeman - change - 19 Feb 2021
Status New Closed
Closed_Date 0000-00-00 00:00:00 2021-02-19 16:38:31
Closed_By brianteeman
avatar brianteeman brianteeman - close - 19 Feb 2021
avatar brianteeman
brianteeman - comment - 19 Feb 2021

Done - was waiting on you ;)

Add a Comment

Login with GitHub to post a comment