Skip to content

Commit

Permalink
Support for modules build
Browse files Browse the repository at this point in the history
  • Loading branch information
laoneo committed Oct 24, 2024
1 parent 2a1c385 commit 337ab45
Show file tree
Hide file tree
Showing 8 changed files with 2,450 additions and 1,222 deletions.
8 changes: 4 additions & 4 deletions npm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,9 @@ Example
The code to perform the tasks is running on node.js. It uses tools like sass compiler, babel, rollup or minifyjs. All the local assets of an extension should not be placed in the media folder, because the transpile tasks does copy them there. So you would end in an infinite loop. We at Digital Peak do place the assets always in the resource folder on the same level as the media folder is.
If the assets file contains a "docBlock" property then all comments in the generated JS files are stripped out and the "docBlock" is added to the top of the generated file.

## ES6 modules
[ES6 module support](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules) is a first class citizen in DPDocker. Means all javascript files are created in a way that they can be loaded with the type="module" parameter. Like that are dynamic imports possible and all the javascript code is run in strict support. Modules are built with [rollup](https://rollupjs.org/es-module-syntax/) and dynamic imports are placed within the asset definition.

### Install/update
npm install or update is executed within the package folder. All dependencies will then be available in the node_modules folder.

Expand All @@ -58,7 +61,7 @@ Both tasks are transpiling the files in the same way. Sass files are transpiled
From the Javascript dependencies are only the none minified files used. The minified file is generated out of this one and not the one from the package itself. Like that we have it consistent across the whole project and it is possible to generate map files correctly for local development.

## Result
All assets are on the right location.
All assets are placed in the destination from the dest attribute. Vendor Javascript imports are placed within the vendor directory when dynamically imported whle modules are placed within the modules folder.

## Asset file documentation
The asset.json file does define the assets of an extension. It can include local assets or external dependencies.
Expand All @@ -77,9 +80,6 @@ The vendor property can contain an array of web dependencies like Javascript, CS
### docBlock
A docblock which is added to the top of the generated JS files. All the other ones are then stripped out of the file.

### Browser compatibility
The default browsers definition where the assets are compiled to, is that they must have a usage of over 0.25% and maintained or iOS/Safari 8 compatible while Internet Explorer is not supported anymore. When you want change the compatibility then you can define a compatibility property in the config part. Supported are the compatibility definitions from [babel preset env](https://babeljs.io/docs/en/babel-preset-env#targetsbrowsers), which is based on the [browserlist project](https://github.com/browserslist/browserslist).

### Example
Example of assets.json file in com_foo/resources:
```
Expand Down
2 changes: 1 addition & 1 deletion npm/run-watch.sh
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,4 @@
# @copyright Copyright (C) 2020 Digital Peak GmbH. <https://www.digital-peak.com>
# @license http://www.gnu.org/licenses/gpl-3.0.html GNU/GPL

EXTENSION=$1 INCLUDE_VENDOR= docker compose -f $(dirname $0)/docker-compose.yml run --rm watch
EXTENSION=$1 INCLUDE_VENDOR=$2 docker compose -f $(dirname $0)/docker-compose.yml run --rm watch
21 changes: 19 additions & 2 deletions npm/scripts/build.js
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,14 @@
// The needed libs
const fs = require('fs');
const path = require('path');
const js = require('./jsbuilder');
const css = require('./cssbuilder');
const util = require('./util');

util.findFilesRecursiveSync(path.resolve(process.argv[2]), 'assets.json').forEach(file => {
util.findFilesRecursiveSync(path.resolve(process.argv[2] + (3 in process.argv && process.argv[3] !== 'all' ? '/' + process.argv[3] : '')), 'assets.json').forEach((file) => {
// Loading the assets from the assets file of the extension
console.log('Started building assets from config ' + file);

const assets = JSON.parse(fs.readFileSync(file, 'utf8'));
if (!assets.config) {
assets.config = {};
Expand All @@ -21,8 +24,10 @@ util.findFilesRecursiveSync(path.resolve(process.argv[2]), 'assets.json').forEac
});

async function buildAssets(root, assets, includeVendor) {
const promises = [];

// Looping over the assets
assets.local.forEach((asset) => {
assets.local.forEach(async (asset) => {
if (!fs.existsSync(root + '/' + asset.src)) {
return;
}
Expand All @@ -36,6 +41,16 @@ async function buildAssets(root, assets, includeVendor) {
// Delete the assets directory first
util.deleteDirectory(root + '/' + asset.dest);

if (assets.modules) {
// Build the JS asset
promises.push(js.buildAsset(root, asset, assets.config));

// Build the style asset
promises.push(css.buildAsset(root, asset, assets.config));

return;
}

// Traverse the directory and build the assets
util.getFiles(root + '/' + asset.src).forEach((file) => {
// Files starting with an underscore are treated as imports and do not need to be built
Expand All @@ -48,6 +63,8 @@ async function buildAssets(root, assets, includeVendor) {
});
});

await Promise.all(promises);

// Check if the vendor dir needs to be built as well
if (!includeVendor || !assets.vendor) {
return;
Expand Down
140 changes: 140 additions & 0 deletions npm/scripts/cssbuilder.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
/**
* @package DPCalendar
* @copyright Copyright (C) 2024 Digital Peak GmbH. <https://www.digital-peak.com>
* @license http://www.gnu.org/licenses/gpl-3.0.html GNU/GPL
*/

// The needed libs
const fs = require('fs');
const path = require('path');
const sass = require('sass');
const watch = require('node-watch');
const util = require('./util');

async function buildAsset(root, asset, config) {
// Traverse the directory and build the assets
util.getFiles(root + '/' + asset.src).forEach((file) => {
// Files starting with an underscore are treated as imports and do not need to be built
if (path.basename(file).indexOf('_') === 0 || file.indexOf('.js') !== -1) {
return;
}

// Transpile the file
transpile(file, file.replace(asset.src, asset.dest).replace('scss', 'css').replace('.css', '.min.css'), config);
});
};

function watchAssets(root, assets) {
console.log('Building index for ' + root);

let index = [];
let filesToWatch = [];

// Load extra dev assets when available
if (assets.localDev) {
assets.localDev.forEach((asset) => filesToWatch.push(root + '/' + asset));
}

assets.local.forEach((asset) => {
if (!fs.existsSync(root + '/' + asset.src)) {
return;
}

filesToWatch.push(root + '/' + asset.src);

// If it is a single file add it to the index only
if (!fs.lstatSync(root + '/' + asset.src).isDirectory()) {
index[root + '/' + asset.src] = root + '/' + asset.dest;
return;
}

// Traverse the directory and build the assets
util.getFiles(root + '/' + asset.src).forEach(file => {
// Files starting with an underscore are treated as imports and do not need to be built
if (path.basename(file).indexOf('_') === 0 || file.indexOf('.js') !== -1) {
return;
}

index[file] = file.replace(asset.src, asset.dest).replace('scss', 'css');
});
});

console.log('Watching ' + root + ' for changes');
watch(
filesToWatch,
{ recursive: true },
(type, file) => {
if (type != 'update' || file.indexOf('.js') !== -1) {
return;
}

if (index[file] == null) {
console.log('Building the whole extension because of file: ' + file.replace(root + '/', ''));
assets.local.forEach((asset) => buildAsset(root, asset, assets.config));
return;
}

console.log('Transpiling the file ' + file.replace(root + '/', '') + ' to ' + index[file].replace(root + '/', ''));
try {
transpile(file, index[file], assets.config);
} catch (e) {
console.log(e.message);
}
}
);
}

/**
* Transpile function which can handle Javascript, SASS and CSS files.
*
* @param string source The full path of the source file
* @param string destination The full path of the destination file
* @param object config Some configuration options
*/
function transpile(source, destination, config) {
// Ensure that the target directory exists
if (!fs.existsSync(path.dirname(destination))) {
fs.mkdirSync(path.dirname(destination), { recursive: true });
}

// Transpile the files
switch (path.extname(source).replace('.', '')) {
case 'scss':
case 'css':
// Define the content and file path, based on the extension of the destination
if (destination.indexOf('.min.css') > -1) {
const result = sass.compile(source, { outFile: destination, style: 'compressed', loadPaths: [config.moduleRoot + '/node_modules'] });

// Write the minified content to the file
fs.writeFileSync(destination, (config.docBlock ? config.docBlock + '\n' : '') + result.css.toString().trim());
return;
}

// Compile sass files
const result = sass.compile(source, {
outFile: destination,
style: 'expanded',
indentType: 'tab',
indentWidth: 1,
sourceMap: true,
loadPaths: [config.moduleRoot + '/node_modules']
});

// Normalize the extension
destination = destination.replace('.css', '.min.css');

// Write the map content to the destination file with the adjusted paths
const mapRoot = destination.substring(0, destination.indexOf('/resources') + 1);
const segments = destination.substring(destination.indexOf('/media')).split('/');
fs.writeFileSync(destination + '.map', JSON.stringify(result.sourceMap).replaceAll('file://' + mapRoot, '../'.repeat(segments.length - 2)));

// Write the content to the destination file with the source mapping
fs.writeFileSync(destination, result.css.toString().trim() + '\n/*# sourceMappingURL=' + path.basename(destination) + '.map */');
}
}

module.exports = {
buildAsset: buildAsset,
watchAssets: watchAssets,
transpile: transpile
}
155 changes: 155 additions & 0 deletions npm/scripts/jsbuilder.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
/**
* @package DPCalendar
* @copyright Copyright (C) 2024 Digital Peak GmbH. <https://www.digital-peak.com>
* @license http://www.gnu.org/licenses/gpl-3.0.html GNU/GPL
*/

// The needed libs
const path = require('path');
const rollup = require('rollup');
const resolve = require('@rollup/plugin-node-resolve');
const replace = require('@rollup/plugin-replace');
const terser = require('@rollup/plugin-terser');
const svg = require('rollup-plugin-svg');
const vue = require('rollup-plugin-vue');
const license = require('rollup-plugin-license');
const css = require('rollup-plugin-import-css');
const urlresolve = require('rollup-plugin-url-resolve');
const util = require('./util');

async function buildAsset(root, asset, config) {
try {
const rollupConfig = getConfig(root, asset, config);
if (Object.keys(rollupConfig.input).length === 0) {
return;
}

// Compress and remove debug and comments
rollupConfig.plugins.push(terser({
module: true,
compress: {
drop_console: true,
drop_debugger: true,
},
format: { comments: false }
}));

// Add the license header
rollupConfig.plugins.push(license({ banner: { content: config.docBlock } }));

// Create the rollup instance
const bundle = await rollup.rollup(rollupConfig);

// Generate code
await bundle.write(rollupConfig.output);
} catch (e) {
console.log(e);
}
};

async function watchAssets(root, assets) {
assets.local.forEach(async (asset) => {
const rollupConfig = getConfig(root, asset, assets.config);
if (Object.keys(rollupConfig.input).length === 0) {
return;
}

// Delete the directory
util.deleteDirectory(root + '/' + asset.dest);

try {
// Use source maps
rollupConfig.output.sourcemap = true;

// Watch the files
const bundle = await rollup.watch(rollupConfig);

bundle.on('event', (event) => {
switch (event.code) {
case 'START':
startBundleTime = Date.now();
break;
case 'BUNDLE_START':
console.log(`Started to create the following bundles to ${event.output}:\n - ${Object.values(event.input).join('\n - ').replaceAll(root, '')}`);
break;
case 'BUNDLE_END':
console.log(`Finished to create bundles ${Array.isArray(event.output) ? event.output.join(',') : event.output} in ${Date.now() - startBundleTime}ms`);
break;
case 'END':
console.log(`Waiting for changes`);
break;
default:
break;
}
});
} catch (e) {
console.log(e);
}
});
}

function getConfig(root, asset, config) {
// Traverse the directory and build the assets
const files = {};
util.getFiles(root + '/' + asset.src).forEach((file) => {
// Files starting with an underscore are treated as imports and do not need to be built
if (path.basename(file).indexOf('_') === 0 || file.indexOf('.js') === -1) {
return;
}

files[file.replace(root + '/js/', '').replace('.js', '')] = file;
});

return {
input: files,
output: {
dir: root + '/' + asset.dest,
format: 'es',
entryFileNames: '[name].min.js',
chunkFileNames: (chunkInfo) => {
// The path segments
const segments = (chunkInfo.facadeModuleId ?? chunkInfo.moduleIds[0]).split('/');

// Special handling for libs
if (segments.indexOf('node_modules') !== -1) {
// The root libraries name
let name = 'vendor/';

// Add the package name and module
name += segments.splice(segments.findIndex((p) => p === 'node_modules') + 1, 2).join('/').replace('@', '');

// Normalize the filename
name += '/' + chunkInfo.name.replace('.min', '').replace('.esm', '') + '.min.js';

return name;
}

// Get the module folder
const name = segments.at(-2);

// Return the internal modules with package name as subpath
return 'modules/' + (name !== 'js-modules' ? name + '/' : '') + '[name].min.js';
}
},
plugins: [
replace({
preventAssignment: true,
values: {
'process.env.NODE_ENV': JSON.stringify('development'),
'process.env.VUE_ENV': JSON.stringify('browser')
}
}),
resolve.nodeResolve({ modulePaths: [config.moduleRoot + '/node_modules'] }),
urlresolve(),
svg(),
vue(),
css()
],
context: 'window'
};
}

module.exports = {
buildAsset: buildAsset,
watchAssets: watchAssets
}
Loading

0 comments on commit 337ab45

Please sign in to comment.