Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate YAML-based Regex Rules to CommonJS Modules #215

Merged
merged 3 commits into from
Feb 6, 2025

Conversation

shorwood
Copy link
Contributor

@shorwood shorwood commented Jan 29, 2025

Hello !

First, thank you for building and maintaining this library—it’s been a huge help. While bundling our application into a Docker container, We ran into issues because this library depends on the file system at runtime (YAML regexes). We patched it locally by converting everything to CommonJS modules, and I’m sharing that patch here in case others encounter the same problem.

Issue

  • Bundling: This module breaks when bundled because it relies on regex definitions that aren’t parsed during bundling and therefore don’t exist in the bundled output.
pnpx esbuild ./index.js --bundle --platform=node > out.js
import DeviceDetector from 'node-device-detector'
const device = new DeviceDetector().detect(USER_AGENT) // <-- breaks

Workaround

  • Allow External Dependencies: One workaround is to bundle with external dependencies so the module can load YAML files at runtime. However, this prevents some projects from being self-contained in a single server.js file. This is an issue for us.
pnpx esbuild ./index.js --bundle --platform=node --packages=external > out.js

Solution

  • Migrate to CommonJS: Switching to CommonJS ensures node-device-detector and its regex definitions can be bundled together in a single output without relying on the presence of node_modules.

Considerations

  • Performance: Although direct require statements might raise overhead concerns, module resolution caches them by default, so performance impact should be minimal.
  • Minimal Changes: We deliberately only removed the filesystem dependency. We’ve kept the original YAML files in place for any external references.

Changes

  • Added a script to generate CommonJS modules from YAML files.
  • Converted the existing YAML-based regex definitions into .js modules.
  • Updated the parser classes to require these .js modules directly.
  • Retained the original YAML files for any external references.

Let me know what you think and thanks again for all your work on this project!

This commit introduces a new utility script that converts YAML regex files to CommonJS modules. The script recursively processes all YAML files in the regexes directory and generates corresponding .js files with the same structure. This is a preparatory step for removing the filesystem dependency in the main package, making it more suitable for various JavaScript environments.
This commit transforms the regexes detection rules from YAML to CommonJS format as part of the ongoing effort to remove filesystem dependencies. The conversion maintains all existing regex patterns and device detection logic while making the rules more suitable for direct consumption in JavaScript environments.
This commit completes the migration from YAML to CommonJS modules by removing all YAML file loading logic and replacing it with direct requires. The changes include removing the base regex directory path constant, collection map caching, and file loading methods from the abstract parser. All parser classes now directly require their respective CommonJS regex modules instead of loading YAML files at runtime.
Comment on lines +106 to +108
if (/\.ya?ml$/.test(file) === false) {
return;
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to avoid checking the new .js files.

Comment on lines -21 to -30
static ready() {
return collection !== void 0;
}

static init() {
if (helper.hasFile(path)) {
collection = helper.loadYMLFile(path);
}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer required as the collection is ready once this module is resolved.

Comment on lines -42 to -51
static ready() {
return collection !== void 0;
}

static init() {
if (helper.hasFile(path)) {
collection = helper.loadYMLFile(path);
}
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer required as the collection is ready once this module is resolved.

Comment on lines -33 to -61
get collection() {
if (!this.hasLoadCollection()) {
return null;
}
return collectionMap[this.fixtureFile];
}

hasLoadCollection() {
return collectionMap[this.fixtureFile] !== void 0
}

/**
* load collection
*/
loadCollection() {
if (!this.hasLoadCollection()) {
collectionMap[this.fixtureFile] = this.loadYMLFile(this.fixtureFile);
}
}

/**
* load yaml file
* @param {string} file
* @returns {*}
*/
loadYMLFile(file) {
return helper.loadYMLFile(BASE_REGEXES_DIR + file);
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No longer required as the collections are in memory.

@sanchezzzhak
Copy link
Owner

sanchezzzhak commented Jan 31, 2025

  1. [v] We can convert all yml files to common js
  2. !!! Releasing package as single file in npm will definitely not be done.

Why?

  1. The generated code is huge to analyze, and it looks so-so from the wrappers.
  2. Well, anyone can use and prepare the package in a single file system in pipeline/getaction/ansible/jenkins or any other system.

@sanchezzzhak
Copy link
Owner

Well, looked through all new code, everything's fine.

Starting with version 2.2.0, yml files will no publish in npm package and Package will be released without dependencies.

Thank you for contribution.

@sanchezzzhak sanchezzzhak merged commit c4de6cb into sanchezzzhak:master Feb 6, 2025
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants