Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mz/bundle test cover #1146

Merged
merged 10 commits into from
Oct 3, 2024
Merged

Mz/bundle test cover #1146

merged 10 commits into from
Oct 3, 2024

Conversation

mingxuanzhangsfdx
Copy link
Member

@mingxuanzhangsfdx mingxuanzhangsfdx commented Oct 2, 2024

What does this PR do?

This PR adds several checks for bundling to prevent potential changes from breaking the process, including:

  1. avoid dynamic imports.
    i. report an error when detecting dynamic imports. —supported:dynamic-import=false and
    ii. set log-override as error for dynamic imports
  2. pay attention to the usage of fs read from a local file - generally esbuild will not be able to bundle the local file referenced by fs directly, tsc can do so by specifying the artifacts in tsconfig.json
    i. add a check in husky whenever there is a new commit to send warnings of every fs.read* usage in the codebase
  3. if there are modules that are not bundle-able, and we are not aware of, the test workflow should fail.
  4. the relative path to get transformStream in pino-logger should not be changed.
    i. check during husky commit hook
    ii. report an error if the target reference does not exist in updateForBundling.js
    iii. add explanation to where transformStream is referenced to raise developers’ caution
    iv. during bundling, if const searchString = /${process.cwd()}${require("path").sep}tmp-lib/g; is not detected in the bundled file, an error should be raised and let Mingxuan Zhang or others from IDEx team know to figure it out.

What issues does this PR fix or reference?

@W-16304459@

@mingxuanzhangsfdx mingxuanzhangsfdx requested a review from a team as a code owner October 2, 2024 20:39
@@ -25,17 +25,12 @@ jobs:
registry-url: 'https://registry.npmjs.org'
cache: yarn
- uses: salesforcecli/github-workflows/.github/actions/yarnInstallWithRetries@main
- name: Install esbuild Dependencies
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I specify these dev dependencies in package.json, so I removed this step in the workflow.

- name: Update for Bundling
run: |
node scripts/updateForBundling.js
- name: Generate Bundle
- name: Generate Bundle and Check if there is any dependency that needs to be externalized and is not in the whitelist.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used this script to bundle to detect if there is any dependencies that should be externalized.

@@ -2,3 +2,4 @@
. "$(dirname "$0")/_/husky.sh"

yarn lint && yarn pretty-quick --staged
node ./scripts/scanTs.js
Copy link
Member Author

@mingxuanzhangsfdx mingxuanzhangsfdx Oct 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added this extra step to scan all ts artifacts to list usages of fs.read*

peternhale
peternhale previously approved these changes Oct 3, 2024
@@ -19,6 +19,8 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated the node version to the latest because originally it uses node v18 which is not compatible with the dev dependency, npm-dts, which I just added

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the minimum supported node version for sfdx-core? Is there a way to not change nodejs version?

@@ -19,6 +19,8 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the minimum supported node version for sfdx-core? Is there a way to not change nodejs version?

// Combine stdout and stderr to check the entire output
const output = `${stdout}\n${stderr}`;
// Check if the output contains the error string of esbuild
if (output.includes('[require-resolve-not-external]')) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the error output like, would this statement be false if there were multiple errors in that array? you could just check for require-resolve-not-external?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The output from bundling process is like this
image
So each error with details will list in a row.

const fs = require('fs');
const path = require('path');

// Function to update package.json
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could this comment be updated to why we need to update the pjson? - same for all similar jsdoc-style comments

const packageJson = JSON.parse(data);

// Update package name if necessary
if (packageJson.name && packageJson.name === '@salesforce/core') {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional accessor? packageJson?.name===?


// Function to update logger.ts
function updateLoggerTs() {
const loggerPath = './src/logger/logger.ts';
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can this and checkTransformStreamPath be combined? they're doing a lot of the same? at least share some of the variables

@WillieRuemmele WillieRuemmele merged commit 1c92ea7 into main Oct 3, 2024
73 checks passed
@WillieRuemmele WillieRuemmele deleted the mz/bundle-test-cover branch October 3, 2024 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants