Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarify how DevCmd should augment PATH when multiple node_modules exist in the directory hierarchy #18

Open
lehnerpat opened this issue Jan 5, 2021 · 0 comments

Comments

@lehnerpat
Copy link
Contributor

lehnerpat commented Jan 5, 2021

This issue relates to the augmentation of the PATH environment variable to allow easy use of bin scripts provided by npm packages, i.e. for use inside dev commands.
It does not affect the resolution of the right dev_cmds directory.

I had previously assumed that npm, npx, and yarn would search for bin scripts in the .bin directories of all node_modules directories up the path hierarchy. This is not the case.
Instead, only the node_modules/.bin directory next to the current package.json file (i.e. the closest package.json going up the path hierarchy) is considered. (Or some more, if the CWD is inside nested node_modules directories, but that's not very relevant for DevCmd).

This is most clearly recognizable (though still not obvious IMO) in the npm-path package, which has extracted npm's lookup/path augmentation algorithm. See getPath in timoxley/npm-path#src/index.js

We should keep this behavior in mind, because it differs significantly to how plain node modules (import/require) are resolved.
And we should consider well whether we want to stick to this behavior or make custom additions to $PATH.
And either way, document the behavior.


To illustrate, consider the following setup (which is a situation we want to support according to our design goals):

|
+-- dev_cmds/
| +-- node_modules/
| | +-- .bin/
| | \-- ...
| +-- build.ts
| \-- package.json   <-- (A)
+-- node_modules/
| +-- .bin/
|   \-- ...
+-- ...
\-- package.json     <-- (B)

Let's talk about running a dev command, i.e. starting a script in dev_cmds/ with the CWD also set to dev_cmds/.

The npm PATH augmentation (as implemented by npm and npm-path, and implemented sufficiently similarly by yarn, when running scripts) produces an augmented NPM_PATH as follows, where $PATH is the non-augmented (i.e. OS-provided) PATH:

  • If the inner package.json (A) exists, we get roughly export NPM_PATH = $(realpath dev_cmds/node_modules/.bin):$PATH. The outer bin dir (node_modules/.bin/) is not on the augmented PATH.

  • If the inner package.json (A) does not exist, we get roughly export NPM_PATH = $(realpath node_modules/.bin):$PATH. The inner bin dir (dev_cmd/node_modules/.bin/) is not on the augmented PATH.

This has the following effects:

  • If (A) does not exist and all dependencies (devcmd, ts-node, etc.) are included in (B), everything works fine.
  • If (A) does exist, and all all dependencies (devcmd, ts-node, etc.) are included in (A), anything available in (A) works fine (e.g. running other dev commands). Bin-dependencies included only in (B) cannot be directly executed.
  • If (A) does exist and includes the devcmd dependency, but the ts-node dependency is included only in (B) (e.g. because the project itself uses it), dev commands cannot be run. (At the moment, it fails silently, see Devcmd fails silently when ts-node is not available #17, but even if an error is emitted, this failure seems unexpected to me.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant