Checking harhat version by requiring @nomicfoundation/ethereumjs-vm package leads to errors

[oumar-fall]

hardhat version: 2.18.0
hardhat-tracer version: 2.7.0

Problem

No matter the version of HardHat, I keep having the following error:

ERROR

Please upgrade your hardhat version to 2.11 or above.
This error is generated by plugin "hardhat-tracer" because it is 
dependent on some features available in hardhat >=2.11.0 <3.0.0.

npm i hardhat@latest

or

yarn add hardhat@latest

What I understand

I understand that this is comming from this part of the code:

try {
  // Try to check if hardhat version is compatible by checking if the package is available
  require("@nomicfoundation/ethereumjs-vm");
} catch {
...

Since the package is not defined in hardhat-tracer's dependencies, I kinda expect ts to throw the following error:

Your application tried to access @nomicfoundation/ethereumjs-vm, but it isn't declared in your dependencies; this makes the require call ambiguous and unsound

Which leads to the error

Potential solution

Adding the package as a dependency might be a solution

[zemse]

I just tried reproducing this but it works on 2.18 as well as 2.19, on a fresh hardhat project created using npx hardhat.

• Are you using node.js or something else like bun?
• It seems that the module resolution is working differently in your environment. Is your project public so I can have a look?

Adding the package as a dependency might be a solution

Unfortunately adding a dependency to this project would not work since this plugin intends to use an internal dependency used by hardhat. If hardhat installs the @nomicfoundation/ethereumjs-vm package, we should be able to import it usually.

[oumar-fall]

I'm indeed using node.js but the project is a monorepo leveraging on yarn workspaces with yarn 3.

Unfortunately, the project is not public...

Unfortunately adding a dependency to this project would not work since this plugin intends to use an internal dependency used by hardhat. If hardhat installs the @nomicfoundation/ethereumjs-vm package, we should be able to import it usually.

makes sense

[zemse]

With yarn v3, I also face this issue with @nomicfoundation/hardhat-toolbox package if it is required before hardhat-tracer in the hardhat config.

Error: @nomicfoundation/hardhat-toolbox tried to access @nomicfoundation/hardhat-chai-matchers (a peer dependency) but it isn't provided by your application; this makes the require call ambiguous and unsound.

And if hardhat-tracer is required first, then I can reproduce exact problem you are mentioning. So this sounds like many other packages must be facing this issue when they depend on dependencies from peer dependencies.

---

After a bunch of playing around with yarn v3/v4, I've found that you can add to your .yarnrc.yml and do yarn install:

nodeLinker: node-modules

This switches the modules stuff to old node-modules style which also get rids of the errors. But this also might prevent some benefits introduced in yarn v3.

[oumar-fall]

switching back to node-modules is unfortunately not possible for me...

I will try to investigate deeper to find a solution

[zemse]

Sure, if you find any patch that allows this plugin to work with yarn v3/v4, happy to add it here / merge your PR!

[Mouradif]

Same issue here. I wanted to use this project where I also use huff-deployer.

The huff-deployer package doesn't work with the latest Hardhat version (since they switched to Ethers 6) so I am using Hardhat 2.14 which repends on [email protected]

@oumar-fall is this the case for you too?

[zemse]

This should work for 2.14 (as it's 2.11 and above).

Are you using something like yarn v3 instead of legacy node_modules? That does not allow this plugin to get access to the VM in order to get the trace data. You can still use v1.2.1 which instead uses debug_tt rpc instead of direct VM access.

Or if your repo is public I can try having a look at the problem.

[Rubilmax]

A workaround is to set pnpMode: loose in .yarnrc.yml but it isn't optimal. Ideally, you define @nomicfoundation/ethereumjs-vm as a dependency because you import Vm from it (and any package you're importing from should be declared as a dependency of this package)

because the error is:

Error: hardhat-tracer tried to access @nomicfoundation/ethereumjs-vm, but it isn't declared in its dependencies; this makes the require call ambiguous and unsound.

[zemse]

Ideally, you define @nomicfoundation/ethereumjs-vm as a dependency because you import Vm from it (and any package you're importing from should be declared as a dependency of this package)

The v1 of this plugin was slow due to debug_tt overheads. In the current v2 of this plugin I had to use a non-standard apis from hardhat, basically hijack the JS VM that hardhat is using. Unfortunately I cannot add it as a dependency because it does not allow me to do some hacks.

But I am taking help from hardhat team to officially expose the apis that are required for this plugin to work and if it works out well, it will solve this problem.

[Rubilmax]

Thanks for the reply! I understand the hacks you are doing. But I don't get what is blocking when adding ethereumjs-vm as a dependency? Is it a typing error?

[zemse]

As patch versions are released for VM package, users might have a situation where hardhat-tracer uses a VM which is not exactly equal to the one hardhat uses, then hacky hooks added to the VM.create would not be triggered at all causing this plugin to not work. Simply reinstalling both hardhat and hardhat-tracer would fix this issue, but that would consume debug time for people who randomly find their code doesn't work like it used to (after upgrading just hardhat which will try to install latest VM on it, while tracer still depending older VM).

It would work if hardhat simply expose VM it's using, but it's risky on their side as it's very internal thing to be exposed this way. Discussing a more high-level api.

Also typing error is usually not a problem because thankfully typescript is happy even if you have two different package versions with compatible interface on deep level (unlike rust which creates a scene and you have to either do or die).

[Rubilmax]

To be clear we are not suggesting to replace the check with a dependency reference ^^ We'd simply like ethereumjs-vm to be referenced as a dependency, in addition to the require check

If this suggestion is implemented, I don't get this part:

users might have a situation where hardhat-tracer uses a VM which is not exactly equal to the one hardhat uses, then hacky hooks added to the VM.create would not be triggered at all causing this plugin to not work

How would the hacky hooks work more if the ethereumjs-vm is not referenced as a dependency than if it was? I believe the code behaves the same whether a version is referenced as dependency or not

[zemse]

When single @nomicfoundation/ethereumjs-vm exists, then npm puts it on the top level node_modules.

├── node_modules
│   ├── @nomicfoundation/ethereumjs-vm
│   ├── hardhat
│   ├── hardhat-tracer
├── package.json

This way, hardhat-tracer when it requires @nomicfoundation/ethereumjs-vm, the module resolution still works, even if it's actually not declared as a dependency. I understand that is causing issues with modern module resolutions which work differently.

If the suggestion is implemented, it is possible that versions differ and that would cause the following (if user is upgrading hardhat and there's a new patch version available for VM):

├── node_modules
│   ├── hardhat
│   │   ├── node_modules
│   │       ├── @nomicfoundation/ethereumjs-vm (v7.0.4)
│   ├── hardhat-tracer
│       ├── node_modules
│           ├── @nomicfoundation/ethereumjs-vm (v7.0.3)
├── package.json

Since VM object of v7.0.3 is updated by hardhat-tracer, it would not effect the VM v7.0.4 (for example) used by hardhat.

Also note that after adding a dependency, the require check would pass 100% of the time, because a @nomicfoundation/ethereumjs-vm would exist in the node_modules, even if user had hardhat 2.10 or lower (which used a different package for VM) or the new hardhat with EDR.

Does this make sense?

[Rubilmax]

Right, thanks for taking time to explain this, I learned something!

For now, let's consider this solution enough then!

[zemse]

Just released [email protected]. Hardhat now officially exposes APIs which this plugin needed, so this issue should be resolved. Whenever you get time please let me know if that is true.

[Rubilmax]

I can't manage to run [email protected] with [email protected]: hardhat test --v does not seem to print logs on error

[zemse]

Oh sorry, it will work with 2.23 (unreleased), this is a mistake from my side for releasing early, they actually have a pre-release of that minor version at [email protected].

I also need to add a check ensuring hardhat version should be >2.22. Will release a fix once hardhat dev branch is released. Will update here once done.

[zemse]

Just released [email protected], that should work with hardhat>=2.22.5.