Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a pure Wasm image classification benchmark #270

Merged
merged 6 commits into from
Apr 29, 2024

Conversation

adambratschikaye
Copy link
Contributor

Add a benchmark which performs image classification in pure Wasm using tract and ONNX. This can be useful to track the performance of float heavy computations.

Add a benchmark which performs image classification in pure Wasm using
tract and ONNX. This can be useful to track the performance of float
heavy computations.
@adambratschikaye
Copy link
Contributor Author

I think the rebuild failure is due to libsodium now requiring zig 0.12, but the Dockerfile uses zig 0.10.1.

@adambratschikaye
Copy link
Contributor Author

Rebuild failure should be fixed in #271

echo "Downloading model to $MODEL_FILE"

if which wget >/dev/null ; then
wget $MODEL_URL -O $MODEL_FILE
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An assets directory needs to be created first.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, thanks I'll fix that.

@jlb6740
Copy link
Collaborator

jlb6740 commented Apr 26, 2024

@adambratschikaye I was not familiar with tract but was introduced by your issue here: bytecodealliance/wasmtime#8313, so thanks for submitting. I think getting this in now is great but ultimately maybe this directory could be refactored to include multiple inferencing scenarios.

@adambratschikaye
Copy link
Contributor Author

I'm not exactly sure what's up with the current failure. It looks like the mac tests are failing with:

stderr=```" INFO  sightglass_cli::benchmark > Using benchmark engine:
/Users/runner/work/sightglass/sightglass/engines/wasmtime/libengine.dylib
Error: dlopen(/Users/runner/work/sightglass/sightglass/engines/wasmtime/libengine.dylib, 0x0005): tried: \'/Users/runner/work/sightglass/sightglass/engines/wasmtime/libengine.dylib\' (mach-o file, but is an incompatible architecture (have \'x86_64\', need \'arm64e\' or \'arm64\')),
\'/System/Volumes/Preboot/Cryptexes/OS/Users/runner/work/sightglass/sightglass/engines/wasmtime/libengine.dylib\' (no such file),
\'/Users/runner/work/sightglass/sightglass/engines/wasmtime/libengine.dylib\' (mach-o file, but is an incompatible architecture (have \'x86_64\', need \'arm64e\' or \'arm64\'))
Error: benchmark subprocess did not exit successfully\n"```

so maybe something is up with the cached wasmtime engine for mac?

@jlb6740
Copy link
Collaborator

jlb6740 commented Apr 29, 2024

@adambratschikaye Looks like you were right. Not sure the corruption, but it was using an incorrect cached version of the wasmtime-engine. I removed the cached image for macos and reran so that it had to build again, and now all tests pass.

@jlb6740 jlb6740 merged commit 3adb762 into bytecodealliance:main Apr 29, 2024
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants