Skip to content

Commit

Permalink
Documented vectory gem benchmarking results (#5)
Browse files Browse the repository at this point in the history
  • Loading branch information
maxirmx authored Nov 17, 2023
1 parent 148789d commit d5bf1d1
Show file tree
Hide file tree
Showing 10 changed files with 75 additions and 15 deletions.
5 changes: 2 additions & 3 deletions README.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
= Tebako benchmarking

Tebako (https://github.com/tamatebako/tebako) is an executable packager.
This repository contains scripts that are used for tebako benchmarking and benchmarking results which are available
This repository contains scripts that are used for https://github.com/tamatebako/tebako[tebako] benchmarking and benchmarking results which are available
at results folder at https://github.com/tamatebako/tebako-benchmarking/blob/main/results/RESULTS.adoc

This is work area and does not not pretend to have a production quality or be reusable in any way.
This repository is a work area and does not not pretend to have a production quality or be reusable in any way.
53 changes: 41 additions & 12 deletions results/RESULTS.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,10 @@ image::hello-world-benchmarking.jpg["Hello, world!" benchmarking results]
A comparative analysis of the simpliest application shows that the Tebako package has a performance advantage over long runs.
This is because the runtime library is served from an in-memory file system with significantly better access times.

In short runs, Tebako loses because the package includes many files and components that are not used by the application, but are still loaded into memory.
This creates a penalty of 0.3 seconds, which is however negligible in runs experiments.
In short runs, Tebako loses because the package includes many files and components that are not used by the application, but are loaded into memory anyway.
This creates a penalty of 0.3 seconds, which is however negligible in for larger application.

== coradoc gem

https://rubygems.org/gems/coradoc
== https://rubygems.org/gems/coradoc[coradoc gem]

```
puts "Hello! This is coradoc benchmarking test."
Expand Down Expand Up @@ -63,15 +61,11 @@ argv[0].to_i.times do
Coradoc::Transformer.transform(syntax_tree)
end
```
With this test we compare intensive native Ruby code processing. Tebako package shows stable 20% performance gain.
This gain is a result of faster access to Ruby standard library as explained above.
Also the test shows that there is no difference in code execution between native Ruby and tebako package.
With this test we benchmark Ruby gem without native extensions, i.e.: intensive Ruby code processing.

image::coradoc-benchmarking.jpg[coradoc benchmarking results]

== emf2svg gem

https://rubygems.org/gems/emf2svg
== https://rubygems.org/gems/emf2svg[emf2svg gem]

```
require "tempfile"
Expand Down Expand Up @@ -99,9 +93,44 @@ argv[0].to_i.times do
end
end
```

With this test we benchmark Ruby gem with reltively small native extensions.
The test shows that Tebako package takes extra time to setup native extension because it extracts extension as a file from memfs into temp folder and load back as a shared library.

image::emf2svg-benchmarking.jpg[emf2svg benchmarking results]

== https://rubygems.org/gems/vectory[vectory gem]

```
require "tempfile"

puts "Hello! This is vectory benchmarking test."

if (argv = ARGV).empty?
puts "No arguments given"
exit(1)
end

if argv[0].to_i < 1
puts "Argument must be a positive integer"
exit(1)
end

argv[0].to_i.times do
require "vectory"

svg = Vectory::Emf.from_path(File.join(__dir__, "fixtures", "img.emf")).to_svg.content

Tempfile.create(["output", ".svg"]) do |tempfile|
tempfile.write(svg)
puts "SVG written to #{tempfile.path}"
end
end
```
With this test we benchmark Ruby gem with reltively small native extensions.
The test shows that Tebako package takes extra time to setup native extension because it extracts extension as a file from memfs into temp folder and load back as a shared library.

image::vectory-benchmarking.jpg[vectory benchmarking results]

== Execution environment

```
Expand Down
Binary file modified results/img/coradoc-benchmarking.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified results/img/emf2svg-benchmarking.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added results/img/vectory-benchmarking.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified results/raw/results.xlsx
Binary file not shown.
1 change: 1 addition & 0 deletions tebako-benchmarking.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -62,5 +62,6 @@ Gem::Specification.new do |spec|
spec.add_dependency "tebako", "~> 0.5.5"
spec.add_dependency "tebako-runtime", "~> 0.2.1"
spec.add_dependency "thor", "~> 1.2"
spec.add_dependency "vectory", "~> 0.2.0"
spec.add_dependency "yaml", "~> 0.2.1"
end
5 changes: 5 additions & 0 deletions tests/vectory-test/Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# frozen_string_literal: true

source "https://rubygems.org"

gem "vectory", "~> 0.2.0"
Binary file added tests/vectory-test/fixtures/img.emf
Binary file not shown.
26 changes: 26 additions & 0 deletions tests/vectory-test/vectory-test-run.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# frozen_string_literal: true

require "tempfile"

puts "Hello! This is vectory benchmarking test."

if (argv = ARGV).empty?
puts "No arguments given"
exit(1)
end

if argv[0].to_i < 1
puts "Argument must be a positive integer"
exit(1)
end

argv[0].to_i.times do
require "vectory"

svg = Vectory::Emf.from_path(File.join(__dir__, "fixtures", "img.emf")).to_svg.content

Tempfile.create(["output", ".svg"]) do |tempfile|
tempfile.write(svg)
puts "SVG written to #{tempfile.path}"
end
end

0 comments on commit d5bf1d1

Please sign in to comment.