DeviceTiming is a tool for measuring parse & execution times for JavaScript files. DeviceTiming has server and client components - the server waits for the clients to send timing data, stores it and produces static HTML reports. The client is added to your javascript files individually along with instrumentation to perform the tests. This assumes you have some kind of development or test server running your website's code, which you modify for this purpose and then restore. It is a tool built for testing in a controlled environment, use in production considered harmful.
Clone the repo and install the dependecies:
git clone https://github.com/etsy/DeviceTiming.git
cd DeviceTiming
npm installNote that DeviceTiming will modify your javascript files - it is not reccomended that you use this tool without having a backup of your code.
So first, backup your code:
cp /path/to/your/js /path/to/your/js-unmodifiedThen start the DeviceTiming server and provide a path to the javascript you wish to test.
./devicetiming server /path/to/your/jsThis will first recursively instrument all the code in the path provided.
Then it will start listening at port 8537 for beacons from the client code.
You can provide a differnt port with -p, see devicetiming -h for more.
When you're done you can restore your backed up copy:
cp /path/to/your/js-unmodified /path/to/your/jsSee devicetiming --help
To run the tests, visit your development server hosting the instrumented code from each device you wish to test. We use Adobe Edge Inspect in our device lab to perform these tests across all the devices at once. Once you've loaded a page with instrumented code in the browser, it will continuously reload the page to send additional runs to the server. It will continue until you stop it by manually closing or leaving that page.
The instrumentation assumes that the DeviceTiming server can be found at the same hostname as the locations you hit in the browser, if you need to use a different hostname for those beacons, use the --hostname argument with devicetiming server.
The first draft of our tests was done by adding a timer to our primary javascript bundle that started on the first line, and ended on the last. This effectively timed the initial execution time of the js file. However, it left out parsing time. In order to capture both we crafted the instrument.js processor for our compiled JavaScripts that performs these steps on each file:
-
Add a timer call at the top of the file, then turn the full contents into a properly escaped JavaScript string.
-
Wrap that string in an
evalcall -
Add the timing code -- first we mark time before the eval call, then from there to the one at the beginning of the eval string we call the "parse" time. From the inside eval call timer, until a timer just after eval runs captures the "exec" time. This is functionally equivalent to the original, unmodified file executing in the browser.
It seems that some combinations of devices / browsers have an optimization that caches the bytecode generated by the parsing step and then skips it whenever possible. Additionally, it's possible that browsers that don't exhibit this behavior in our tests still have this optimization, but only for scripts that are parsed through a <script> tag and not strings run through eval. In other words, a cache keyed by URI rather than content hash.
See slides and resources from Unpacking the Black Box: Benchmarking JS Parsing and Execution on Mobile Devices, a talk Daniel Espeset gave about this work at Velocity NYC 2014.
Open an issue, submit a pull request or tweet at @danielespeset on twitter.
Made by Performance and Frontend Infrastructure at Etsy.