Performance

Performance results

We measure the performance with this package with two scenerio:

1. python startup time when importing popular packages;
2. pyperformance results against supported version w./w.o. cds enabled.

Python import time

$ nox -s test_import_third_party_perf
...
$ pyperf compare_to perf-import-3.8-raw.json perf-import-3.8-cds.json --table | tee perf-import-3.8.table
+----------------------+---------------------+------------------------+
| Benchmark            | perf-import-3.8-raw | perf-import-3.8-cds    |
+======================+=====================+========================+
| boto3                | 118 ms              | 101 ms: 1.17x faster   |
+----------------------+---------------------+------------------------+
| requests             | 80.9 ms             | 69.4 ms: 1.17x faster  |
+----------------------+---------------------+------------------------+
| numpy                | 88.4 ms             | 73.8 ms: 1.20x faster  |
+----------------------+---------------------+------------------------+
| pyparsing            | 46.5 ms             | 41.7 ms: 1.11x faster  |
+----------------------+---------------------+------------------------+
| sqlalchemy           | 151 ms              | 137 ms: 1.10x faster   |
+----------------------+---------------------+------------------------+
| werkzeug             | 70.1 ms             | 59.9 ms: 1.17x faster  |
+----------------------+---------------------+------------------------+
| aiohttp              | 120 ms              | 107 ms: 1.12x faster   |
+----------------------+---------------------+------------------------+
| google-cloud-storage | 185 ms              | 160 ms: 1.16x faster   |
+----------------------+---------------------+------------------------+
| flask                | 128 ms              | 112 ms: 1.15x faster   |
+----------------------+---------------------+------------------------+
| azure-core           | 78.6 ms             | 67.2 ms: 1.17x faster  |
+----------------------+---------------------+------------------------+
| jsonschema           | 91.1 ms             | 81.8 ms: 1.11x faster  |
+----------------------+---------------------+------------------------+
| Pillow               | 32.4 ms             | 28.5 ms: 1.14x faster  |
+----------------------+---------------------+------------------------+
| scipy                | 96.9 ms             | 81.0 ms: 1.20x faster  |
+----------------------+---------------------+------------------------+
| tensorflow           | 1.50 sec            | 1.23 sec: 1.22x faster |
+----------------------+---------------------+------------------------+
| seaborn              | 758 ms              | 677 ms: 1.12x faster   |
+----------------------+---------------------+------------------------+
| azureml-core         | 511 ms              | 418 ms: 1.22x faster   |
+----------------------+---------------------+------------------------+
| pytorch              | 956 ms              | 850 ms: 1.12x faster   |
+----------------------+---------------------+------------------------+
| Geometric mean       | (ref)               | 1.15x faster           |
+----------------------+---------------------+------------------------+
$ pyperf compare_to perf-import-3.9-raw.json perf-import-3.9-cds.json --table | tee perf-import-3.9.table
+----------------------+---------------------+------------------------+
| Benchmark            | perf-import-3.9-raw | perf-import-3.9-cds    |
+======================+=====================+========================+
| boto3                | 121 ms              | 100 ms: 1.20x faster   |
+----------------------+---------------------+------------------------+
| requests             | 82.5 ms             | 70.7 ms: 1.17x faster  |
+----------------------+---------------------+------------------------+
| numpy                | 104 ms              | 86.6 ms: 1.21x faster  |
+----------------------+---------------------+------------------------+
| pyparsing            | 47.5 ms             | 43.2 ms: 1.10x faster  |
+----------------------+---------------------+------------------------+
| sqlalchemy           | 154 ms              | 138 ms: 1.12x faster   |
+----------------------+---------------------+------------------------+
| werkzeug             | 74.1 ms             | 64.5 ms: 1.15x faster  |
+----------------------+---------------------+------------------------+
| aiohttp              | 121 ms              | 107 ms: 1.13x faster   |
+----------------------+---------------------+------------------------+
| google-cloud-storage | 184 ms              | 155 ms: 1.19x faster   |
+----------------------+---------------------+------------------------+
| flask                | 129 ms              | 110 ms: 1.17x faster   |
+----------------------+---------------------+------------------------+
| azure-core           | 77.5 ms             | 64.2 ms: 1.21x faster  |
+----------------------+---------------------+------------------------+
| jsonschema           | 90.6 ms             | 80.4 ms: 1.13x faster  |
+----------------------+---------------------+------------------------+
| Pillow               | 32.1 ms             | 28.0 ms: 1.14x faster  |
+----------------------+---------------------+------------------------+
| scipy                | 110 ms              | 91.7 ms: 1.20x faster  |
+----------------------+---------------------+------------------------+
| tensorflow           | 1.44 sec            | 1.19 sec: 1.21x faster |
+----------------------+---------------------+------------------------+
| seaborn              | 766 ms              | 688 ms: 1.11x faster   |
+----------------------+---------------------+------------------------+
| azureml-core         | 496 ms              | 390 ms: 1.27x faster   |
+----------------------+---------------------+------------------------+
| pytorch              | 956 ms              | 838 ms: 1.14x faster   |
+----------------------+---------------------+------------------------+
| Geometric mean       | (ref)               | 1.17x faster           |
+----------------------+---------------------+------------------------+
$ pyperf compare_to perf-import-3.10-raw.json perf-import-3.10-cds.json --table | tee perf-import-3.10.table
+----------------------+----------------------+------------------------+
| Benchmark            | perf-import-3.10-raw | perf-import-3.10-cds   |
+======================+======================+========================+
| boto3                | 119 ms               | 97.3 ms: 1.22x faster  |
+----------------------+----------------------+------------------------+
| requests             | 82.6 ms              | 70.3 ms: 1.17x faster  |
+----------------------+----------------------+------------------------+
| numpy                | 104 ms               | 85.7 ms: 1.21x faster  |
+----------------------+----------------------+------------------------+
| pyparsing            | 47.5 ms              | 42.6 ms: 1.11x faster  |
+----------------------+----------------------+------------------------+
| sqlalchemy           | 158 ms               | 138 ms: 1.15x faster   |
+----------------------+----------------------+------------------------+
| werkzeug             | 71.9 ms              | 62.0 ms: 1.16x faster  |
+----------------------+----------------------+------------------------+
| aiohttp              | 121 ms               | 107 ms: 1.14x faster   |
+----------------------+----------------------+------------------------+
| google-cloud-storage | 182 ms               | 156 ms: 1.16x faster   |
+----------------------+----------------------+------------------------+
| flask                | 130 ms               | 110 ms: 1.18x faster   |
+----------------------+----------------------+------------------------+
| azure-core           | 75.8 ms              | 63.3 ms: 1.20x faster  |
+----------------------+----------------------+------------------------+
| jsonschema           | 90.5 ms              | 79.8 ms: 1.13x faster  |
+----------------------+----------------------+------------------------+
| Pillow               | 31.9 ms              | 27.9 ms: 1.14x faster  |
+----------------------+----------------------+------------------------+
| scipy                | 111 ms               | 92.7 ms: 1.19x faster  |
+----------------------+----------------------+------------------------+
| tensorflow           | 1.47 sec             | 1.23 sec: 1.19x faster |
+----------------------+----------------------+------------------------+
| seaborn              | 759 ms               | 670 ms: 1.13x faster   |
+----------------------+----------------------+------------------------+
| azureml-core         | 500 ms               | 394 ms: 1.27x faster   |
+----------------------+----------------------+------------------------+
| pytorch              | 981 ms               | 869 ms: 1.13x faster   |
+----------------------+----------------------+------------------------+
| Geometric mean       | (ref)                | 1.17x faster           |
+----------------------+----------------------+------------------------+
$ pyperf compare_to perf-import-3.11-raw.json perf-import-3.11-cds.json --table | tee perf-import-3.11.table
+----------------------+----------------------+------------------------+
| Benchmark            | perf-import-3.11-raw | perf-import-3.11-cds   |
+======================+======================+========================+
| boto3                | 121 ms               | 98.4 ms: 1.23x faster  |
+----------------------+----------------------+------------------------+
| requests             | 82.3 ms              | 67.8 ms: 1.21x faster  |
+----------------------+----------------------+------------------------+
| numpy                | 104 ms               | 84.5 ms: 1.23x faster  |
+----------------------+----------------------+------------------------+
| pyparsing            | 52.1 ms              | 45.8 ms: 1.14x faster  |
+----------------------+----------------------+------------------------+
| sqlalchemy           | 164 ms               | 141 ms: 1.17x faster   |
+----------------------+----------------------+------------------------+
| werkzeug             | 76.8 ms              | 63.9 ms: 1.20x faster  |
+----------------------+----------------------+------------------------+
| aiohttp              | 124 ms               | 105 ms: 1.18x faster   |
+----------------------+----------------------+------------------------+
| google-cloud-storage | 180 ms               | 151 ms: 1.20x faster   |
+----------------------+----------------------+------------------------+
| flask                | 136 ms               | 112 ms: 1.21x faster   |
+----------------------+----------------------+------------------------+
| azure-core           | 81.7 ms              | 66.9 ms: 1.22x faster  |
+----------------------+----------------------+------------------------+
| jsonschema           | 91.5 ms              | 79.4 ms: 1.15x faster  |
+----------------------+----------------------+------------------------+
| Pillow               | 34.0 ms              | 29.3 ms: 1.16x faster  |
+----------------------+----------------------+------------------------+
| scipy                | 112 ms               | 92.8 ms: 1.20x faster  |
+----------------------+----------------------+------------------------+
| tensorflow           | 1.60 sec             | 1.26 sec: 1.27x faster |
+----------------------+----------------------+------------------------+
| seaborn              | 785 ms               | 681 ms: 1.15x faster   |
+----------------------+----------------------+------------------------+
| azureml-core         | 519 ms               | 407 ms: 1.28x faster   |
+----------------------+----------------------+------------------------+
| pytorch              | 1.03 sec             | 894 ms: 1.15x faster   |
+----------------------+----------------------+------------------------+
| Geometric mean       | (ref)                | 1.20x faster           |
+----------------------+----------------------+------------------------+