Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks #44

Merged
merged 54 commits into from
Aug 15, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
dbcd9f3
Add STL benchmark from ML with diagnostic information
mborland May 18, 2023
b2a3787
Add boost.spirit.karma
mborland May 18, 2023
94a45e7
Improve readability and add boost.charconv::from_chars
mborland May 25, 2023
fd91b0f
Use r.ptr instead of last in fallback routine
mborland May 25, 2023
e016fc3
Add parser benchmark
mborland May 25, 2023
2d0f30f
Add integer and parser benchmarking
mborland May 25, 2023
f5724e8
Add boost.spirit.qi from example
mborland May 25, 2023
6c12833
Merge remote-tracking branch 'origin/develop' into benchmarks
mborland Aug 4, 2023
b7e0fed
Add additional tests
mborland Aug 4, 2023
578cbb8
Add shortest printf
mborland Aug 4, 2023
90dbcf1
Add benchmark tables
mborland Aug 4, 2023
1782f48
Add tables for MSVC
mborland Aug 4, 2023
7e968ee
Add integer from_chars benchmarks
mborland Aug 4, 2023
58b005f
Input data from MSVC 14.3
mborland Aug 4, 2023
64c3b19
Add MacOS ARM benchmarks
mborland Aug 4, 2023
e3c3f33
Add integer printf benchmark
mborland Aug 7, 2023
1319cf1
Add stl and boost integer from_chars benchmarks
mborland Aug 7, 2023
4b7719a
Add boost.lexical_cast and boost.spirit.karma integer benchmarks
mborland Aug 7, 2023
6a21e1a
Add boost.spirit.qi integer benchmarks
mborland Aug 7, 2023
d2311c6
Add boost.lexical_cast parse benchmark
mborland Aug 7, 2023
a828c87
Add boost.lexical_cast parse floating point benchmarks
mborland Aug 7, 2023
216a1cb
Add integer benchmark tables
mborland Aug 7, 2023
d5ad095
Merge remote-tracking branch 'origin/develop' into benchmarks
mborland Aug 7, 2023
fb30159
Update benchmark table with merged PR
mborland Aug 7, 2023
8e383d3
Remove obsolete benchmark file
mborland Aug 8, 2023
77c77b1
Add verification for 64-bit integer from_chars
mborland Aug 8, 2023
d09d107
Add verification to lexical cast parser
mborland Aug 8, 2023
4e8202a
Change qi parser inputs
mborland Aug 8, 2023
67af5a4
Improve uint64_t qi parsing and verification
mborland Aug 10, 2023
91ab2be
Improve uint32_t qi parsing and verification
mborland Aug 10, 2023
8f1bdf6
Fix qi parsing of floating point
mborland Aug 10, 2023
11c42f5
Update linux benchmarks
mborland Aug 10, 2023
d0cbc11
Update windows benchmarks
mborland Aug 10, 2023
47cc769
Simplify qi parser
mborland Aug 10, 2023
431a67d
Update MacOS benchmarks
mborland Aug 10, 2023
de67d9d
Add double-conversion to string
mborland Aug 10, 2023
d7ef1c2
Add google double conversion from chars
mborland Aug 10, 2023
a986f4c
Update benchmarks table
mborland Aug 10, 2023
ed59efb
Rearrange benchmarks for table of contents
mborland Aug 10, 2023
16a47b3
Add double-conversion to test jamfile
mborland Aug 10, 2023
20827fe
Document how to run the benchmarks
mborland Aug 10, 2023
5a5dae7
Update supported compilers section
mborland Aug 10, 2023
53e2fc6
Fix punctuation
mborland Aug 10, 2023
f86270a
Fix to_chars reference
mborland Aug 10, 2023
ae0d1d7
Add double-conversion to mac benchmarks
mborland Aug 11, 2023
b3bb636
Fix double-conversion in docs
mborland Aug 11, 2023
5e6044a
Add verification to DoubleToStringConverter
mborland Aug 11, 2023
9088263
Add verification to StringToDoubleConverter
mborland Aug 11, 2023
9d0f51d
Add verification to lexical_cast
mborland Aug 11, 2023
e27d540
Add verification to boost.spirit.qi for floats
mborland Aug 11, 2023
b20729e
Add verification to lexical_cast parse
mborland Aug 11, 2023
21f9044
Add policy and verification to boost spirit karma
mborland Aug 11, 2023
3af91c1
Change karma policy to match shortest
mborland Aug 14, 2023
9de42cb
Add uint32_t verification
mborland Aug 14, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions config/Jamfile
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ import modules ;
import path ;

lib quadmath ;
lib double-conversion ;

exe has_float128 : has_float128.cpp quadmath ;
exe has_double_conversion : has_double_conversion.cpp double-conversion ;

explicit has_float128 ;
explicit has_double_conversion ;
19 changes: 19 additions & 0 deletions config/has_double_conversion.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
// Copyright 2023 Matt Borland
// Distributed under the Boost Software License, Version 1.0.
// https://www.boost.org/LICENSE_1_0.txt

#include <double-conversion/double-conversion.h>

int main()
{
using namespace double_conversion;

const int flags = 0;
int processed;

auto converter = StringToDoubleConverter(flags, 0.0, 0.0, "inf", "nan");

auto result = converter.StringToDouble("0.0", 3, &processed);

return static_cast<int>(result);
}
1 change: 1 addition & 0 deletions doc/charconv.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ include::charconv/chars_format.adoc[]
include::charconv/from_chars.adoc[]
include::charconv/to_chars.adoc[]
include::charconv/reference.adoc[]
include::charconv/benchmarks.adoc[]
include::charconv/sources.adoc[]
include::charconv/copyright.adoc[]

Expand Down
251 changes: 251 additions & 0 deletions doc/charconv/benchmarks.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,251 @@
////
Copyright 2023 Matt Borland
Distributed under the Boost Software License, Version 1.0.
https://www.boost.org/LICENSE_1_0.txt
////

= Benchmarks
:idprefix: benchmarks

The values are relative to the performance of `std::printf` and `std::strtoX`.
Larger numbers are more performant (e.g. 2.00 means twice as fast, and 0.50 means it takes twice as long).

== How to run the Benchmarks

To run the benchmarks yourself navigate to the test folder, and define `BOOST_CHARCONV_RUN_BENCHMARKS` when running the tests.
An example on linux with b2: `../../../b2 cxxstd=20 toolset=gcc-13 define=BOOST_CHARCONV_RUN_BENCHMARKS STL_benchmark -a release` .

Additionally, you will need the following:

* A compiler with full `<charconv>` support:
** GCC 11 or newer
** MSVC 19.24 or newer
* https://github.com/google/double-conversion[libdouble-conversion]

== x86_64 Linux

Data in tables 1 - 4 were run on Ubuntu 23.04 with x86_64 architecture using GCC 13.1.0 with libstdc++.

=== Floating Point

.to_chars floating point with the shortest representation
|===
|Function|Relative Performance (float / double)

|std::printf
|1.00 / 1.00
|Boost.lexical_cast
|0.55 / 0.46
|Boost.spirit.karma
|1.80 / 2.62
|std::to_chars
|3.53 / 4.87
|Boost.Charconv.to_chars
|3.64 / 4.92
|Google double-conversion
|1.19 / 1.85
|===

.from_chars floating point with scientific formatting
|===
|Function|Relative Performance (float / double)

|std::strto(f/d)
|1.00 / 1.00
|Boost.lexical_cast
|0.33 / 0.42
|Boost.spirit.qi
|3.17 / 4.65
|std::from_chars
|3.23 / 5.77
|Boost.Charconv.from_chars
|3.28 / 5.75
|Google double-conversion
|1.16 / 1.30
|===

=== Integral

.to_chars base 10 integers
|===
|Function|Relative Performance (uint32_t / uint64_t)

|std::printf
|1.00 / 1.00
|Boost.lexical_cast
|1.77 / 1.41
|Boost.spirit.karma
|2.55 / 1.47
|std::to_chars
|3.86 / 2.25
|Boost.Charconv.to_chars
|3.81 / 2.25
|===

.from_chars base 10 integers
|===
|Function|Relative Performance (uint32_t / uint64_t)

|std::strto(ul,ull)
|1.00 / 1.00
|Boost.lexical_cast
|0.53 / 0.52
|Boost.spirit.qi
|2.24 / 1.49
|std::from_chars
|1.97 / 1.68
|Boost.Charconv.from_chars
|2.54 / 1.78
|===

== x86_64 Windows

Data in tables 5 - 8 were run on Windows 11 with x86_64 architecture using MSVC 14.3 (V17.7.0).

=== Floating Point

.to_chars floating point with the shortest representation
|===
|Function|Relative Performance (float / double)

|std::printf
|1.00 / 1.00
|Boost.lexical_cast
|0.50 / 0.70
|Boost.spirit.karma
|2.23 / 7.58
|std::to_chars
|5.58 / 15.77
|Boost.Charconv.to_chars
|5.62 / 15.26
|===

.from_chars floating point with scientific formatting
|===
|Function|Relative Performance (float / double)

|std::strto(f/d)
|1.00 / 1.00
|Boost.lexical_cast
|0.14 / 0.20
|Boost.spirit.qi
|2.03 / 4.58
|std::from_chars
|1.01 / 1.23
|Boost.Charconv.from_chars
|2.06 / 5.21
|===

=== Integral

.to_chars base 10 integers
|===
|Function|Relative Performance (uint32_t / uint64_t)

|std::printf
|1.00 / 1.00
|Boost.lexical_cast
|0.68 / 0.68
|Boost.spirit.karma
|2.75 / 1.67
|std::to_chars
|2.75 / 2.10
|Boost.Charconv.to_chars
|2.75 / 2.06
|===

.from_chars base 10 integers
|===
|Function|Relative Performance (uint32_t / uint64_t)

|std::strto(ul,ull)
|1.00 / 1.00
|Boost.lexical_cast
|0.46 / 0.39
|Boost.spirit.qi
|1.94 / 1.63
|std::from_chars
|2.43 / 2.18
|Boost.Charconv.from_chars
|2.68 / 2.27
|===

== ARM MacOS

Data in tables 9-12 were run on MacOS Ventura 13.5 with M1 Pro architecture using Homebrew GCC 13.1.0 with libstdc++.

=== Floating Point

.to_chars floating point with the shortest representation
|===
|Function|Relative Performance (float / double)

|std::printf
|1.00 / 1.00
|Boost.lexical_cast
|0.52 / 0.12
|Boost.spirit.karma
|1.40 / 1.40
|std::to_chars
|3.01 / 2.96
|Boost.Charconv.to_chars
|3.03 / 2.96
|Google double-conversion
|1.22 / 1.16
|===

.from_chars floating point with scientific formatting
|===
|Function|Relative Performance (float / double)

|std::strto(f/d)
|1.00 / 1.00
|Boost.lexical_cast
|0.06 / 0.06
|Boost.spirit.qi
|1.12 / 1.06
|std::from_chars
|1.32 / 1.65
|Boost.Charconv.from_chars
|1.28 / 1.63
|Google double-conversion
|0.45 / 0.32

|===

=== Integral

.to_chars base 10 integers
|===
|Function|Relative Performance (uint32_t / uint64_t)

|std::printf
|1.00 / 1.00
|Boost.lexical_cast
|2.08 / 1.75
|Boost.spirit.karma
|4.17 / 2.06
|std::to_chars
|6.25 / 4.12
|Boost.Charconv.to_chars
|6.25 / 4.12
|===

.from_chars base 10 integers
|===
|Function|Relative Performance (uint32_t / uint64_t)

|std::strto(ul,ull)
|1.00 / 1.00
|Boost.lexical_cast
|0.56 / 0.54
|Boost.spirit.qi
|1.39 / 1.33
|std::from_chars
|1.92 / 1.65
|Boost.Charconv.from_chars
|2.27 / 1.65
|===

Special thanks to Stephan T. Lavavej for providing the basis for the benchmarks.

5 changes: 3 additions & 2 deletions doc/charconv/chars_format.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,8 @@ The integer part will be between 0 and 9 inclusive. The fraction and exponent wi
The exponent will always have a minimum of 2 digits.

=== Fixed Format
Fixed format will be of the form `2.30` or `3090`. An exponent will not appear with this format. If the precision of `to_chars` exceeds that of the type (e.g. std::numeric_limits<double>::chars10), 0s will be appended to the end.
Fixed format will be of the form `2.30` or `3090`. An exponent will not appear with this format.
If the precision of `to_chars` exceeds that of the type (e.g. `std::numeric_limits<double>::chars10`), 0s will be appended to the end.

=== Hex Format
Hex format will be of the form `1.0ep+5`. The integer part will always be 0 or 1. The exponent will be with a p instead of an e like in base 10 formats, because e is a valid hex value.
Expand All @@ -45,4 +46,4 @@ Each hexadecimal digit corresponds to a specific group of bits, making it easier
This can be helpful for debugging or analyzing floating-point arithmetic operations (e.g. Computing https://en.wikipedia.org/wiki/Unit_in_the_last_place[ULP] distances).

=== General
General format will be the shortest representation of a number in either fixed or general format (e.g. `1234` instead of `1.234e+03`.
General format will be the shortest representation of a number in either fixed or general format (e.g. `1234` instead of `1.234e+03`.
4 changes: 2 additions & 2 deletions doc/charconv/from_chars.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ from_chars_result from_chars(const char* first, const char* last, Real& value, c
** std::errc::invalid_argument - invalid argument (e.g. parsing a negative number into an unsigned type)
** std::errc::result_out_of_range - result out of range (e.g. overflow)
* operator== - compares the values of ptr and ec for equality
* operator!- - compares the value of ptr and ec for inequality
* operator! - compares the value of ptr and ec for inequality

== from_chars
* first, last - valid range to parse
Expand All @@ -53,7 +53,7 @@ from_chars_result from_chars(const char* first, const char* last, Real& value, c
* On std::errc::result_out_of_range we return ±0 for small values (e.g. 1.0e-99999) or ±HUGE_VAL for large values (e.g. 1.0e+99999) to match the handling of `std::strtod`.
This is a divergence from the standard which states we should return the `value` argument unmodified.
* These functions have been tested to support all built-in floating-point types and those from C++23's `<stdfloat>`
** Long doubles can be either 64, 80, or 128-bit but must be IEEE 754 compliant. An example of a non-compliant, and therefore unsupported format is `ibm128`.
** Long doubles can be either 64, 80, or 128-bit, but must be IEEE 754 compliant. An example of a non-compliant, and therefore unsupported format is `ibm128`.
** Use of `__float128` or `std::float128_t` requires compiling with `-std=gnu++xx` and linking GCC's `libquadmath`.

== Examples
Expand Down
2 changes: 1 addition & 1 deletion doc/charconv/overview.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ assert(!strncmp(buffer, "123456", 6)); // Strncmp returns 0 on match

== Supported Compilers

* GCC 4.8 or later
* GCC 5 or later
* Clang 3.7 or later
* Visual Studio 2015 (14.0) or later

Expand Down
8 changes: 4 additions & 4 deletions doc/charconv/to_chars.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ struct to_chars_result
};

template <typename Integral>
BOOST_CHARCONV_CONSTEXPR to_chars_result(char* first, char* last, Integral value, int base = 10) noexcept;
BOOST_CHARCONV_CONSTEXPR to_chars_result to_chars(char* first, char* last, Integral value, int base = 10) noexcept;

template <typename Integral>
BOOST_CHARCONV_CONSTEXPR to_chars_result<bool>(char* first, char* last, Integral value, int base) noexcept = delete;
BOOST_CHARCONV_CONSTEXPR to_chars_result to_chars<bool>(char* first, char* last, Integral value, int base) noexcept = delete;

template <typename Real>
to_chars_result(char* first, char* last, Real value, chars_format fmt = chars_format::general, int precision) noexcept;
to_chars_result to_chars(char* first, char* last, Real value, chars_format fmt = chars_format::general, int precision) noexcept;
----

== to_chars_result
Expand All @@ -48,7 +48,7 @@ See xref:chars_format.adoc[chars_format overview] for description.

=== to_chars for integral types
* All built-in integral types are allowed except bool which is deleted
* from_chars for integral type is constexpr (BOOST_CHARCONV_CONSTEXPR is defined) when compiled using `-std=c++14` or newer and a compiler with `__builtin_ is_constant_evaluated`
* from_chars for integral type is constexpr (BOOST_CHARCONV_CONSTEXPR is defined) when compiled using `-std=c++14` or newer and a compiler with `\__builtin_ is_constant_evaluated`
* These functions have been tested to support `\__int128` and `unsigned __int128`

=== to_chars for floating point types
Expand Down
Loading