New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix floating point representation issue #67

Merged

JosephTLyons merged 2 commits into main from fix-floating-point-representation-issue

Nov 15, 2024

Owner

JosephTLyons commented Nov 15, 2024 •

edited

Loading

// Would produce a floating point error in v1.3.1: 2.7119999999999997
FloatTestData(
  input: "2.712",
  expected_program_output: Ok(2.712),
  expected_python_output: Ok("2.712"),
)

// Would produce a floating point error in v1.3.1: 7.5776864147000005
FloatTestData(
  input: "7.5776864147",
  expected_program_output: Ok(7.5776864147),
  expected_python_output: Ok("7.5776864147"),
),

JosephTLyons added 2 commits

November 15, 2024 02:22

WIP

0e5104b

WIP

286c71c

JosephTLyons merged commit b3130d5 into main

1 check passed

JosephTLyons deleted the fix-floating-point-representation-issue branch

November 15, 2024 07:31

JosephTLyons mentioned this pull request

Fix floating-point precision errors #40

Closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet