Skip to content

Commit

Permalink
Update CHANGELOG
Browse files Browse the repository at this point in the history
Signed-off-by: Stefan Marr <[email protected]>
  • Loading branch information
smarr committed Jan 18, 2025
1 parent 08554ad commit ed2cc0f
Showing 1 changed file with 87 additions and 7 deletions.
94 changes: 87 additions & 7 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,83 @@

## [Unreleased]

## [1.3.0] Env Vars part of RunId, and Support for Machine-Specific Settings - 2025-01-20

This release has three major changes that will affect how ReBench interprets configurations.

### 1. `RunId` identity is now based on all details, e.g. also env vars (#271)

Until now, RunId identity was based on the command line generated from the
configuration. However, this meant, we could not distinguish between different
runs with for instance different environment variables.
From now on, RunId identity is based on all details of the configuration,
which may mean that configuration files may result in more distinct RunIds
than they did previously.

### 2. Build commands are combined and run as a single shell script (#269)

The `build:` key takes a list of commands, from which previously, each command
was run separately. The is used to detect repeated commands and avoid executing
them multiple times. However, doing this for individual commands was confusing
and not very useful. Instead, we now combine all build commands into a single
shell script.

### 3. Support for machine-specific settings (#272)

A new lowest-priority configuration level is introduced, which is meant to define
machine-specific settings. This can be useful, for instance for settings like this:

```yaml
machines:
smaller-machine:
cores: [1, 2]
larger-machine:
cores: [1, 2, 4, 8]
```
These settings are combined into a configuration based on the usual rules.
So, if a benchmark suite or a benchmark give specific values for `cores:`, these
have higher priority. However, one would likely use it, avoiding
benchmark-specific values.

When executing ReBench on the corresponding machine, these settings can be
selected with the `-m` option. Thus, the larger machine configuration is used
when running:

```bash
rebench -m larger-machine rebench.conf
```

### Other Minor Changes

#### Features
- add support for ReBenchDB API Version 2 (#236)
- expand `~` in paths, right before invocation (#240)
- add summary for columns where all values are the same (#256)

#### Changes
- make output for currently running benchmark more compact (#237)
- invoke denoise with absolute path to avoid need for having it on a secure_path (#238, #273)
- rename the machines variable/filters to a tags variable/filter (#264)
- support Python 3.13 as latest version, drop support for Python 3.8 (#268)
- reduce number of git invocations to get source information (#275)

#### Bug Fixes
- added missing machine column heading for summary table (#246)
- behave more gracefully on bare-bone setup, e.g. without git (#245)
- raise error in `TimeAdapter` to match other adapters (#254)
- make profiling with `perf` more robust (#255)
- make denoise more robust to absent tools and running as root with a user-level installation (#260)
- fix handling of ctrl-c interrupts (#262)
- avoid unnecessary warning about data reporting, when no data is to be reported (#277)

#### Development
- use black formatting (#267)
- use mypy type checking and add some annotations (#270)
- use `pip install —editable` in CI to avoid incorrect coverage reporting (#279)

**Full Changelog**: https://github.com/smarr/ReBench/compare/v1.2.0...v1.3.0

## [1.2.0] Custom Gauge Adapters - 2023-08-06

The main feature of this release is the new support for custom
Expand All @@ -20,7 +97,7 @@ Other new features:
- make the current invocation accessible in the command as `%(invocation)s` (#230)

Other changes:
- fix bug where `'None'` instead of `null` was reported to ReBenchDB (#232)
- make sure `null` is not reported as `'None'` to ReBenchDB (#232)
- fix handling of environment variables when sudo is used (#210)
- try `gtime` from MacPorts as alternative `time` command on macOS (#212)
- update py-cpuinfo to work on macOS with ARM-base CPUs (#212)
Expand All @@ -29,6 +106,8 @@ Other changes:

Thanks to @naomiGrew for the contributions!

**Full Changelog**: https://github.com/smarr/ReBench/compare/v1.1.0...v1.2.0

## [1.1.0] Denoise - 2023-02-21

This release focuses on reducing the noise from the system (#143, #144).
Expand Down Expand Up @@ -68,9 +147,9 @@ Other notable improvements:
- make gauge adapter names in configurations case-insensitive (#202)
- improve documentation (#197, #198)
- use PyTest for unit tests (#192)

**Full Changelog**: https://github.com/smarr/ReBench/compare/v1.0.1...v1.1.0


## [1.0.1] - 2020-06-23

Expand All @@ -83,7 +162,7 @@ This is a bug fix release.

This is the first official release of ReBench as a "feature-complete" product.
Feature-complete here means, it is a tried and tested tool for benchmark
execution. It is highly
execution. It is highly
[configurable](https://rebench.readthedocs.io/en/latest/config/),
[documented](https://rebench.readthedocs.io/en/latest/),
and [successfully used](https://github.com/smarr/ReBench#use-in-academia).
Expand Down Expand Up @@ -136,7 +215,7 @@ Thank you!
- added testing of Python 3.7 (#121) and ruamel.yaml (#123)
- ensure config is YAML 1.2 compliant (#123)
- added support for ReBenchDB (#129, #130)

- fixed issues with error reporting (#128)
- fixed handling of input size configuration (#117)

Expand All @@ -145,7 +224,7 @@ Thank you!
- added `--setup-only` option, to run one benchmark for each setup (#110, #115)
- added `ignore_timeout` setting to accept known timeouts without error (#118)
- added `retries_after_failure` setting (#107, #108)

- fixed data loading, which ignored warmup setting (#111, #116)
- fixed how settings are inherited for follow documentation (#112, #113)
- fixed message for consecutive failures (#109)
Expand Down Expand Up @@ -219,7 +298,8 @@ Thank you!
- [0.6.0] - 2014-05-19
- [0.5.0] - 2014-03-25

[Unreleased]: https://github.com/smarr/ReBench/compare/v1.2.0...HEAD
[Unreleased]: https://github.com/smarr/ReBench/compare/v1.3.0...HEAD
[1.3.0]: https://github.com/smarr/ReBench/compare/v1.2.0...v1.3.0
[1.2.0]: https://github.com/smarr/ReBench/compare/v1.1.0...v1.2.0
[1.1.0]: https://github.com/smarr/ReBench/compare/v1.0.1...v1.1.0
[1.0.1]: https://github.com/smarr/ReBench/compare/v1.0.0...v1.0.1
Expand Down

0 comments on commit ed2cc0f

Please sign in to comment.