Skip to content

Commit

Permalink
When comparing we can't expect the same number of events
Browse files Browse the repository at this point in the history
React 18 does do multiple "FunctionCall" events on a single frame, making it impossible to compare when sampling the same amount.

So, we rely solely on time to compare both implementations.

We've also increased the time before starting collecting data
  • Loading branch information
pirelenito committed Nov 14, 2023
1 parent 54841da commit 99b76d6
Showing 1 changed file with 6 additions and 9 deletions.
15 changes: 6 additions & 9 deletions examples/benchmarking/compare.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,7 @@ import { mkdirpSync } from 'fs-extra'

const ERROR = 10
const ITERATIONS = 10
const SAMPLE_SIZE = 500
const OFFSET_FRAMES = 2
const MINIMUM_SAMPLE_SIZE = 100

mkdirpSync('./tmp')

Expand Down Expand Up @@ -36,7 +35,7 @@ const compare = async (optionA: string, optionB: string, targetRelativePerforman
const filename = path.join(__dirname, './dist', `${example}.html`)
await page.goto(`file://${filename}`)

await timeout(100)
await timeout(1000)

await page.tracing.start({ path: traceFile })

Expand All @@ -46,13 +45,11 @@ const compare = async (optionA: string, optionB: string, targetRelativePerforman

const { traceEvents } = require(traceFile)

const events = traceEvents.filter((event: TraceEvent) => event.name === 'MinorGC' || event.name === 'FunctionCall')
const sampledEvents = events.slice(OFFSET_FRAMES, SAMPLE_SIZE + OFFSET_FRAMES)
const events = traceEvents.filter((event: TraceEvent) => event.name === 'FunctionCall')
const totalTime = events.reduce((total: number, event: TraceEvent) => total + (event.dur ?? 0), 0)

const totalTime = sampledEvents.reduce((total: number, event: TraceEvent) => total + event.dur, 0)

if (sampledEvents.length !== SAMPLE_SIZE) {
console.log(`Not enough samples. Measured "${events.length}" of target "${SAMPLE_SIZE}".`)
if (events.length < MINIMUM_SAMPLE_SIZE) {
console.log(`Not enough samples. Measured "${events.length}" of target "${MINIMUM_SAMPLE_SIZE}".`)
process.exit(1)
}

Expand Down

0 comments on commit 99b76d6

Please sign in to comment.