Revealing constructor vs global assertions

[mindplay-dk]

This is another question about the overall architecture of the framework (or test-frameworks in general, I suppose) and just something I've been wondering about. 🤔

Tests use this sort of "revealing constructor" pattern... where the tester t is provided to you, and it's parent test has a definitive life-cycle - something like:

import { test } from 'zora';

test(`some stuff`, t => {
  t.test(`one thing`, t => {
    t.ok(true, "ok");
  });
  t.test(`another thing`, t => {
    t.ok(true, "great");
  });
});

That is, calling t.test or t.ok etc. in the one thing and another thing tests happens in the context of the some stuff test, with t having a life-cycle that begins and ends with the call and return from the spec function.

We enforce a definitive life-cycle by preventing any "late" messages:

zora/zora/src/test.js

Lines 33 to 39 in 4ba9b40

	if (done) {
	throw new Error(`test "${description}"
	tried to collect an assertion after it has run to its completion.
	You might have forgotten to wait for an asynchronous task to complete
	------
	${spec.toString()}`);
	}

So far so good.

What I'm wondering about, is that, for the top level of the test, you have this assertion object with no definitive life-cycle.

That is, nothing prevents you from creating a harness, running some tests, running a reporter - and then running more tests:

import {hold, report, test} from 'zora';
import {createDiffReporter} from 'zora-reporters';

hold(); // prevent the default test harness to start its reporting

test(`one thing`, t => {
  t.ok(true, "ok");
});

const reporter = createDiffReporter();

report({reporter});

test(`another thing`, t => {
  t.ok(false, "oh whoops"); // 👈 too late
});

Is this by design for some reason?

How come we don't use this sort of life-cycle enforcement for the top level?

Something like:

const result = await run(test => {
  test(`one thing`, t => {
    t.ok(true, "ok");
  });

  test(`another thing`, t => {
    t.ok(true, "great");
  });
});

const reporter = createDiffReporter();

report(result);

This way, you only have access to test within the scope of that main closure - there's no making assertions outside the context of an actual test being run, no altering the result after we're done running the tests, there's an error message if you attempt to make assertions after the test has finished and the report has been generated. The same guarantees provided for nested tests.

I suspect it's mostly a convenience thing? Something about the optics of "looking like most test-frameworks"?

I'm not suggesting it's a big problem or anything. 🙂

Just wondering if there's any other reason for that design?