Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing: expectDiagnostics() should provide an option to ignore ordering #5818

Open
3 tasks done
garethj-msft opened this issue Jan 30, 2025 · 2 comments
Open
3 tasks done
Labels
compiler:core Issues for @typespec/compiler feature New feature or request good first issue Good for newcomers triaged:core
Milestone

Comments

@garethj-msft
Copy link
Member

Clear and concise description of the problem

Overview

Code churn can easily cause diagnostics to appear in changing orders over time, and this should, in general, not be considered a breaking change.

However, library tests that use this helper function will fail when diagnostic order changes.
It's onerous for every library test author to have to add a wrapper to pre-sort the collections; rather the core library should have a mode that enables this with a simple binary switch, as an addition to the options object.

The existing strict option could be used, but that would technically be a behavioral breaking change to the test condition.

Current

compiler/expect.ts/expectDiagnostics(
  diagnostics: readonly Diagnostic[],
  match: DiagnosticMatch | DiagnosticMatch[],
  options = {
    strict: true,
  });

Expected

compiler/expect.ts/expectDiagnostics(
  diagnostics: readonly Diagnostic[],
  match: DiagnosticMatch | DiagnosticMatch[],
  options = {
    strict: true,
    fixedOrder: true
  });

Checklist

  • Follow our Code of Conduct
  • Read the docs.
  • Check that there isn't already an issue that request the same feature to avoid creating a duplicate.
@timotheeguerin
Copy link
Member

That would be nice but I think in general it would be better practice to make sure your test only emit a single diagnostic.
Not doing order makes it also potentially harder to see what isn't matching(would just be able to say didn't find this exect diagnostic in this list).

@markcowl markcowl added the design:needed A design request has been raised that needs a proposal label Feb 3, 2025
@markcowl markcowl added this to the Backlog milestone Feb 3, 2025
@markcowl markcowl added triaged:core feature New feature or request and removed design:needed A design request has been raised that needs a proposal labels Feb 3, 2025
@markcowl
Copy link
Contributor

markcowl commented Feb 3, 2025

Have expectDiagnostics sort the diagnostics

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
compiler:core Issues for @typespec/compiler feature New feature or request good first issue Good for newcomers triaged:core
Projects
None yet
Development

No branches or pull requests

3 participants