Skip to content

Commit

Permalink
Add tests for :bug
Browse files Browse the repository at this point in the history
These exposed a few issues with the output formatting, so those are also
addressed here.
  • Loading branch information
sellout committed Dec 10, 2024
1 parent de75614 commit c88d9fd
Show file tree
Hide file tree
Showing 6 changed files with 93 additions and 15 deletions.
45 changes: 30 additions & 15 deletions unison-cli/src/Unison/Codebase/Transcript/Runner.hs
Original file line number Diff line number Diff line change
Expand Up @@ -210,18 +210,19 @@ run isTest verbosity dir codebase runtime sbRuntime nRuntime ucmVersion baseURL
liftIO $ writeIORef hasErrors True
liftIO (liftA2 (,) (readIORef allowErrors) (readIORef expectFailure)) >>= \case
(False, False) -> liftIO . dieWithMsg $ Pretty.toPlain terminalWidth msg
(True, True) ->
transcriptFailure
out
( Text.pack . Pretty.toPlain terminalWidth $
Pretty.lines
[ "The stanza above marked with `:error :bug` is now failing with",
Pretty.border 2 msg,
"so you can remove `:bug` and close any appropriate Github issues. If the error message is \
\different from the expected error message, open a new issue and reference it in this transcript."
]
)
Nothing
(True, True) -> do
appendFailingStanza
fixedBug out $
Text.unlines
[ "The stanza above marked with `:error :bug` is now failing with",
"",
"```",
Text.pack $ Pretty.toPlain terminalWidth msg,
"```",
"",
"so you can remove `:bug` and close any appropriate Github issues. If the error message is different \
\from the expected error message, open a new issue and reference it in this transcript."
]
(_, _) -> pure ()

apiRequest :: APIRequest -> IO [APIRequest]
Expand Down Expand Up @@ -467,12 +468,10 @@ run isTest verbosity dir codebase runtime sbRuntime nRuntime ucmVersion baseURL
"The transcript was expecting an error in the stanza above, but did not encounter one."
Nothing
(False, True, False) -> do
appendFailingStanza
transcriptFailure
fixedBug
out
"The stanza above with `:bug` is now passing! You can remove `:bug` and close any appropriate Github \
\issues."
Nothing
(_, _, _) -> pure ()

authenticatedHTTPClient <- AuthN.newAuthenticatedHTTPClient tokenProvider ucmVersion
Expand Down Expand Up @@ -528,6 +527,22 @@ transcriptFailure out heading mbody = do
<> foldr ((:) . CMarkCodeBlock Nothing "") [] mbody
)

fixedBug :: IORef (Seq Stanza) -> Text -> IO b
fixedBug out body = do
texts <- readIORef out
-- `CMark.commonmarkToNode` returns a @DOCUMENT@, which won’t be rendered inside another document, so we strip the
-- outer `CMark.Node`.
let CMark.Node _ _DOCUMENT bodyNodes = CMark.commonmarkToNode [CMark.optNormalize] body
UnliftIO.throwIO . RunFailure $
texts
<> Seq.fromList
( Left
<$> [ CMark.Node Nothing CMark.PARAGRAPH [CMark.Node Nothing (CMark.TEXT "🎉") []],
CMark.Node Nothing (CMark.HEADING 2) [CMark.Node Nothing (CMark.TEXT "You fixed a bug!") []]
]
<> bodyNodes
)

data Error
= ParseError (P.ParseErrorBundle Text Void)
| RunFailure (Seq Stanza)
Expand Down
5 changes: 5 additions & 0 deletions unison-src/transcripts/errors/obsolete-bug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
This transcript will error, because we’re claiming that the stanza has a bug, but `help` works as expected.

``` ucm :bug
scratch/main> help edit
```
15 changes: 15 additions & 0 deletions unison-src/transcripts/errors/obsolete-bug.output.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
This transcript will error, because we’re claiming that the stanza has a bug, but `help` works as expected.

``` ucm :bug
scratch/main> help edit
edit
`edit foo` prepends the definition of `foo` to the top of the most recently saved file.
`edit` without arguments invokes a search to select a definition for editing, which requires that `fzf` can be found within your PATH.
```

🎉

## You fixed a bug\!

The stanza above with `:bug` is now passing\! You can remove `:bug` and close any appropriate Github issues.
5 changes: 5 additions & 0 deletions unison-src/transcripts/errors/obsolete-error-bug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
This transcript will fail, because we’re claiming that the stanza has a bug, but `do.something` errors as expected.

``` ucm :error :bug
scratch/main> do.something
```
19 changes: 19 additions & 0 deletions unison-src/transcripts/errors/obsolete-error-bug.output.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
This transcript will fail, because we’re claiming that the stanza has a bug, but `do.something` errors as expected.

``` ucm :error :bug
scratch/main> do.something
```

🎉

## You fixed a bug\!

The stanza above marked with `:error :bug` is now failing with

```
⚠️
I don't know how to do.something. Type `help` or `?` to get
help.
```

so you can remove `:bug` and close any appropriate Github issues. If the error message is different from the expected error message, open a new issue and reference it in this transcript.
19 changes: 19 additions & 0 deletions unison-src/transcripts/idempotent/bug.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
This tests that `:bug` behaves similarly to `:error` when the stanza fails.

``` ucm :bug
scratch/main> do.something
⚠️
I don't know how to do.something. Type `help` or `?` to get
help.
```

And when combined with `:error`, it should expect a successful result.

``` ucm :error :bug
scratch/main> help edit
edit
`edit foo` prepends the definition of `foo` to the top of the most recently saved file.
`edit` without arguments invokes a search to select a definition for editing, which requires that `fzf` can be found within your PATH.
```

0 comments on commit c88d9fd

Please sign in to comment.