[BUG] @suppress wrapping examples in tests suppresses test report

[mtfishman]

@lkdvos I noticed putting @suppress here:

ITensorPkgSkeleton.jl/templates/test/test/runtests.jl

Line 45 in 07abef5

@suppress @eval @safetestset $file begin

suppresses the output of the example tests, which was confusing me since I wasn't sure if they were running properly.

Probably you did that because putting macro calls inside of @safetestset fails (or at least I'm having trouble getting that to work). Maybe we can just use @eval module $(gensym()) there, or we can make our own @safetestset that works with macros?

[mtfishman]

This issue is less pressing if #37 is implemented since the results of those tests would be captured by the outer testset anyway.

[lkdvos]

Yeah, the reason is indeed that putting things inside the macro is a bit tricky. The point is that @safetestset does exactly what it promises: It attempts to make sure that no code can leak out or into this specific testset, such that for all intents and purposes this can be considered as a test run in isolation.
Putting @suppress inside of the the test code violates that, since that macro is not available in the isolated namespace.
(This is also why I directly interpolate the file into the expression, since you cannot easily access that variable from within the isolated namespace)
In principle it's not too hard to bypass this, or come up with our own @safetestset, but something like this probably also works:

@safetestset $file begin
using Suppressor: @suppress
@suppress include($file)
end

[mtfishman]

Oh I see, yes that would probably work. I just assumed it was a deeper problem with @safetestset since I've had trouble with using macros inside@safetestset before, but that solution sounds good (assuming it works).