Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add an exclude list for known failing Hifive1 tests #485

Merged
merged 2 commits into from
Jul 13, 2023

Conversation

MarekVCodasip
Copy link
Contributor

This commit adds a list of known failing tests based on: riscv-collab/riscv-openocd#869 (comment)

@TommyMurphyTM1234
Copy link
Contributor

Copy link
Collaborator

@timsifive timsifive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are some here that I feel ought to pass, but this is a good list to have for people who want to do regression testing.

@timsifive timsifive merged commit 5e62d38 into riscv-software-src:master Jul 13, 2023
@TommyMurphyTM1234
Copy link
Contributor

Sorry to resurrect an old commit but I'm confused!
With a build of all the latest tools (GCC toolchain, Spike from master/head, OpenOCD) I get this:

./gdbserver.py targets/SiFive/HiFive1.py

...

:::::::::::::::::::::::::::[ ran 72 tests in 351s ]:::::::::::::::::::::::::::
18 tests returned not_applicable
44 tests returned pass
5 tests returned fail
   EtriggerTest > logs/20231005-220017-HiFive1-EtriggerTest.log
   IcountTest > logs/20231005-220038-HiFive1-IcountTest.log
   InstantHaltTest > logs/20231005-220051-HiFive1-InstantHaltTest.log
   Semihosting > logs/20231005-220332-HiFive1-Semihosting.log
   SemihostingFileio > logs/20231005-220339-HiFive1-SemihostingFileio.log
5 tests returned exception
   ItriggerTest > logs/20231005-220109-HiFive1-ItriggerTest.log
   MemTestReadInvalid > logs/20231005-220222-HiFive1-MemTestReadInvalid.log
   MemorySampleMixed > logs/20231005-220229-HiFive1-MemorySampleMixed.log
   MemorySampleSingle > logs/20231005-220241-HiFive1-MemorySampleSingle.log
   RepeatReadTest > logs/20231005-220325-HiFive1-RepeatReadTest.log

./gdbserver.py targets/SiFive/HiFive1-flash.py

...

:::::::::::::::::::::::::::[ ran 72 tests in 731s ]:::::::::::::::::::::::::::
18 tests returned not_applicable
36 tests returned pass
14 tests returned exception
   DebugBreakpoint > logs/20231005-222345-HiFive1Flash-DebugBreakpoint.log
   DebugExit > logs/20231005-222442-HiFive1Flash-DebugExit.log
   DebugFunctionCall > logs/20231005-222529-HiFive1Flash-DebugFunctionCall.log
   EtriggerTest > logs/20231005-222648-HiFive1Flash-EtriggerTest.log
   Hwbp1 > logs/20231005-222655-HiFive1Flash-Hwbp1.log
   Hwbp2 > logs/20231005-222742-HiFive1Flash-Hwbp2.log
   ItriggerTest > logs/20231005-222908-HiFive1Flash-ItriggerTest.log
   MemTestReadInvalid > logs/20231005-223023-HiFive1Flash-MemTestReadInvalid.log
   MemorySampleMixed > logs/20231005-223029-HiFive1Flash-MemorySampleMixed.log
   MemorySampleSingle > logs/20231005-223042-HiFive1Flash-MemorySampleSingle.log
   Registers > logs/20231005-223124-HiFive1Flash-Registers.log
   RepeatReadTest > logs/20231005-223216-HiFive1Flash-RepeatReadTest.log
   TooManyHwbp > logs/20231005-223330-HiFive1Flash-TooManyHwbp.log
   UserInterrupt > logs/20231005-223446-HiFive1Flash-UserInterrupt.log
4 tests returned fail
   IcountTest > logs/20231005-222835-HiFive1Flash-IcountTest.log
   InstantHaltTest > logs/20231005-222849-HiFive1Flash-InstantHaltTest.log
   Semihosting > logs/20231005-223224-HiFive1Flash-Semihosting.log
   SemihostingFileio > logs/20231005-223233-HiFive1Flash-SemihostingFileio.log

The tests listed in the YAML exclude file added by this commit are still being run and cause failures or exceptions.
Shouldn't they be skipped as a result of this YAML file?
Or am I misunderstanding something here?

@TommyMurphyTM1234
Copy link
Contributor

TommyMurphyTM1234 commented Oct 5, 2023

Ah - never mind - looks like I have to explicitly pass the excludes file for it to be used...

./gdbserver.py targets/SiFive/HiFive1.py --exclude-tests hifive1_excludes.yaml

...

:::::::::::::::::::::::::::[ ran 72 tests in 234s ]:::::::::::::::::::::::::::
28 tests returned not_applicable
44 tests returned pass

Sorry about the unnecessary traffic!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants