Update RV64I sraw test cases

[trdthg]

Description

• Add sraw to unsgn_rs2 group, rs2 is shift, which should be unsign
• The sraw testcases were very old with a covarage of 98.20% (109/111), this patch regenerate the cases with riscv_ctg (without -r), making covarageto 100%.

Related Issues

Ratified/Unratified Extensions

• Ratified
• Unratified

List Extensions

List the extensions that your PR affects. In case of unratified extensions, please provide a link to the spec draft that was referred to make this PR.

Reference Model Used

• SAIL
• Spike
• Other - < SPECIFY HERE >

Mandatory Checklist:

• All tests are compliant with the test-format spec present in this repo ?
• Ran the new tests on RISCOF with SAIL/Spike as reference model successfully ?
• Ran the new tests on RISCOF in coverage mode
• Link to Google-Drive folder containing the new coverage reports (See this for more info): < https://drive.google.com/drive/folders/1DzS75_cLWSi--lD02cv06JEMYv-Bba9u >

Optional Checklist:

• Were the tests hand-written/modified ?
• Have you run these on any hard DUT model ? Please specify name and provide link if possible in the description
• If you have modified arch_test.h Please provide a detailed description of the changes in the Description section above.

[allenjbaum]

This looks a bit strange; there is 1/4 the number of tests, yet you get better coverage. Can you explain that? (I'm not sure what the -r option does)

…

On Mon, Nov 18, 2024 at 6:54 PM MingZhu Yan ***@***.***> wrote: Description - Add sraw to unsgn_rs2 group - The sraw testcases were very old with a covarage of 98.20% (109/111), this patch regenerate the cases with riscv_ctg (without -r), making covarageto 100%. Related Issues Ratified/Unratified Extensions - Ratified - Unratified List Extensions List the extensions that your PR affects. In case of unratified extensions, please provide a link to the spec draft that was referred to make this PR. Reference Model Used - SAIL - Spike - Other - < SPECIFY HERE > Mandatory Checklist: - All tests are compliant with the test-format spec present in this repo ? - Ran the new tests on RISCOF with SAIL/Spike as reference model successfully ? - Ran the new tests on RISCOF in coverage mode <https://riscof.readthedocs.io/en/stable/commands.html#coverage> - Link to Google-Drive folder containing the new coverage reports (See this <https://github.com/riscv-non-isa/riscv-arch-test/blob/main/CONTRIBUTION.md#uploading-test-stats> for more info): < SPECIFY HERE > Optional Checklist: - Were the tests hand-written/modified ? - Have you run these on any hard DUT model ? Please specify name and provide link if possible in the description - If you have modified arch_test.h Please provide a detailed description of the changes in the Description section above. ------------------------------ You can view, comment on, or merge this pull request online at: #565 Commit Summary - 44dcc0c <44dcc0c> Update sraw test cases File Changes (2 files <https://github.com/riscv-non-isa/riscv-arch-test/pull/565/files> ) - *M* riscv-isac/riscv_isac/InstructionObject.py <https://github.com/riscv-non-isa/riscv-arch-test/pull/565/files#diff-e066b8aa9a510e9da214153baa4d8d23a3beeac32158435990251baf4178a922> (2) - *M* riscv-test-suite/rv64i_m/I/src/sraw-01.S <https://github.com/riscv-non-isa/riscv-arch-test/pull/565/files#diff-bbb0c123690b7eac48e0c2c8a139871c782c18fea4f34f9611ce451e2a114816> (856) Patch Links: - https://github.com/riscv-non-isa/riscv-arch-test/pull/565.patch - https://github.com/riscv-non-isa/riscv-arch-test/pull/565.diff — Reply to this email directly, view it on GitHub <#565>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJSLJFACZ2SIT66D72T2BKR5VAVCNFSM6AAAAABSBBQT72VHI2DSMVQWIX3LMV43ASLTON2WKOZSGY3TANJTHE2TGNY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[trdthg]

I thought this is strange too.

My ctg command is below:

riscv_ctg -v debug -d ./tests/ -cf ./coverage/dataset.cgf -cf ./coverage/i/rv64i.cgf -bi rv64i -p2

I checked the 35 generated tests:

• op1, op2, rd are basically gradually decreasing, from x31 to x0, covering all cases of coverage
• The first 7 and the last 2 cover all of val_comb.rfmt_base_shift

It does seem to be covered, after all, 111 is the sum of these coverage points

The last generation was 4 years ago, and I don't know exactly what happened.

---

(I'm not sure what the -r option does)

-r, --randomize Randomize Outputs.

> If randomization is enabled we use the `MinConflictsSolver` solver to find solutions.

riscv-arch-test/riscv-ctg/riscv_ctg/generator.py

Lines 493 to 496 in cd94912

	if self.random:
	problem = Problem(MinConflictsSolver())
	else:
	problem = Problem()

I haven't tested the behavior of MinConflictsSolver, and I suspect it may lead to generated tests being difficult to reproduce

After some testing, I found that the tests ctg generated each time is different. This is completely different from what I expected. I also had another person try it, and the results were also different.

[allenjbaum]

I don't know how ctg selects a seed, but it doesn't seem to be user selectable and appears to be random, as far as I know.

…

On Wed, Nov 20, 2024 at 2:33 AM MingZhu Yan ***@***.***> wrote: I thought this is strange too. My ctg command is below: riscv_ctg -v debug -d ./tests/ -cf ./coverage/dataset.cgf -cf ./coverage/i/rv64i.cgf -bi rv64i -p2 I checked the 35 generated tests: - op1, op2, rd are basically gradually decreasing, from x31 to x0, covering all cases of coverage - The first 7 and the last 2 cover all of val_comb.rfmt_base_shift It does seem to be covered, after all, 111 is the sum of these coverage points The last generation was 4 years ago, and I don't know exactly what happened. ------------------------------ (I'm not sure what the -r option does) -r, --randomize Randomize Outputs. > If randomization is enabled we use the `MinConflictsSolver` solver to find solutions. https://github.com/riscv-non-isa/riscv-arch-test/blob/cd94912fed2aab75d7d5f115b441da0813fdce8d/riscv-ctg/riscv_ctg/generator.py#L493-L496 I haven't tested the behavior of MinConflictsSolver, and I suspect it may lead to generated tests being difficult to reproduce After some testing, I found that the tests ctg generated each time is different. This is completely different from what I expected. I also had another person try it, and the results were also different. — Reply to this email directly, view it on GitHub <#565 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHPXVJXB6OVROETIBS72D5D2BRQOPAVCNFSM6AAAAABSBBQT72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIOBYGIYDGNBZGI> . You are receiving this because you commented.Message ID: ***@***.***>

[UmerShahidengr]

Multiple coverpoints are getting hit with a single testcase, which is reducing the test size to 25%, can we do it for all existing tests?

[UmerShahidengr]

It is a very optimal approach of generating the tests and completing the coverage.

[UmerShahidengr]

Approved