Error checking potentially not computed correctly if CRC and data type fail #4

petertorelli · 2018-10-19T18:19:14Z

Reported:

If the seedcrc doesn’t match a known type then total_errors=-1, However if the call to check_data_types()then fails then we could end up with a count of zero and a pass.

	switch (seedcrc) { /* test known output for common seeds */
		case 0x8a02: /* seed1=0, seed2=0, seed3=0x66, size 2000 per algorithm */
			known_id=0;
			ee_printf("6k performance run parameters for coremark.\n");
			break;
		case 0x7b05: /*  seed1=0x3415, seed2=0x3415, seed3=0x66, size 2000 per algorithm */
			known_id=1;
			ee_printf("6k validation run parameters for coremark.\n");
			break;
		case 0x4eaf: /* seed1=0x8, seed2=0x8, seed3=0x8, size 400 per algorithm */
			known_id=2;
			ee_printf("Profile generation run parameters for coremark.\n");
			break;
		case 0xe9f5: /* seed1=0, seed2=0, seed3=0x66, size 666 per algorithm */
			known_id=3;
			ee_printf("2K performance run parameters for coremark.\n");
			break;
		case 0x18f2: /*  seed1=0x3415, seed2=0x3415, seed3=0x66, size 666 per algorithm */
			known_id=4;
			ee_printf("2K validation run parameters for coremark.\n");
			break;
		default:
			total_errors=-1;
			break;
	}
	if (known_id>=0) {
		for (i=0 ; i<default_num_contexts; i++) {
			results[i].err=0;
			if ((results[i].execs & ID_LIST) && 
				(results[i].crclist!=list_known_crc[known_id])) {
				ee_printf("[%u]ERROR! list crc 0x%04x - should be 0x%04x\n",i,results[i].crclist,list_known_crc[known_id]);
				results[i].err++;
			}
			if ((results[i].execs & ID_MATRIX) &&
				(results[i].crcmatrix!=matrix_known_crc[known_id])) {
				ee_printf("[%u]ERROR! matrix crc 0x%04x - should be 0x%04x\n",i,results[i].crcmatrix,matrix_known_crc[known_id]);
				results[i].err++;
			}
			if ((results[i].execs & ID_STATE) &&
				(results[i].crcstate!=state_known_crc[known_id])) {
				ee_printf("[%u]ERROR! state crc 0x%04x - should be 0x%04x\n",i,results[i].crcstate,state_known_crc[known_id]);
				results[i].err++;
			}
			total_errors+=results[i].err;
		}
	}
	total_errors+=check_data_types();

The text was updated successfully, but these errors were encountered:

Align linux64 performance tweak to freedom_metal

petertorelli · 2022-08-31T01:59:03Z

This should not be closed. But a future fix is still WIP.

Sesib · 2022-09-21T05:13:05Z

getting "listcrc" mismatch in Coremark run. What could be the potential issue?

petertorelli · 2022-09-21T17:26:53Z

You can enable CORE_DEBUG to print useful debugging messages. If you have a platform that passes w/o the crc error, generate debug logs for both systems and compare where things went wrong and debug from there.

anuani21 · 2022-09-22T11:57:24Z

Hi sir,

Am running coremark benchmark in riscv32 bit baremetal. build success. when i load the coremark.bare.riscv, am getting incorrect crc list ,crc matrix and crcstate value for 2k performance run algorithm.

Am getting correct seedcrc value, can you please help me to resolve the issue??

anuani21 · 2022-09-22T11:57:59Z

when i am running CORE_DEBUG also am getting the same issue sir.

petertorelli · 2022-09-22T14:49:49Z

Setting CORE_DEBUG won't fix the issue, but it will help you understand where the problem might be. If you look at the code, you will see that the values printed in the CORE_DEBUG statements are the same values used to generate the CRCs. If the values differ between a good run and a bad run, you need to investigate why.

anuani21 · 2022-09-23T04:59:44Z

Hi sir,

I have tried the core_debug for examine the problem, but am facing some trap issue.
Can you please help me how to fix the issue sir?
output.log

anuani21 · 2022-09-23T06:50:10Z

Sir,

Can you please share your core_debug log for my reference sir?

petertorelli · 2022-09-23T16:10:22Z

I notice your matrix size is 11x11. This means different run parameters were specified from the default: ./coremark.exe 0x3415 0x3415 0x66 0 7 1 2000 Try running "out of the box" first to see why your code is failing.

Here is a successful CORE_DEBUG=1 log: run2.log.

anuani21 · 2022-09-24T08:40:55Z

Hi,
Can you please share core debug log for run1.log sir?

anuani21 · 2022-09-24T09:45:23Z

Hi sir, Can you please share the link for 32bit baremetal coremark source in riscv wrapper?

…

On Fri, 23 Sep, 2022, 9:40 pm Peter Torelli, ***@***.***> wrote: I notice your matrix size is 11x11. This means different run parameters were specified from the default: ./coremark.exe 0x3415 0x3415 0x66 0 7 1 2000 Try running "out of the box" first to see why your code is failing. Here is a successful CORE_DEBUG=1 log: run2.log <https://github.com/eembc/coremark/files/9635312/run2.log>. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3G6FF7WSHLJF53YSZQ4GMTV7XI7RANCNFSM4F6EYE6Q> . You are receiving this because you commented.Message ID: ***@***.***>

anuani21 · 2022-09-26T09:03:51Z

Hi sir,

Here I have shared my run2.log. list initalization value is differ from your log.Can you please examine and tell me what is the issue?
validation.log

gowdanp · 2022-09-26T12:52:26Z

Hi,

I am also facing the same issue as discussed above. I have attached the log file taken on COM port by enabling CORE_DEBUG option. Can you please help me understanding this issue.

Thank you..

coremark.log

petertorelli · 2022-09-26T15:27:36Z

@anuani21 -

Here is run1.log. Regarding your second question, EEMBC does not have source code for different platforms.

petertorelli · 2022-09-26T15:46:55Z

@anuani21 & @gowdanp

The fact that idx and data are so far off is hint. This repository is for fixing CoreMark issues, not debugging user problems. It is up to you to root cause the problem and identify if there is a problem with CoreMark that we need to fix. Debugging problems like this can vary. If you look at the valid log files, you can see that the very first out put of the "Initialize list" for run1.log should be:

[0000,8080][0001,7373][0002,7a7a][0003,6161][0004,6868][0005,1717][3402,0505][3406,6161][340e,2121][3412,0505]

But yours is:

[ffff8000,8080][ffffffff,ffff][0502,0505][0512,0505][0c0d,0c0c][0c1d,0c0c][1700,1717][1705,1717][1e03,1e1e][1e13,1e1e][210e,2121][211e,2121][2809,2828][2819,2828][330c,3333][331c,3333][3a0f,3a3a][3a1f,3a3a][451a,4545][4c05,4c4c][5718,5757][5e1b,5e5e][6103,6161][6106,6161][6801,6868][6804,6868][7301,7373][7304,7373][7a02,7a7a][7a07,7a7a]

Ok, why is this? What is your next level of debug results? If you look at the code you can see that list->info->idx and list->info->data16 are initialized to 0x000 and 0x8080 inside core_list_init. After the blocks are added, the list is traversed and the first 20% of the items are assigned i++ and the remaining 80% are assigned a simple pseudo-random number.

Instead of seeing 1, 2, 3, 4... I see 0xffff8000, 0xffffffff, 0x502, 0x512... that right there is a big problem. Obviously assigning i++ isn't going to cause that problem, so it is likely a pointer problem. Is the code using the wrong user defined types for pointers? Is there an assumption about pointer sizes in CoreMark that is invalid. These are just guesses. You need to debug it and file an issue with what is broken, if that is the case.

anuani21 · 2022-09-28T04:14:24Z

Thanks, I'll check them out.

…

On Mon, Sep 26, 2022 at 9:17 PM Peter Torelli ***@***.***> wrote: @anuani21 <https://github.com/anuani21> & @gowdanp <https://github.com/gowdanp> The fact that idx and data are so far off is hint. This repository is for fixing CoreMark issues, not debugging user problems. It is up to you to root cause the problem and identify if there is a problem with CoreMark that we need to fix. Debugging problems like this can vary. If you look at the valid log files, you can see that the very first out put of the "Initialize list" for run1.log should be: [0000,8080][0001,7373][0002,7a7a][0003,6161][0004,6868][0005,1717][3402,0505][3406,6161][340e,2121][3412,0505] But yours is: [ffff8000,8080][ffffffff,ffff][0502,0505][0512,0505][0c0d,0c0c][0c1d,0c0c][1700,1717][1705,1717][1e03,1e1e][1e13,1e1e][210e,2121][211e,2121][2809,2828][2819,2828][330c,3333][331c,3333][3a0f,3a3a][3a1f,3a3a][451a,4545][4c05,4c4c][5718,5757][5e1b,5e5e][6103,6161][6106,6161][6801,6868][6804,6868][7301,7373][7304,7373][7a02,7a7a][7a07,7a7a] Ok, why is this? What is your next level of debug results? If you look at the code you can see that list->info->idx and list->info->data16 are initialized to 0x000 and 0x8080 inside core_list_init. After the blocks are added, the list is traversed and the first 20% of the items are assigned i++ and the remaining 80% are assigned a simple pseudo-random number. Instead of seeing 1, 2, 3, 4... I see 0xffff8000, 0xffffffff, 0x502, 0x512... that right there is a big problem. Obviously assigning i++ isn't going to cause that problem, so it is likely a pointer problem. Is the code using the wrong user defined types for pointers? Is there an assumption about pointer sizes in CoreMark that is invalid. These are just guesses. You need to debug it and file an issue with what is broken, if that is the case. — Reply to this email directly, view it on GitHub <#4 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/A3G6FFZQJHDHN4FSTQFG23DWAHAPVANCNFSM4F6EYE6Q> . You are receiving this because you were mentioned.Message ID: ***@***.***>

anuani21 · 2022-09-28T09:40:31Z

Sir,

I have one doubt, In coremark seed values are not getting
correctly(seed1=0, seed2=0, seed3=0x66). Am getting random seed number for
seed1 and seed 2.seed 3 value alone getting correctly as 66 all the
time,but seed 1 and 2 values differing. Is there any idea,what is the issue
sir?
seed.log

anuani21 · 2022-10-08T08:00:27Z

#4 (comment)

There are four types of errors: - Unknown seed, - CRC mismatch, - Invalid types, - Execution time < 10s. All of these errors are handled by the 'total_errors' variable. However, handling of the 'unknown seed' error differs from other types: it sets 'total_errors' to -1, while other types of errors increment this variable. This behavior leads to situations where one error overrides another. For example, if you compile the program incorrectly and get an unknown seed, and then run the program for less than 10 seconds, you will get total_errors = 0 (because one error sets it to -1, while another increments it to 0). See eembc#4. As a result, you'll receive "Correct operation validated" when you shouldn't.

petertorelli added bug Something isn't working invalid This doesn't seem right labels Oct 19, 2018

petertorelli self-assigned this Oct 19, 2018

lyellread pushed a commit to GaloisInc/coremark that referenced this issue Jan 12, 2020

Merge pull request eembc#4 from sifive/align_linux64

c181806

Align linux64 performance tweak to freedom_metal

petertorelli added futurefix Will be added to future release. and removed invalid This doesn't seem right labels Jul 2, 2021

petertorelli closed this as completed Aug 26, 2021

petertorelli reopened this Aug 31, 2022

HepoH3 mentioned this issue Jul 29, 2024

Fix error collision #61

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error checking potentially not computed correctly if CRC and data type fail #4

Error checking potentially not computed correctly if CRC and data type fail #4

petertorelli commented Oct 19, 2018

petertorelli commented Aug 31, 2022

Sesib commented Sep 21, 2022

petertorelli commented Sep 21, 2022

anuani21 commented Sep 22, 2022

anuani21 commented Sep 22, 2022

petertorelli commented Sep 22, 2022

anuani21 commented Sep 23, 2022

anuani21 commented Sep 23, 2022

petertorelli commented Sep 23, 2022

anuani21 commented Sep 24, 2022

anuani21 commented Sep 24, 2022 via email

anuani21 commented Sep 26, 2022

gowdanp commented Sep 26, 2022

petertorelli commented Sep 26, 2022

petertorelli commented Sep 26, 2022

anuani21 commented Sep 28, 2022 via email

anuani21 commented Sep 28, 2022

anuani21 commented Oct 8, 2022

Error checking potentially not computed correctly if CRC and data type fail #4

Error checking potentially not computed correctly if CRC and data type fail #4

Comments

petertorelli commented Oct 19, 2018

petertorelli commented Aug 31, 2022

Sesib commented Sep 21, 2022

petertorelli commented Sep 21, 2022

anuani21 commented Sep 22, 2022

anuani21 commented Sep 22, 2022

petertorelli commented Sep 22, 2022

anuani21 commented Sep 23, 2022

anuani21 commented Sep 23, 2022

petertorelli commented Sep 23, 2022

anuani21 commented Sep 24, 2022

anuani21 commented Sep 24, 2022 via email

anuani21 commented Sep 26, 2022

gowdanp commented Sep 26, 2022

petertorelli commented Sep 26, 2022

petertorelli commented Sep 26, 2022

anuani21 commented Sep 28, 2022 via email

anuani21 commented Sep 28, 2022

anuani21 commented Oct 8, 2022