Why Does DMEM not have a Global Reset?

[DS-567]

Hi Stephan,

Sorry for bothering you as I'm sure you're busy 😅.

Is there a reason why the DMEM module doesn't have a global reset?

I am running multiple individual fault injection experiments in a single batch and rely on the global reset input signal to reset the Neorv32 core back to a default state so each fault injection experiment doesn't influence the next one. I have noticed that unless I re-cycle power to the FPGA between experiments, I can see different datasets from Neorv32. I am just thinking that as I inject faults during the boot code (e.g. <500 cycles), the stack etc in DMEM might not be setup correctly and the core won't function correctly anymore, so without powering down or resetting DMEM, this could be the culprit of my problem I ma experiencing!

Also are there any other modules that store state between global reset inside the Neorv32 RTL?

[stnolting]

Hey @DS-567!

Is there a reason why the DMEM module doesn't have a global reset?

The processor's instruction and data memories (as well as all other memories like caches) were designed for optimal FPGA utilization. Hence, they are described in a way that the synthesizer can map them to FPGA blockRAM. Unfortunately, blockRAM does not provide a full hardware reset (by the way, no "main memory" provides a real reset - think of the computer's RAM chips or even the HDD).

The blockRAM of a FPGA is highly optimized in terms of their elementary flip flops. Adding a dedicated reset to each flip flop would require additional chip area (and would increase power consumption). I guess this might be the reason why absolutely no FPGA vendor has implemented something like this (so far?).

Also are there any other modules that store state between global reset inside the Neorv32 RTL?

Very good question!

All memories of the NEORV32 that implement "large" and "flat"/RAM-like read/write data storage are described in a way so they can be mapped to blockRAM in order to optimize FPGA utilization, timing and energy consumption. Hence, these memories do not have a reset capability.

Right now, the following modules are mapped to blockRAM and do not have a dedicated reset:

• IMEM and DMEM
• the caches's data storage (but not the tag and flag storage)
• the CPU's register file

[DS-567]

The processor's instruction and data memories (as well as all other memories like caches) were designed for optimal FPGA utilization. Hence, they are described in a way that the synthesizer can map them to FPGA blockRAM. Unfortunately, blockRAM does not provide a full hardware reset (by the way, no "main memory" provides a real reset - think of the computer's RAM chips or even the HDD).

The blockRAM of a FPGA is highly optimized in terms of their elementary flip flops. Adding a dedicated reset to each flip flop would require additional chip area (and would increase power consumption). I guess this might be the reason why absolutely no FPGA vendor has implemented something like this (so far?).

Ah that makes sense, ok.

Right now, the following modules are mapped to blockRAM and do not have a dedicated reset:

IMEM and DMEM
the caches's data storage (but not the tag and flag storage)
the CPU's register file

Alright, so these modules will store any data even after the core has been reset. I am trying to understand what happens between fault injection runs as I naively assumed this would reset the entire Neorv32 logic.

the caches's data storage (but not the tag and flag storage)

Is this iCache and dCahce? I have not included the generics in my top level Neorv32 instantiation so I think I have these disabled? So is it correct to say I can ignore these modules with regards to storing data?

the CPU's register file

Ok I have looked at the following boot code for my Neorv32 applications and included it below. It appears that both the sp and gp registers are initialised to zero in <__crt0_pointer_init>. And the trap setup CSR's mstatus, mie and mtval are also set to zero. I notice that the ra register and registers a0 to a5 are not reset. Why do they not get reset, especially the ra register (or is this done via the AUIPC instruction instead?)?

main.elf:     file format elf32-littleriscv


Disassembly of section .text:

00000000 <__crt0_entry>:
   0:	30005073          	csrwi	mstatus,0

00000004 <__crt0_cpu_csr_init>:
   4:	30401073          	csrw	mie,zero
   8:	00000097          	auipc	ra,0x0
   c:	13408093          	addi	ra,ra,308 # 13c <__crt0_trap_handler>
  10:	30509073          	csrw	mtvec,ra

00000014 <__crt0_pointer_init>:
  14:	80002117          	auipc	sp,0x80002
  18:	fe810113          	addi	sp,sp,-24 # 80001ffc <__crt0_stack_begin+0x0>
  1c:	80001197          	auipc	gp,0x80001
  20:	84818193          	addi	gp,gp,-1976 # 80000864 <__crt0_stack_begin+0xffffe868>

00000024 <__crt0_reg_file_init>:
  24:	00000213          	li	tp,0
  28:	00000293          	li	t0,0
  2c:	00000313          	li	t1,0
  30:	00000393          	li	t2,0
  34:	00000413          	li	s0,0
  38:	00000493          	li	s1,0
  3c:	00000813          	li	a6,0
  40:	00000893          	li	a7,0
  44:	00000913          	li	s2,0
  48:	00000993          	li	s3,0
  4c:	00000a13          	li	s4,0
  50:	00000a93          	li	s5,0
  54:	00000b13          	li	s6,0
  58:	00000b93          	li	s7,0
  5c:	00000c13          	li	s8,0
  60:	00000c93          	li	s9,0
  64:	00000d13          	li	s10,0
  68:	00000d93          	li	s11,0
  6c:	00000e13          	li	t3,0
  70:	00000e93          	li	t4,0
  74:	00000f13          	li	t5,0
  78:	00000f93          	li	t6,0

0000007c <__crt0_copy_data>:
  7c:	1f800593          	li	a1,504
  80:	80000617          	auipc	a2,0x80000
  84:	f8060613          	addi	a2,a2,-128 # 80000000 <__crt0_stack_begin+0xffffe004>
  88:	80000697          	auipc	a3,0x80000
  8c:	fdc68693          	addi	a3,a3,-36 # 80000064 <__crt0_stack_begin+0xffffe068>
  90:	00c58e63          	beq	a1,a2,ac <__crt0_clear_bss>

00000094 <__crt0_copy_data_loop>:
  94:	00d65c63          	bge	a2,a3,ac <__crt0_clear_bss>
  98:	0005a703          	lw	a4,0(a1)
  9c:	00e62023          	sw	a4,0(a2)
  a0:	00458593          	addi	a1,a1,4
  a4:	00460613          	addi	a2,a2,4
  a8:	fedff06f          	j	94 <__crt0_copy_data_loop>

000000ac <__crt0_clear_bss>:
  ac:	80000717          	auipc	a4,0x80000
  b0:	fb870713          	addi	a4,a4,-72 # 80000064 <__crt0_stack_begin+0xffffe068>
  b4:	80000797          	auipc	a5,0x80000
  b8:	fb078793          	addi	a5,a5,-80 # 80000064 <__crt0_stack_begin+0xffffe068>

000000bc <__crt0_clear_bss_loop>:
  bc:	00f75863          	bge	a4,a5,cc <__crt0_call_constructors>
  c0:	00072023          	sw	zero,0(a4)
  c4:	00470713          	addi	a4,a4,4
  c8:	ff5ff06f          	j	bc <__crt0_clear_bss_loop>

000000cc <__crt0_call_constructors>:
  cc:	00000417          	auipc	s0,0x0
  d0:	12c40413          	addi	s0,s0,300 # 1f8 <__etext>
  d4:	00000497          	auipc	s1,0x0
  d8:	12448493          	addi	s1,s1,292 # 1f8 <__etext>

000000dc <__crt0_call_constructors_loop>:
  dc:	00945a63          	bge	s0,s1,f0 <__crt0_call_constructors_loop_end>
  e0:	00042083          	lw	ra,0(s0)
  e4:	000080e7          	jalr	ra
  e8:	00440413          	addi	s0,s0,4
  ec:	ff1ff06f          	j	dc <__crt0_call_constructors_loop>

000000f0 <__crt0_call_constructors_loop_end>:
  f0:	00000513          	li	a0,0
  f4:	00000593          	li	a1,0
  f8:	090000ef          	jal	ra,188 <main>

000000fc <__crt0_main_exit>:
  fc:	30401073          	csrw	mie,zero
 100:	34051073          	csrw	mscratch,a0

00000104 <__crt0_call_destructors>:
 104:	00000417          	auipc	s0,0x0
 108:	0f440413          	addi	s0,s0,244 # 1f8 <__etext>
 10c:	00000497          	auipc	s1,0x0
 110:	0ec48493          	addi	s1,s1,236 # 1f8 <__etext>

00000114 <__crt0_call_destructors_loop>:
 114:	00945a63          	bge	s0,s1,128 <__crt0_call_destructors_loop_end>
 118:	00042083          	lw	ra,0(s0)
 11c:	000080e7          	jalr	ra
 120:	00440413          	addi	s0,s0,4
 124:	ff1ff06f          	j	114 <__crt0_call_destructors_loop>

00000128 <__crt0_call_destructors_loop_end>:
 128:	00000093          	li	ra,0
 12c:	00008463          	beqz	ra,134 <__crt0_main_aftermath_end>
 130:	000080e7          	jalr	ra

00000134 <__crt0_main_aftermath_end>:
 134:	10500073          	wfi
 138:	ffdff06f          	j	134 <__crt0_main_aftermath_end>

0000013c <__crt0_trap_handler>:
 13c:	ff810113          	addi	sp,sp,-8
 140:	00812023          	sw	s0,0(sp)
 144:	00912223          	sw	s1,4(sp)
 148:	34202473          	csrr	s0,mcause
 14c:	02044663          	bltz	s0,178 <__crt0_trap_handler_end>
 150:	34102473          	csrr	s0,mepc
 154:	00041483          	lh	s1,0(s0)
 158:	0034f493          	andi	s1,s1,3
 15c:	00240413          	addi	s0,s0,2
 160:	34141073          	csrw	mepc,s0
 164:	00300413          	li	s0,3
 168:	00941863          	bne	s0,s1,178 <__crt0_trap_handler_end>
 16c:	34102473          	csrr	s0,mepc
 170:	00240413          	addi	s0,s0,2
 174:	34141073          	csrw	mepc,s0

00000178 <__crt0_trap_handler_end>:
 178:	00012403          	lw	s0,0(sp)
 17c:	00412483          	lw	s1,4(sp)
 180:	00810113          	addi	sp,sp,8
 184:	30200073          	mret

00000188 <main>:

Also the trap handling CSR's (mcause, mepc, mtval) are not blockRAM so they contain a soft reset?

And all other parts of Neorv32 are then reset upon the top level reset signal going low?

[stnolting]

Alright, so these modules will store any data even after the core has been reset.

That's right.

Just a note: when you (re-)upload the bitstream all the block RAMs (and so IMEM + DMEM as well) will get initialized with all zeros - at least that is default config for most FPGA toolchains.

Is this iCache and dCahce? I have not included the generics in my top level Neorv32 instantiation so I think I have these disabled? So is it correct to say I can ignore these modules with regards to storing data?

Yeah, these are the caches. But you can ignore them if they are not explicitly enabled via the according generics.

Ok I have looked at the following boot code for my Neorv32 applications and included it below. It appears that both the sp and gp registers are initialised to zero in <__crt0_pointer_init>.

Actually, sp (= the stack pointer) is set to the end of the DMEM (= 0x80001ffc here) as the stack grows downwards:

  14:	80002117          	auipc	sp,0x80002
  18:	fe810113          	addi	sp,sp,-24 # 80001ffc <__crt0_stack_begin+0x0>

Accordingly, gp (= global pointer) is set to the ~middle of the DMEM, but actually most software does not use that register at all:

  1c:	80001197          	auipc	gp,0x80001
  20:	84818193          	addi	gp,gp,-1976 # 80000864 <__crt0_stack_begin+0xffffe868>

And the trap setup CSR's mstatus, mie and mtval are also set to zero.

mie and mstatus are set to zero to disable interrupts globally. mtval is read-only so you do not need to worry about that.

I notice that the ra register and registers a0 to a5 are not reset. Why do they not get reset, especially the ra register (or is this done via the AUIPC instruction instead?)?

a0 to a5 do not have any explicit code to set them to zero. Instead, they "initialized" by some other parts of crt0 (in terms of setting them to a known value), for example a1:

  7c:	1f800593          	li	a1,504

ra is also not explicitly initialized. However, it will be written when main is called:

  f8:	090000ef          	jal	ra,188 <main>

Also the trap handling CSR's (mcause, mepc, mtval) are not blockRAM so they contain a soft reset?

Correct. All CSRs provide a dedicated hardware reset.

And all other parts of Neorv32 are then reset upon the top level reset signal going low?

Are we talking about the CPU core or the entire processor?

The CPU provides a full hardware reset (except for the register file).

All remaining parts of the processor (interconnect, IO, peripherals) also provide a hardware reset. Besides the memories, there are some registers that might survive a hardware reset, but that should not impact a clean reboot as it is the software's task to (re-)initialize all used peripherals.

[DS-567]

Just a note: when you (re-)upload the bitstream all the block RAMs (and so IMEM + DMEM as well) will get initialized with all zeros - at least that is default config for most FPGA toolchains.

DMEM will be initialised with all zeros but IMEM will be initialized to the application image won't it?

Yeah, these are the caches. But you can ignore them if they are not explicitly enabled via the according generics.

Perfect 👌.

Actually, sp (= the stack pointer) is set to the end of the DMEM (= 0x80001ffc here) as the stack grows downwards:
Accordingly, gp (= global pointer) is set to the ~middle of the DMEM, but actually most software does not use that register at all:

My bad, yeah they are initialised to some known value anyway.

So the register file is not an issue as it will be 'reset' by the boot up code after a soft reset of Neorv32.

Are we talking about the CPU core or the entire processor?

The CPU provides a full hardware reset (except for the register file).

All remaining parts of the processor (interconnect, IO, peripherals) also provide a hardware reset. Besides the memories, there are some registers that might survive a hardware reset, but that should not impact a clean reboot as it is the software's task to (re-)initialize all used peripherals.

I think you have answered all my concerns here, a soft reset and a normal boot up sequence should leave a clean slate for the next fault injection run. Even for DMEM, it is safe to say it is going to be written to before being read from anyway 👍. Thanks Stephan!

[stnolting]

DMEM will be initialised with all zeros but IMEM will be initialized to the application image won't it?

Right, IMEM is pre-initialized with the application code if the bootloader is disabled ("direct boot").

From a HDL point of view, the DMEM is not initialized at all, but as mentioned earlier most synthesis tools will initialize "uninitialized" blockRAMs with all zero.

Even for DMEM, it is safe to say it is going to be written to before being read from anyway

Right.

Thanks Stephan!

You're welcome!

[tmeissner]

From a HDL point of view, the DMEM is not initialized at all, but as mentioned earlier most synthesis tools will initialize "uninitialized" blockRAMs with all zero.

I would not rely on synthesis tools initialize BRAMs with zero as this is very technology dependent. There are even FPGA-architectures which BRAMs can't be initialized with user data at all by a FW-file. For these FPGAs (mostly flash based) you have to load the BRAM data from other memories in or outside the FPGA.

[stnolting]

I would not rely on synthesis tools initialize BRAMs with zero as this is very technology dependent

Full ACK!

There are even FPGA-architectures which BRAMs can't be initialized with user data at all by a FW-file.

Do you have an example? I thought that all "classical" BRAMs can always be initialized by the bitstream (at least at Intel & AMD). 😅 But of course there are also a lot of other memory primitives that might not provide such a feature (e.g. the ice40up's SPRAM).

[tmeissner]

Do you have an example? I thought that all "classical" BRAMs can always be initialized by the bitstream (at least at Intel & AMD).

"Classical" in form of SRAM-based FPGA-architectures. All Flash-based FPGA-architectures I know - mostly from Microchip, (Smart)Fusion(2) or Polarfire for example - don't allow to program the BRAMs with the bit stream. That's a limitation which hits us not rarely 🥲

[stnolting]

Good point. I was only thinking about my plain all-day SRAM-based ones.
I think Intel's MAX10 FPGAs have similar restrictions. Even though they are SRAM-based the bitstream is stored in an on-chip flash, which does not seem to have the capacity to also store the RAM image 🤔