Execute-In-Place (XIP)

[stnolting]

I have implemented a transparent Wishbone-to-SPI bridge (stnolting/wb_spi_bridge) that supports execute-in-place (XIP). This bridge allows to map a serial memory (like a flash or EEPROM) into the processor's address space. I have tested the bridge being directly connected to the processor's Wishbone port (no Wishbone interconnect required).

Obviously, data accesses have a very high latency. I am using the following configurations:

Processor:

-- External memory interface --
MEM_EXT_EN         => true,
MEM_EXT_TIMEOUT    => 8191,  -- required so we don't have a timeout
MEM_EXT_PIPE_MODE  => true,
MEM_EXT_BIG_ENDIAN => false, -- the bridge uses little-endian byte-order
MEM_EXT_ASYNC_RX   => false,

Bridge:

SPI_CLK_DIV  => 32, -- 100MHz / 32*2 = 1.5MHz SPI clock
SPI_ABYTES   => 2,  -- 16-bit address width
SPI_CPHA     => '0',
SPI_CPOL     => '0', -- SPI clock mode 0
WB_ADDR_BASE => x"F0000000",
WB_ADDR_SIZE => 64*1024 -- 64kB SPI memory

I have tested the read/write capabilities using the "bus explorer" example program (sw/example/bus_explorer).

An executable boot image for the SPI memory can be created using the makefile's bootloader target and programming the generated main.bin file to the flash.

$ make clean_all bootloader

So if your FPGA is running out of block RAM for the instruction memory, this is a very nice alternative. 😉 Enabling the processor's instruction cache hides a lot of the bridge's latency.

[stnolting]

Related to #103: I'm not sure if this is something that should be added as "default" module to the processor. Maybe it is better to keep this as an external IP block that can be used to create a custom SoC... 🤔
Any thoughts?

[jeremyherbert]

I personally would think that adding it to the default is better. The neorv32 is quite a capable processor, it's only a matter of time before people start using it with LwIP, mbedtls and other stacks; these wouldn't really fit in RAM only unless you had some external SDRAM or similar.

Do you see the overall use case of neorv32 to be more of a hidden processor that does some housekeeping, or more in the direction of the Quicklogic/Gowin chips with a hard processor IP ? Do you expect to have people updating the firmware of a device without touching the gateware/bitfiles?

[stnolting]

I personally would think that adding it to the default is better. The neorv32 is quite a capable processor, it's only a matter of time before people start using it with LwIP, mbedtls and other stacks; these wouldn't really fit in RAM only unless you had some external SDRAM or similar.

Good point. I think I will implement QSPI for the bridge first before thinking about how it could be included into the processor. There are still some odds making integration a bit complicated (address space mapping, unpredictable latency, ...).

Do you see the overall use case of neorv32 to be more of a hidden processor that does some housekeeping, or more in the direction of the Quicklogic/Gowin chips with a hard processor IP ?

When I started this project I thought about the core are some background "orchestration unit" that handles protocol stacks to interface with the actual FPGA design. Now that we have a rich set of ready-to-go peripherals and even FreeRTOS and Zephyr OS support there might be a shift in the potential use cases.

Do you expect to have people updating the firmware of a device without touching the gateware/bitfiles?

It depends on the use case. The concept of embedding application software in the bitstream (during synthesis) seemed sufficient for a programmable protocol handling machine. Now we have more potential use cases and I agree that this requires more sophisticated ways of "installing" an executable.

However, these use cases can be widely dispersed. One scenario might be OK with the latency when using SPI XIP. Another one might require a large and unified RAM to keep all the data and (maybe interchangeable) program codes. We cannot add optimized(!) support for all case, so maybe some things should be better left to the final user: defining all the parts beyond the processor's bus interface (memory controllers, custom peripherals, XIP, ...), which also means to select/implement the right boot mechanism for the specific application.

The NEORV32 is still more like an IP block - so just one part of the puzzle. Even though it moves towards being a complete and ready-to-use microcontroller. 😉

[stnolting]

I thought about this XIP concept again. There are some open "ideas" that need to be cleared before adding an XIP module to the processor. I am not sure about these issues, so any kind of feedback is highly appreciated.

• How to map the SPI flash to the CPU's address space? Should this be static or configurable by a memory-mapped register?
  • How does this fit into the processor's boot configurations?
• How to configure the XIP module in terms of clock speed and clock modes? Again, static or by a register?
• I think is would be sufficient to have a read-only XIP. If one needs to write data to the SPI flash this could also be done by the SPI peripheral.
• Single-mode SPI should be sufficient for the beginning. Yes, it is 4 times slower than QSPI, but the i-cache will (hopefully) hide some of the latency.
• ...

[umarcor]

I guess that in this specific case there might be a performance gain by having it as a module of the processor instead of a SystemTop example? If performance is the same, I would recommend not adding it inside. This is very useful for the very small FPGAs, but irrelevant for most Xilinx and Intel devices.

[stnolting]

It is not primarily a performance thing. As @jeremyherbert (and you as well) mentioned there are setups that are memory constrained. For example the iCE40up "only" has 128kB of internal memory. If you want to run larger software applications adding a SDRAM controller seems out of scope for these small FPGAs. The XIP as module would be a convenient way (*) to add external memory without a huge hardware overhead.

*) Convenience is the main point here. Of course you can add the module to a system top and connect it to the Wishbone interface and define all the memory-mapping/boot config issue according to your needs.

[jeremyherbert]

My opinions:

For FPGA applications, I think that it is ok if the memory map should be configured at gateware build time, not with a memory mapped register. I personally don't think it is something that needs to change at boot/runtime. SPI clock mode/speed I think could go either way.

XIP should be read only, yes.

Single SPI is a good start I think. QSPI could always come later.

[stnolting]

@jeremyherbert I agree. SPI clock mode and speed could be initialized with minimal configuration, i.e. clock mode 0 (seems to be supported by all SPI flash memories) and maximum clock prescaler (minimal SPI clock).

Configuring the address mapping could be done in two ways:

• leave it all to the SoC designer - it is the responsibility of the designer to create a Wishbone interconnect, add the XIP bridge and configure all the address mappings and the CPU boot mode (so it starts fetching instructions via the bus interface)
• provide a processor-internal XIP module that can be configured (address mapping) by a generic

[stnolting]

I have started implementing a first XIP module for the processors (-> #244). It only supports the standard SPI protocol yet, but QSPI is planned for the future. However, the instruction cache can hide some of the protocol's latency.

I have tested the setup on my Intel DE0-nano board - and it works quite nicely 😄