About the solution for deadlocks

[omeag]

Hello sir,
Have you addressed the deadlock issue in your code? If so, could you please share the approach you utilized and point out the location in the code where the solution is implemented?

[alexforencich]

Deadlock issue? What are you referring to, exactly?

[omeag]

The deadlock caused by out-of-order reads in AXI and the deadlock caused by outstanding write.

[alexforencich]

In terms of what? the crossbar?

[omeag]

yes

[alexforencich]

It's handled the same way the Xilinx crossbar handles it - each AXI ID can only be used for one "route" at a time, and the transactions are counted. This eliminates issues related to interfacing with multiple downstream devices.

Also, if AXI devices don't follow the protocol and return responses in a timely manner, that can certainly result in a hang. But the assumption is that all of the devices follow the protocol.

[omeag]

I see. Thus, may I confirm if data reordering is not supported in your crossbar?
Thank you for the explanation.

[alexforencich]

So, the one thing that's currently not supported is cycle-by-cycle read data interleaving. That does not currently work; I haven't had time to sit down and fix it properly. So you might have issues, say, interfacing with DDR on a Zynq via the crossbar. But, it's fine to reorder both read and write responses, that works correctly.

[omeag]

Got it, thank you very much!

[omeag]

And could you please provide some guidance on where in the code I could find the implementation mechanism for reordering?

[alexforencich]

The enforcement is handled here: https://github.com/alexforencich/verilog-axi/blob/master/rtl/axi_crossbar_addr.v

[omeag]

I've been studying this code, but I'm not clear on it.
Is it through thread control that allows read/write response reordering?
If there's no response received, the thread remain active, allowing arbitration and decoding for the next read/write operation. Am I understanding it correctly?

[alexforencich]

AXI ordering rules are like this: ordering must be preserved within the same ID value, but operations with different ID values can be reordered freely. So, if there is only one device on the crossbar, nothing additional needs to be done (and disabling the "thread" logic in this case to save logic resources is on my to-do list). For multiple devices, reordering can be introduced if the same ID is used to access multiple devices. So, the crossbar simply blocks this preemptively, and enforces that each ID can only access one device at a time. Each "thread" is a context for keeping track of ID/destination/transaction count. Every operation needs to "allocate" a thread, and the thread is released after all of the outstanding operations associated with the thread complete. If there are no threads available or if a thread is allocated to a different destination, then the new operation is not accepted into the crossbar until the situation is resolved. Operations with different IDs can be handled concurrently, up to the number of threads implemented.

[alexforencich]

Also, I should add that each SI is basically completely independent of the others. The ID field is extended with the SI index, so operations from different SI get different ID values when they're handed off to downstream devices, so ordering of operations from different SI is not relevant and nothing needs to be done to preserve or enforce anything. Each SI has its own copy of the "thread" logic, so even if one SI runs out of threads, the other SIs can continue to issue operations through the crossbar.

[omeag]

Thank you very much for your clarification, I appreciate it.