ZCU106 SFP+ example

[elmohandis]

Hi Alex,

Thank you for your wonderful work.

Quick question regarding the ZCU106 example with 10G SFP+. There you instantiate two transceivers, although the second one does not seem to be doing anything useful (static TX Data, no RX data connected..). Is this old dead code or is it some sort of workaround?

I am trying to get your example running with a similar board and I only instantiated one transceiver (with HAS_COMMON=1). I am still having issues getting CDR to lock and I was wondering why you have two transceiver instances in an example design with a single SFP+.

Thanks again!

[alexforencich]

In all of the example designs, I connect all of the transceivers that are wired to appropriate connectors, but only connect one instance of the UDP stack to the "first" one. So, I suppose from the standpoint of the example design, it's dead code, but the idea is maybe you want to use all of the ports for some reason, so might as well bring them all up. Also, previously I used GTY transceiver wizard cores with multiple channels in one core, and it's annoying to have to go regenerate that with a different number of channels, so I just generated it once with all of the channels active. Then later on, I moved to the more flexible setup that I have now.

It should work fine with a single instance set to HAS_COMMON=1. All of the ports that cross-connect to other instances can simply be left open.

What board is it that you're trying to bring up? Does the TX side of the link work (as in, if you wire it to a 10G NIC, does it say that the link is up?)

[elmohandis]

Hi Alex,

the board has nothing fancy on the SFP+. No caps/resistors or discrete components. The RX Termination level was selected based on the best sweep value and is most probably of little impact due to SFP module AC Coupling, which you have mentioned.

The major benefit came from adjusting the TX level. This relates well to our board since the traces cover a considerably longer distance in comparison with most devkits. We have FPGA -> few cm trace -> Samtec Board to Board connector -> a healthy trace of around 20cm (!) -> SFP+ Module. Layout is done properly with good SI practices. The large attenuation requires more TX level I guess.

As to why DFE was having issues im not sure. I will try to run more tests with DFE active + adjusted TX power.

[alexforencich]

Interesting. From my understanding, the TX level should not make a huge difference, as long traces will result in high frequency attenuation, which changes the shape of the waveform and generates ISI. So, adjusting the pre-cursor and post-cursor pre-emphasis settings will probably help more than just cranking up the TX power, as this will help to inject more high frequency information to compensate for the attenuation. In fact, if you crank up the TX power too much, then the pre-emphasis settings are less effective because you'll start clipping at the transmitter.

Presumably you were doing this testing with the SFP+ looped back somehow? It might make sense to play around with the RX settings using eye scan and data coming from a NIC or switch, instead of loopback, so you can adjust the RX settings in isolation from the TX side. Also, electrical or optical loopback? Optical modules can have limiting amplifiers, which can potentially help clean up the signal a bit, vs. an electrical loopback that just physically loops the TX back into the RX.

[elmohandis]

Also found this info:

The 8B/10B coding of the ethernet link does not guarantee enough randomness of the data for the DFE mode to be stable in all circumstances.
Xilinx also have a KB article that could be relevant.

I believe the length of the actual fiber itself is irrelevant for the equalization of the electrical signal between the FPGA and the SFP+ module (hmmmm).

[alexforencich]

But 10G Ethernet uses 64b/66b, not 8b/10b. So that shouldn't be an issue since it's scrambled with an LFSR.

The length of the fiber can be relevant under certain conditions - multimode fiber creates modal dispersion, and both single mode and multimode fiber create chromatic dispersion. Chromatic dispersion isn't really an issue until you start getting into the km range I think. But modal dispersion can definitely be an issue. If the modules have CDRs, then they should clean this up. Otherwise, the effects will be visible to the FPGA.

[elmohandis]

An update:

I might have jumped to conclusions about DFE. I tried the actual design mit equalization set to Auto with default insertion loss of 20dB (that should select DFE according to Xilinx docs). The link worked, however the custom txdiffctrl & gains were still active.

I then tried again without the cranked up TX power and I was not able to get a link.

Further investigation would require instantiating the IBERT core in parallel with the productive IP for more insight. I think however I will let it be with DFE + increased TX power.

Thought I would let you (and everyone reading) know.