This repository is currently being migrated. It's locked while the migration is in progress.
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME.ef_vi
120 lines (87 loc) · 4.74 KB
/
README.ef_vi
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
ef_vi
=====
ef_vi is an API that gives applications access to the network adapter's
datapath. That is, it can be used to send and receive raw Ethernet frames
directly from user-level. The advantage of using this interface is that
it incurs very little CPU overhead, and so yields the best possible
performance.
ef_vi is a zero-copy interface. This means that the user gets direct
access to the buffers in memory that the hardware uses, avoiding a copy
that is usually incurred by higher-level interfaces. Zero copy can be a
big win for applications that:
- have very high throughput requirements, or
- are filtering received data, or
- are forwarding data.
The downside of using ef_vi is that you have to implement the upper-layer
protocols yourself, including IP, UDP, TCP etc. It is relatively
straighforward to use ef_vi for UDP, but much more difficult for TCP which
is very complex to implement from scratch. (Best to leave that to
OpenOnload!). ef_vi can be used in an application at the same time as
OpenOnload, so an application might use ef_vi for some especially
performance sensitive UDP traffic, and sockets with OpenOnload for TCP and
other traffic.
ef_vi is included in the OpenOnload distribution, and is used by the
OpenOnload user-level sockets library to access the hardware's datapath.
Overview
========
Users of ef_vi must first allocate a virtual interface (VI), encapsulated
by the type "ef_vi". A VI includes:
- A receive descriptor ring. (For receiving packets).
- A transmit descriptor ring. (For sending packets).
- An event queue.
To transmit a packet, the application writes the packet contents
(including all headers) into one or more packet buffers, and calls
ef_vi_transmit(). One or more descriptors that describe the packet are
queued in the transmit ring, and a doorbell is "rung" to tell the NIC that
the transmit ring is ready.
To receive, descriptors (each identifying a buffer) are queued in the
receive ring by calling ef_vi_receive_init() and _post(). When packets
arrive at the network adapter and are directed to a VI, they are written
into the buffers in fifo order.
The event queue is a channel from the NIC to software which notifies
software when packets arrive from the network, and when transmits complete
(so that the buffers can be freed or reused). The application retrieves
these events by calling ef_eventq_poll().
The buffers used for packet data are special: They must be pinned so that
they cannot be paged, and they must be registered for DMA using an
ef_memreg.
The NIC uses a special address space to identify locations in registered
memory, and such addresses are designated by the type "ef_addr".
Protection domains (ef_pd) provide an address space for registered memory.
All VIs allocated against a given protection domain can access any memory
registered within that protection domain.
Filters are the means by which the NIC decides where to deliver packets it
receives from the network. By default all packets are delivered to the
kernel network stack. Filters are added by the application to direct
received packets to a given VI.
You can request a protection domain be associated with an SR-IOV virtual
function (VF). This allows more memory to be registered. Latency and
packet rate are slightly worse when using VFs due to address translations
in the system IOMMU. See the Solarflare Server Adapter User's Guide for
details of how to enable SR-IOV.
It is also possible to use physical addressing. This mode gives the best
latency and allows unlimited amounts of memory to be registered. However,
an application using physical addressing can cause the network adapter to
read and write arbitrary host memory, and so breaks the system security
model.
Documentation
=============
The ef_vi documentation is generated by doxygen from the relevant
source files. By default, HTML and LaTeX documents are output. A PDF
document can then be created from the LaTeX using pdflatex.
The PDF document can also be downloaded from the Solarflare website.
To create the HTML and LaTeX documents, install doxygen, and then run:
cd src/include/etherfabric/doxygen
doxygen doxyfile_ef_vi
The documents are output into the relevant subdirectories.
To create the PDF document, install TeXLive, and then run:
cd src/include/etherfabric/doxygen/latex
make pdf
There is also a set of sample applications, which are located at:
openonload/src/tests/ef_vi
If you've any questions please contact <[email protected]>.
License
=======
ef_vi is released under the GNU Lesser General Public License -- see the
LICENSE file for details. If this license does not meet your needs,
please contact us at <[email protected]> to discuss other options.