forked from lkl/linux
-
Notifications
You must be signed in to change notification settings - Fork 19
/
lkl.txt
280 lines (203 loc) · 9.15 KB
/
lkl.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
Introduction
============
LKL (Linux Kernel Library) is aiming to allow reusing the Linux kernel code as
extensively as possible with minimal effort and reduced maintenance overhead.
Examples of how LKL can be used are: creating userspace applications (running on
Linux and other operating systems) that can read or write Linux filesystems or
can use the Linux networking stack, creating kernel drivers for other operating
systems that can read Linux filesystems, bootloaders support for reading/writing
Linux filesystems, etc.
With LKL, the kernel code is compiled into an object file that can be directly
linked by applications. The API offered by LKL is based on the Linux system call
interface.
LKL is implemented as an architecture port in arch/lkl. It uses host operations
defined by the application or a host library (tools/lkl/lib).
Supported hosts
===============
The supported hosts for now are POSIX and Windows userspace applications.
Building LKL, the host library and LKL based tools
==================================================
$ make -C tools/lkl
will build LKL as a object file, it will install it in tools/lkl/lib together
with the headers files in tools/lkl/include then will build the host library,
tests and a few of application examples:
* tests/boot - a simple applications that uses LKL and exercises the basic LKL
APIs
* fs2tar - a tool that converts a filesystem image to a tar archive
* cptofs/cpfromfs - a tool that copies files to/from a filesystem image
* lklfuse - a tool that can mount a filesystem image in userspace,
without root priviledges, using FUSE
Building LKL on FreeBSD
-----------------------
$ pkg install binutils gcc49 gnubc
#If you don't have a gcc binary:
$ ln -sf /usr/local/bin/gcc49 /usr/local/bin/gcc
#Prefer ports binutils and GNU bc(1):
$ export PATH=/sbin:/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/usr/lib64/ccache
$ gmake -C tools/lkl
Building LKL on Ubuntu
-----------------------
$ sudo apt-get install libfuse-dev libarchive-dev xfsprogs
# Optional, if you would like to be able to run tests
$ sudo apt-get install btrfs-tools
$ make -C tools/lkl
# To check that everything works:
$ cd tools/lkl
$ make test
Building LKL for Windows
------------------------
In order to build LKL for Win32 the mingw cross compiler needs to be installed
on the host (e.g. on Ubuntu the following packages are required:
binutils-mingw-w64-i686, gcc-mingw-w64-base, gcc-mingw-w64-i686
mingw-w64-common, mingw-w64-i686-dev).
Due to a bug in mingw regarding weak symbols the following patches needs to be
applied to mingw-binutils:
https://sourceware.org/ml/binutils/2015-10/msg00234.html
and i686-w64-mingw32-gas, i686-w64-mingw32-ld and i686-w64-mingw32-objcopy need
to be recompiled.
With that pre-requisites fullfilled you can now build LKL for Win32 with the
following command:
$ make CROSS_COMPILE=i686-w64-mingw32- -C tools/lkl
Building LKL on Windows
------------------------
To build on Windows, certain GNU tools need to be installed. These tools can come
from several different projects, such as cygwin, unxutils, gnu-win32 or busybox-w32.
Below is one minimal/modular set-up based on msys2.
### Common build dependencies:
* [MSYS2](https://sourceforge.net/projects/msys2/) (provides GNU bash and many other utilities)
* Extra utilities from MSYS2/pacman: bc, base-devel
### General considerations:
* No spaces in pathnames (source, prefix, destination,...)!
* Make sure that all utilities are in the PATH.
* Win64 (and MinGW 64-bit crt) is LLP64, which causes conflicts in size of "long" in the
Linux source. Linux (and lkl) can (currently) not
be built on LLP64.
* Cygwin (and msys2) are LP64, like linux.
### For MSYS2 (and Cygwin):
Msys2 will install a gcc tool chain as part of the base-devel bundle. Binutils (2.26) is already
patched for NT weak externals. Using the msys2 shell, cd to the lkl sources and run:
$ make -C tools/lkl
### For MinGW:
Install mingw-w64-i686-toolchain via pacman, mingw-w64-i686-binutils (2.26) is already patched
for NT weak externals. Start a MinGW Win32 shell (64-bit will not work, see above)
and run:
$ make -C tools/lkl
LKL hijack library
==================
LKL hijack library (liblkl-hijack.so) is used to replace system calls used by an
application on the fly so that the application can use LKL instead of the kernel
of host operating system. LD_PRELOAD is used to dynamically override system calls
with this library when you execute a program.
You can usually use this library via a wrapper script.
$ cd tools/lkl
$ ./bin/lkl-hijack.sh ip address show
There are environmental variables to configure the behavior of LKL. The followings
are the list of those variable for your environment.
* LKL_HIJACK_NET_IFTYPE
The interface type in host operating system to connect to LKL.
The following example specifies a tap interface.
```
$ LKL_HIJACK_NET_IFTYPE=tap LKL_HIJACK_NET_IFPARAMS=tap0 lkl-hijack.sh ip address show
```
* LKL_HIJACK_NET_IFPARAMS
Additional configuration parameters for the interface specified by LKL_HIJACK_NET_IFTYPE.
The parameters depend on the interface type (LKL_HIJACK_NET_IFTYPE).
```
$ LKL_HIJACK_NET_IFTYPE=tap LKL_HIJACK_NET_IFPARAMS=tap0 lkl-hijack.sh ip address show
```
* LKL_HIJACK_NET_IP
the IP address of the interface specified by LKL_HIJACK_NET_TAP.
```
$ LKL_HIJACK_NET_IP=198.51.100.5 lkl-hijack.sh ip address show
```
If you want to use DHCP for the IP address assignment, use
LKL_HIJACK_BOOT_CMDLINE with "ip=dhcp" option.
```
$ LKL_HIJACK_BOOT_CMDLINE="ip=dhcp" LKL_HIJACK_NET_IFTYPE=tap \
LKL_HIJACK_NET_IFPARAMS=tap0 lkl-hijack.sh ip address
```
* LKL_HIJACK_NET_NETMASK_LEN
the network mask length of the interface specified by LKL_HIJACK_NET_TAP.
```
$ LKL_HIJACK_NET_IP=198.51.100.5 LKL_HIJACK_NET_NETMASK_LEN=24 lkl-hijack.sh ip address show
```
* LKL_HIJACK_NET_MAC
the MAC address of the interface specified by LKL_HIJACK_NET_TAP.
```
$ LKL_HIJACK_NET_MAC="aa:bb:cc:dd:ee:ff" lkl-hijack.sh ip address show
```
* LKL_HIJACK_NET_GATEWAY
the gateway IP address of LKL network stack.
```
$ LKL_HIJACK_NET_GATEWAY=198.51.100.5 lkl-hijack.sh ip address show
```
* LKL_HIJACK_NET_MTU
the MTU size of the interface specified by LKL_HIJACK_NET_TAP.
```
$ LKL_HIJACK_NET_MTU=1280 lkl-hijack.sh ip address show
```
* LKL_HIJACK_NET_NEIGHBOR
Add a list of permanent neighbor entries in the form of "ip|mac;ip|mac;...". ipv6 are supported
```
$ LKL_HIJACK_NET_NEIGHBOR="192.168.13.100|12:34:56:78:9a:bc;2001:db8:0:f101::3|12:34:56:78:9a:be"
lkl-hijack.sh ip neighbor show
```
* LKL_HIJACK_NET_QDISC
Add a qdisc entry in the form of "root|type;root|type;...".
```
$ LKL_HIJACK_NET_QDISC="root|fq" lkl-hijack.sh tc qdisc
```
* LKL_HIJACK_DEBUG
Setting it causes some debug information (both from the kernel and the
LKL library) to be enabled.
It is also used as a bit mask to turn on specific debugging facilities.
E.g., setting it to 0x100 ("export LKL_HIJACK_DEBUG=0x100") will cause
the LKL kernel to pause after the hijack'ed app exits. This allows one
to debug or collect info from the LKL kernel before it quits.
```
$ LKL_HIJACK_DEBUG=1 lkl-hijack.sh ip address show
```
* LKL_HIJACK_SINGLE_CPU
Pin LKL kernel threads on to a single host cpu. LKL_HIJACK_SINGLE_CPU=1 pins
only LKL kernel threads while LKL_HIJACK_SINGLE_CPU=2 also pins polling
threads.
```
$ LKL_HIJACK_SINGLE_CPU=1 lkl-hijack.sh ip address show
```
* LKL_HIJACK_OFFLOAD
Work as a bit mask to enable selective device offload features. E.g.,
to enable "mergeable RX buffer" (LKL_VIRTIO_NET_F_MRG_RXBUF) +
"guest csum" (LKL_VIRTIO_NET_F_GUEST_CSUM) device features, simply set
it to 0x8002.
See virtio_net.h for a list of offload features and their bit masks.
```
$ LKL_HIJACK_OFFLOAD=0x8002 lkl-hijack.sh ./netserver -D -f
```
* LKL_HIJACK_SYSCTL
Configure sysctl values of the booted kernel via the hijack library. Multiple
entries can be specified.
```
$ LKL_HIJACK_SYSCTL="net.ipv4.tcp_wmem=4096 87380 2147483647"
./bin/lkl-hijack.sh ip address show
```
* LKL_HIJACK_BOOT_CMDLINE
Specify the command line to the kernel boot so that change the configuration
on a kernel instance. For instance, you can change the memory size with
below.
```
$ LKL_HIJACK_BOOT_CMDLINE="mem=1G" LKL_HIJACK_DEBUG=1 lkl-hijack.sh ip add
```
FAQ
===
Q: How is LKL different from UML?
A: UML prodivides a full OS environment (e.g. user/kernel separation, user
processes) and also has requirements (a filesystem, processes, etc.) that makes
it hard to use it for standalone applications. UML also relies heavily on Linux
hosts. On the other hand LKL is designed to be linked directly with the
application and hence does not have user/kernel separation which makes it easier
to use it in standalone applications.
Q: How is LKL different from LibOS?
A: LibOS re-implements high-level kernel APIs for timers, softirqs, scheduling,
sysctl, SLAB/SLUB, etc. LKL behaves like any arch port, implementing the arch
level operations requested by the Linux kernel. LKL also offers a host interface
so that support for multiple hosts can be implemented.