apps/int/vxlan-gpe not support linux kernel version 4.x

[jeromesun14]

compile log:

sunnogo@R04220:/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe$ make
make -C /lib/modules/4.4.0-22-generic/build M=/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe modules
make[1]: Entering directory '/usr/src/linux-headers-4.4.0-22-generic'
  CC [M]  /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.o
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:94:7: error: redefinition of ‘union vxlan_addr’
 union vxlan_addr {
       ^
In file included from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:44:0:
include/net/vxlan.h:119:7: note: originally defined here
 union vxlan_addr {
       ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:100:8: error: redefinition of ‘struct vxlan_rdst’
 struct vxlan_rdst {
        ^
In file included from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:44:0:
include/net/vxlan.h:125:8: note: originally defined here
 struct vxlan_rdst {
        ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:122:8: error: redefinition of ‘struct vxlan_dev’
 struct vxlan_dev {
        ^
In file included from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:44:0:
include/net/vxlan.h:152:8: note: originally defined here
 struct vxlan_dev {
        ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c: In function ‘vxlan_fdb_info’:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:423:9: error: void value not ignored as it ought to be
  return nlmsg_end(skb, nlh);
         ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c: In function ‘vxlan_udp_encap_recv’:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1255:4: error: ‘struct vxlan_sock’ has no member named ‘rcv’
  vs->rcv(vs, skb, vxh->vx_vni);
    ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c: In function ‘vxlan6_xmit_skb’:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1681:7: error: implicit declaration of function ‘vlan_tx_tag_present’ [-Werror=implicit-function-declaration]
    + (vlan_tx_tag_present(skb) ? VLAN_HLEN : 0);
       ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1708:23: warning: passing argument 1 of ‘udp_tunnel6_xmit_skb’ from incompatible pointer type [-Wincompatible-pointer-types]
  udp_tunnel6_xmit_skb(vs->sock, dst, skb, dev, saddr, daddr, prio,
                       ^
In file included from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:37:0:
include/net/udp_tunnel.h:87:5: note: expected ‘struct dst_entry *’ but argument is of type ‘struct socket *’
 int udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk,
     ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1708:33: warning: passing argument 2 of ‘udp_tunnel6_xmit_skb’ from incompatible pointer type [-Wincompatible-pointer-types]
  udp_tunnel6_xmit_skb(vs->sock, dst, skb, dev, saddr, daddr, prio,
                                 ^
In file included from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:37:0:
include/net/udp_tunnel.h:87:5: note: expected ‘struct sock *’ but argument is of type ‘struct dst_entry *’
 int udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk,
     ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1708:2: error: too few arguments to function ‘udp_tunnel6_xmit_skb’
  udp_tunnel6_xmit_skb(vs->sock, dst, skb, dev, saddr, daddr, prio,
  ^
In file included from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:37:0:
include/net/udp_tunnel.h:87:5: note: declared here
 int udp_tunnel6_xmit_skb(struct dst_entry *dst, struct sock *sk,
     ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c: In function ‘vxlan_xmit_skb’:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1787:29: warning: passing argument 1 of ‘udp_tunnel_xmit_skb’ from incompatible pointer type [-Wincompatible-pointer-types]
  return udp_tunnel_xmit_skb(vs->sock, rt, skb, src, dst, tos,
                             ^
In file included from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:37:0:
include/net/udp_tunnel.h:81:5: note: expected ‘struct rtable *’ but argument is of type ‘struct socket *’
 int udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb,
     ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1787:39: warning: passing argument 2 of ‘udp_tunnel_xmit_skb’ from incompatible pointer type [-Wincompatible-pointer-types]
  return udp_tunnel_xmit_skb(vs->sock, rt, skb, src, dst, tos,
                                       ^
In file included from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:37:0:
include/net/udp_tunnel.h:81:5: note: expected ‘struct sock *’ but argument is of type ‘struct rtable *’
 int udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb,
     ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1787:9: error: too few arguments to function ‘udp_tunnel_xmit_skb’
  return udp_tunnel_xmit_skb(vs->sock, rt, skb, src, dst, tos,
         ^
In file included from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:37:0:
include/net/udp_tunnel.h:81:5: note: declared here
 int udp_tunnel_xmit_skb(struct rtable *rt, struct sock *sk, struct sk_buff *skb,
     ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c: In function ‘vxlan_xmit_one’:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1941:34: warning: passing argument 1 of ‘ipv6_stub->ipv6_dst_lookup’ from incompatible pointer type [-Wincompatible-pointer-types]
   if (ipv6_stub->ipv6_dst_lookup(sk, &ndst, &fl6)) {
                                  ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1941:34: note: expected ‘struct net *’ but argument is of type ‘struct sock *’
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1941:38: warning: passing argument 2 of ‘ipv6_stub->ipv6_dst_lookup’ from incompatible pointer type [-Wincompatible-pointer-types]
   if (ipv6_stub->ipv6_dst_lookup(sk, &ndst, &fl6)) {
                                      ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1941:38: note: expected ‘struct sock *’ but argument is of type ‘struct dst_entry **’
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1941:45: warning: passing argument 3 of ‘ipv6_stub->ipv6_dst_lookup’ from incompatible pointer type [-Wincompatible-pointer-types]
   if (ipv6_stub->ipv6_dst_lookup(sk, &ndst, &fl6)) {
                                             ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1941:45: note: expected ‘struct dst_entry **’ but argument is of type ‘struct flowi6 *’
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1941:7: error: too few arguments to function ‘ipv6_stub->ipv6_dst_lookup’
   if (ipv6_stub->ipv6_dst_lookup(sk, &ndst, &fl6)) {
       ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c: At top level:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:2468:12: error: unknown type name ‘vxlan_rcv_t’
            vxlan_rcv_t *rcv, void *data,
            ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:2520:7: error: unknown type name ‘vxlan_rcv_t’
       vxlan_rcv_t *rcv, void *data,
       ^
In file included from include/linux/linkage.h:6:0,
                 from include/linux/kernel.h:6,
                 from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:14:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:2546:19: error: ‘vxlan_sock_add’ undeclared here (not in a function)
 EXPORT_SYMBOL_GPL(vxlan_sock_add);
                   ^
include/linux/export.h:57:16: note: in definition of macro ‘__EXPORT_SYMBOL’
  extern typeof(sym) sym;     \
                ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:2546:1: note: in expansion of macro ‘EXPORT_SYMBOL_GPL’
 EXPORT_SYMBOL_GPL(vxlan_sock_add);
 ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c: In function ‘vxlan_sock_work’:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:2557:8: error: called object ‘vxlan_sock_add’ is not a function or function pointer
  nvs = vxlan_sock_add(net, port, vxlan_rcv, NULL, false, vxlan->flags);
        ^
In file included from include/linux/linkage.h:6:0,
                 from include/linux/kernel.h:6,
                 from /home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:14:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:2546:19: note: declared here
 EXPORT_SYMBOL_GPL(vxlan_sock_add);
                   ^
include/linux/export.h:57:21: note: in definition of macro ‘__EXPORT_SYMBOL’
  extern typeof(sym) sym;     \
                     ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:2546:1: note: in expansion of macro ‘EXPORT_SYMBOL_GPL’
 EXPORT_SYMBOL_GPL(vxlan_sock_add);
 ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c: At top level:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:604:25: warning: ‘vxlan_gro_receive’ defined but not used [-Wunused-function]
 static struct sk_buff **vxlan_gro_receive(struct sk_buff **head, struct sk_buff *skb)
                         ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:669:12: warning: ‘vxlan_gro_complete’ defined but not used [-Wunused-function]
 static int vxlan_gro_complete(struct sk_buff *skb, int nhoff)
            ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:692:13: warning: ‘vxlan_notify_add_rx_port’ defined but not used [-Wunused-function]
 static void vxlan_notify_add_rx_port(struct vxlan_sock *vs)
             ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1219:12: warning: ‘vxlan_udp_encap_recv’ defined but not used [-Wunused-function]
 static int vxlan_udp_encap_recv(struct sock *sk, struct sk_buff *skb)
            ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:2427:13: warning: ‘vxlan_del_work’ defined but not used [-Wunused-function]
 static void vxlan_del_work(struct work_struct *work)
             ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:2434:23: warning: ‘vxlan_create_sock’ defined but not used [-Wunused-function]
 static struct socket *vxlan_create_sock(struct net *net, bool ipv6,
                       ^
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c: In function ‘vxlan_xmit_skb’:
/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.c:1789:1: warning: control reaches end of non-void function [-Wreturn-type]
 }
 ^
cc1: some warnings being treated as errors
scripts/Makefile.build:264: recipe for target '/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.o' failed
make[2]: *** [/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe/vxlan.o] Error 1
Makefile:1396: recipe for target '_module_/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe' failed
make[1]: *** [_module_/home/sunyongfeng/workshop/p4/p4factory/apps/int/vxlan-gpe] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-4.4.0-22-generic'
Makefile:4: recipe for target 'all' failed
make: *** [all] Error 2

[jeromesun14]

I found that vxlan codes changed a lot during Linux kernel 4.x

[jeromesun14]

Steps to compile vxlan kernel module with ubuntu 16.04: (e.g. degrade kernel 4.4.x to 3.19.8)

1. Download Kernel/MainlineBuilds with kernel 3.19.8

• headers, 32-bit, 64-bit
• images, 32-bit, 64-bit

2. install module-init-tools, sudo apt-get install module-init-tools
3. install linux-headers and linux-image, sudo dpkg -i linux-headers-3.19.8-031908_3.19.8-031908.201505110938_all.deb, sudo dpkg -i linux-image-3.19.8-031908-generic_3.19.8-031908.201505110938_amd64.deb
4. modify /usr/src/linux-headers-3.19.8-031908-generic/include/generated/utsrelease.h macro #define UTS_UBUNTU_RELEASE_ABI 031908 to #define UTS_UBUNTU_RELEASE_ABI 31908, as dkms use the macro as octal data.
5. remove old docker images, as we don't have aufs module for 3.19.8 now. sudo mv /var/lib/docker /var/lib/docker.old
6. reboot, and select grub menu "Advanced options for Ubuntu", then select "Ubuntu, with Linux 3.19.8-031908-generic"
7. check kernel has been degraded to 3.19.8, uname -a
8. compile and run apps/int/vxlan-gpe

You may have a look at my blog: startup_with_old_kernel and docker_failed_after_update_kernel.

After degrade linux kernel, I can now compile run vxlan.ko but I encountered into some other issues, and still worked hard to run apps/int.