Easing libremesh virtualization

README.md

@@ -36,6 +36,10 @@ To run the tests simply execute `./run_tests`.
 
 Please read the [[Unit Testing Guide](TESTING.md)] for more details about testing and how to add tests to LibreMesh.
 
+## Virtualization
+LibreMesh allows you to test it without having to deal with routers since they can be lifted libremesh instances on virtual machines using QEMU.
+Please read [VIRTUALIZING.md][14] for more details on node virtualization.
+
 ## Get in Touch with LibreMesh Community
 
 ### Mailing Lists
@@ -83,3 +87,4 @@ Support this project by becoming a sponsor. Your logo will show up here with a l
 [10]: https://github.com/libremesh/network-profiles/
 [12]: https://opencollective.com/libremesh
 [13]: https://libremesh.org/development.html
+[14]: https://github.com/irina556/lime-packages/blob/easing-libremesh-virtualization/VIRTUALIZING.md

VIRTUALIZING.md

@@ -0,0 +1,131 @@
+# Run LibreMesh in a virtual machine
+In this repository, you will find scripts that allow you to raise
+libremesh instances on virtual machines using QEMU.
+This way you can test libremesh without the need to count
+with routers or to flash the libremesh firmware on them, which greatly speeds up the libremesh development process.
+
+## What are the limitations of virtualization?
+This image is not a perfect firmware image, it does not have wifi for example but ethernet network
+LAN and WAN is supported. All the files inside the packages `files/` can be copied into the rootfs,
+overwriting a precooked image that is a full LibreMesh x86_64 image.
+So don't expect that everything runs exactly as in a wireless router but most things will perform
+as expected:
+
+* initialization scripts: uci-defaults, init.d, etc
+* lime-config
+* ubus / rpcd
+* lime-app
+
+ICMPv4 does NOT work between qemu nodes, so ping (v4) will not work as expected. Everything else (including ICMPv6 i.e. ping6) does work as expected, however.
+
+
+## What do you need to run libremesh in a virtual?
+In order to run libremesh in QEMU you need two files:
+- generic-rootfs.tar.gz
+- ramfs.bzImage
+Which can be generated by compiling libremesh from the `buildroot`
+selecting x86 as the target and the option to generate ramgs.bzImage.
+
+LibreRouterOS, a LibreMesh flavor maintained by the LibreRouter team, distributes these images in each of its releases.
+You can find them at:
+https://gitlab.com/librerouter/librerouteros/-/releases
+Example:
+https://repo.librerouter.org/lros/releases/1.5/targets/x86/64/
+
+## How to virtualize a libremesh node?
+
+1. Clone the lime-packages repository
+```
+git clone [email protected]:libremesh/lime-packages.git
+```
+
+2. Install the following packages
+```
+sudo apt install dnsmasq iptables qemu-system-x86
+```
+
+3. Run the script ./tools/qemu_dev_start
+
+```
+cd lime-packages
+sudo ./tools/qemu_dev_start /path/to/rootfs.tar.gz /path/to/ramfs.bzImage
+```
+
+4. To stop the virtual and clean the environment created on the host for its operation:
+```
+sudo ./tools/qemu_dev_stop
+```
+
+In addition, up to 100 qemu nodes can be configured. This can be done using the --node-id N parameter. In this example, the node id is 1 and the host network interface is wlo1:
+```
+sudo ./tools/qemu_dev_start --node-id 1 --enable-wan wlo1 /path/to/rootfs.tar.gz /path/to/ramfs.bzImage
+```
+
+All LAN interfaces of the node are tied together. You can use --enable-wan on just one of the nodes to share its internet connection with the network
+
+## How do I go about giving it access to the internet?
+
+If you want to give the node access to the internet, qemu_dev_start argues “enable-wan” to which you must pass the name of the network interface of your host through which you have an internet connection as a parameter.
+In this example the wifi interface name is wlo1. The command is used as follows:
+
+```
+sudo ./tools/qemu-dev-start --enable-wan wlo1 path/to/rootfs.tar.gz /path/to/ramfs.bzImage
+```
+
+## How do I stomp the image with my libremesh working directory?
+
+If you've been working on libremesh mods and want to test them on a virtual node, you can use the "libremesh-workdir" argument, and step into the root file system without the need to rebuild the images. The command is used as shown next:
+
+```
+sudo ./tools/qemu-dev-start --libremesh-workdir . /path/to/rootfs.tar.gz /path/to/ramfs.bzImage
+```
+
+## How do I virtualize a multi-node libremesh network?
+
+1. Install ansible
+```
+sudo apt install ansible
+```
+or by following the official ansible [installation guide](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html).
+
+2. Copy the `rootfs.tar.gz` and `ramfs.bzImage` images to ./tools/ansible/files
+```
+cd lime-packages
+cp /path/to/rootfs.tar.gz ./tools/ansible/files/generic-rootfs.tar.gz
+cp /path/to/ramfs.bzImage ./tools/ansible/files/ramfs.bzImage
+```
+
+3. Run the playbook qemu_dev_start.yml
+```
+cd lime-packages/tools/ansible
+sudo ansible-playbook qemu_cloud_start.yml
+```
+
+4. To stop the virtual ones and clean the environment created on the host for its operation:
+```
+sudo ansible-playbook qemu_cloud_stop.yml
+```
+
+By default, the network topology is made up of 12 nodes distributed in four different batman/L2 clouds (A, B, C, D) and interconnected between them.
+To modify this topology, you can edit the `./tools/ansible/hosts.yml` file.
+
+You will then be able to access each of the cloud nodes via clusterssh as shown in the playbook output.
+
+## How do I specify in which node of the mesh the internet connection would be?
+
+Edit the `hosts.yml` file and add the `enable_wan` variable with the value
+of the network interface of your host through which you have an internet connection.
+Example:
+
+```
+all:
+  children:
+    cloudA:
+      vars:
+        eth0: lm_cloudA
+        cloud: A
+      hosts:
+        hostA.cloudA.test:
+            enable_wan: wlo1
+(...)
+```

tools/ansible/cloud_and_node_packages.yml

@@ -0,0 +1,27 @@
+- name: Install packages
+  hosts: localhost
+  become: yes
+  connection: local
+  gather_facts: no
+  tasks:
+   - name: Install bridge_utils
+     apt:
+      name: bridge-utils
+      state: present
+   - name: Install clusterssh
+     apt:
+      name: clusterssh
+      state: present
+   - name: Install dnsmasq
+     apt: 
+      name: dnsmasq
+      state: present
+   - name: Install iptables
+     apt:
+      name: iptables
+      state: present
+   - name: Install qemusystem
+     apt:
+      name: qemu-system-x86
+      state: present
+

tools/ansible/files/.placeholder

@@ -0,0 +1 @@
+

tools/ansible/modules/linux_bridge.py

@@ -54,17 +54,11 @@ def __init__ (self, module) :
         self.module = module
         self.bridge = module.params['bridge']
         self.state = module.params['state']
-
         return
 
-    def brctl (self, cmd) :
-
-        return self.module.run_command (['brctl'] + cmd)
-
-
-    def ifconfig (self, cmd) :
+    def ip(self, cmd) : 
 
-        return self.module.run_command (['ifconfig'] + cmd)
+        return self.module.run_command (['ip'] + cmd) 
 
 
     def br_exists (self) :
@@ -79,22 +73,22 @@ def br_exists (self) :
 
 
     def addbr (self) :
-        
-        (rc, out, err) = self.brctl (['addbr', self.bridge])
+
+        (rc, out, err) = self.ip (['link', 'add', 'name', self.bridge, 'type', 'bridge'])
 
         if rc != 0 :
             raise Exception (err)
 
-        self.ifconfig ([self.bridge, 'up'])
+        self.ip(['link','set','up', self.bridge]) 
 
         return
 
 
     def delbr (self) :
 
-        self.ifconfig ([self.bridge, 'down'])
-
-        (rc, out, err) = self.brctl (['delbr', self.bridge])
+        self.ip(['link','set', 'down', self.bridge]) 
+        
+        (rc, out, err) = self.ip (['link', 'del', self.bridge])
 
         if rc != 0 :
             raise Exception (err)

tools/ansible/modules/linux_bridge_port.py

@@ -60,10 +60,9 @@ def __init__ (self, module) :
 
         return
 
+    def ip(self, cmd) :
 
-    def brctl (self, cmd) :
-
-        return self.module.run_command (['brctl'] + cmd)
+        return self.module.run_command (['ip'] + cmd)
 
 
     def port_exists (self) :
@@ -79,7 +78,7 @@ def port_exists (self) :
 
     def addif (self) :
 
-        (rc, out, err) = self.brctl (['addif', self.bridge, self.port])
+        (rc, out, err) = self.ip (['link', 'set', self.port,'master',self.bridge])
 
         if rc != 0 :
             raise Exception (err)
@@ -89,7 +88,7 @@ def addif (self) :
 
     def delif (self) :
 
-        (rc, out, err) = self.brctl (['delif', self.bridge, self.port])
+        (rc, out, err) = self.ip (['link', 'del', self.port,'dev',self.bridge])
 
         if rc != 0 :
             raise Exception (err)

tools/ansible/qemu_cloud_start.yml

@@ -4,8 +4,9 @@
 #  you can use --limit regexp to start just two clouds, for example:
 #  sudo ansible-playbook qemu_cloud_start.yml -l *cloud[AB]*
 
-- name: create bridges for qemu nodes
+- import_playbook: cloud_and_node_packages.yml
 
+- name: create bridges for qemu nodes
   hosts: all
   connection: local
   gather_facts: no
@@ -34,7 +35,7 @@
     ramfs_url: http://repo.libremesh.org/tmp/openwrt-18.06-x86-64-ramfs.bzImage
     rootfs: files/generic-rootfs.tar.gz
     ramfs: files/ramfs.bzImage
-
+    enable_wan_param: "{{ ('--enable-wan ' + enable_wan ) if enable_wan is defined else '' }}"
   tasks:
     - name: download rootfs if not found locally
       tags: download
@@ -78,11 +79,11 @@
         src: lime-node
 
     - name: run qemu
-      shell: (../qemu_dev_start --node-id {{ node_id }} --eth0 {{ lm_ifname }}_0 --eth2 {{ lm_ifname }}_2 {{ rootfs }} {{ ramfs }} &)
+      shell: (../qemu_dev_start --node-id {{ node_id }} --eth0 {{ lm_ifname }}_0 --eth2 {{ lm_ifname }}_2 {{ enable_wan_param }} {{ rootfs }} {{ ramfs }} &)
 
     - name: set_fact
       set_fact:
-        linklocal: "fe80::5000:ff:feab:c0{{ node_id }}%lm_cloud{{cloud}}"
+        linklocal: "root@fe80::5000:ff:feab:c0{{ node_id }}%lm_cloud{{cloud}}"
 
 - name: useful info for doing clusterssh
   hosts: all
@@ -102,7 +103,7 @@
         prompt: |
           ===================================
           now to manage the cloud you can do:
-
-          clusterssh {{ linklocals | join(' ') }}
+          
+          clusterssh -o "-o 'StrictHostKeyChecking=no' -o 'HostKeyAlgorithms=+ssh-rsa'" {{ linklocals | join(' ') }}
 
           ===================================

tools/qemu_dev_start

@@ -216,7 +216,7 @@ if [ -n "$_arg_enable_wan" ]; then
     fi
 
     # DHCP server for WAN ifc
-    dnsmasq -F 172.99.0.100,172.99.0.100 --dhcp-option=3,172.99.0.1 -i "$WAN_IFC" --dhcp-authoritative --log-dhcp
+    dnsmasq -F 172.99.0.100,172.99.0.100 --dhcp-option=3,172.99.0.1 -i "$WAN_IFC" --dhcp-authoritative --log-dhcp --port=5353 --bind-dynamic
 
     # enable forwarding and NAT
     echo 1 > /proc/sys/net/ipv4/ip_forward

tools/qemu_dev_stop

@@ -1,3 +1,4 @@
 NODE_ID=${1:-0}
 [ ${#NODE_ID} = 1 ] && NODE_ID=0${NODE_ID} # Pad with leading zero
 echo system_powerdown | nc -N 127.0.0.1 "454${NODE_ID}"
+ip link del lime_br0