Merge pull request #7 from bellhyve/dev

v1.0

README.md

@@ -27,15 +27,18 @@
 
 # DESCRIPTION
 
-**ngbuddy** ("Netgraph Buddy") is an rc.d script for managing netgraph(4) networks in mixed vm and jail environments. **rc.conf** variables prefixed by **ngbuddy_** are used to manage "permanent" ng_bridge(4) and ng_eiface(4) devices.  Additional tools assist with configuring vm-bhyve and naming their sockets for statistics and graphing.
+**ngbuddy** ("Netgraph Buddy") is an rc.d script for managing netgraph(4) in mixed vm and jail environments. Netgraph provides a more flexible networking solution compared to the traditional if_bridge/epair/tap setup, offering a clearer and shorter list of virtual devices, and performance benefits for some workloads.
+
+**rc.conf** variables prefixed by **ngbuddy_** are used to manage ng_bridge(4) and ng_eiface(4) devices upon service start (and system boot). Additional tools assist with jail interface management, configuring vm-bhyve, naming vm-bhyve sockets, displaying basic statistics, and determining stable MAC addresses to help avoid collisions.
 
 # QUICK START EXAMPLE
 
-The following commands will configure a system for netgraph in a way that is suitable for most common setups on systems where no netgraph configuration currently exists.
+The following commands will configure a system for netgraph.
 
 **service ngbuddy enable**
-:    Set **rc.conf** variables and create a _public_ bridge interface associated with the host system's default route, then create a _private_ bridge linked to a virtual interface named **nghost0** suitable for a local or NAT interfaces. It will append three **rc.conf** lines similar to the following, which you can modify before starting the service:
-
+:    Sets **rc.conf** variables to enable the ngbuddy service. If no bridge definitions are set, the following default bridge definitions will be added: \
+:    _public_: A bridge interface associated with the host system's current default route, allowing guests to interact with the existing network. \
+:    _private_: A bridge linked to a new virtual interface named **nghost0**, suitable for host-only or NAT network with your guests. \
 
 ```sh
 	ngbuddy_enable="YES"
@@ -44,27 +47,25 @@ The following commands will configure a system for netgraph in a way that is sui
 ```
 
 **service ngbuddy start**
-:    This command creates the above interfaces.
+:    Creates bridges and interfaces defined in **rc.conf**.
 
 **service ngbuddy vmconf**
-:    Add the our "public" and "private" bridges to the vm(8) configuration.
-
-If you'd like to use host-only or NAT interface, you must configure the newly created **nghost0** interface. For example, you may want to set up IP addresses, a DNS resolver, and a DHCP server.
+:    Adds our _public_ and _private_ bridges to the vm(8) configuration, as a substitute for the **vm switch ...*** commands.
 
-Once post-configuration is to your liking, create jails or bhyve instances attached to your _public_ or _private_ bridges as you prefer. See the **jail_skel.conf** for assistance configuring jails.
+To get the most out of the _private_ bridge, configure **nghost0** with an IP address and add a NAT service to allow your guests to have access to the network. See the **examples** in the **ngbuddy** repository for demo scripts.
 
 # SUBCOMMANDS
 
-Subcommands are called using **service ngbuddy _SUBCOMMAND_**. Note that all commands rely on ngctl(8) and require root permissions.
+Subcommands are called using **service ngbuddy** _SUBCOMMAND_. Note that all commands rely on ngctl(8) and require root permissions.
 
 **enable**
-:    Create a basic default **ngbuddy** configuration and enable the service.
+:    Enable the **ngbuddy** service. If no bridges are defined, a _public_ and _private_ bridge will be created. See _QUICK START EXAMPLE_ above for details.
 
 **start**
-:    Load the ng_bridge(4) and ng_eiface(4) options configured in **rc.conf**. See _RC.CONF VARIABLES_ below.
+:    Load the bridge and eiface options configured in **rc.conf**. See _RC.CONF VARIABLES_ below for a complete list of options.
 
 **stop**
-:    Destroy all ng_bridge(4) and ng_eiface(4) devices, regardless of whether they were created with **ngbuddy** or not.
+:    Destroy all ng_bridge(4) and ng_eiface(4) devices, even if they were not created with **ngbuddy**.
 
 **restart**
 :    Stop, then start.
@@ -73,22 +74,22 @@ Subcommands are called using **service ngbuddy _SUBCOMMAND_**. Note that all com
 :    Print a list of ng_bridge(4), ng_eiface(4), and ng_socket(4) devices and basic usage statistics.
 
 **bridge** _bridge_ _interface_
-:    Create a bridge and an associated **rc.conf** entry. If the _interface_ exists, _bridge_ will be associated with it. Otherwise, _interface_ will be created as a new ng_eiface(4) device.
+:    Create a bridge and an associated **rc.conf** entry. If the _interface_ exists, _bridge_ will be associated with it. Otherwise, _interface_ will be created as a new eiface node.
 
 **unbridge** _bridge_
-:    Remove the indicated bridge from netgraph and **rc.conf**. This operation will fail if devices appear to be attached to it.
+:    Remove the indicated bridge from netgraph and **rc.conf**.
 
 **jail** _interface_ [_bridge_] 
-:    Create a new ng_eiface(4) associated with the indicated _bridge_. If only one ng_bridge(4) is present, _bridge_ may be omitted.
+:    Create a new eiface associated with the indicated _bridge_. If only one ng_bridge(4) is present, _bridge_ may be omitted.
 
 **unjail** _interface_ [_jail_]
-:    Remove an ng_eiface(4) associated with the indicated _jail_. If the _interface_ matches the jail name, _jail_ may be omitted.
+:    Shut down the eiface associated with the indicated _jail_. If the _interface_ matches the jail name, _jail_ may be omitted.
 
 **create** _interface_ [_bridge_]
-:    Create a new ng_eiface(4) associated with the indicated _bridge_ and add it to **rc.conf** so it will be created on startup. If only one ng_bridge(4) is present, _bridge_ may be omitted.
+:    Create a new eiface associated with the indicated _bridge_ and add it to **rc.conf** so it will be created on startup. If only one bridge is configured, _bridge_ may be omitted.
 
 **destroy** _interface_
-:    Remove the indicated ng_eiface(4) and remove it from **rc.conf**.
+:    Shut down the indicated eiface and remove it from **rc.conf**.
 
 **vmconf**
 :    Add the bridges configured in **rc.conf** to the vm(8) configuration, e.g., **/vm/.config/system.conf**.
@@ -98,7 +99,7 @@ Subcommands are called using **service ngbuddy _SUBCOMMAND_**. Note that all com
 
 # RC.CONF VARIABLES
 
-The following variables can be manually configured Some of the above subcommands will use sysrc(8) to configure rc.conf with the following variables for persistent configuration on service restart or system reboot, which can also be edited manually.
+The following variables can be manually configured. Some of the above subcommands will use sysrc(8) to configure rc.conf with the following variables for persistent configuration on service restart or system reboot.
 
 _ngbuddy_enable_
 :    Set to _YES_ to enable the service. 
@@ -107,13 +108,13 @@ _ngbuddy\_(_BRIDGE_)\_if_
 :    Link a new ng_bridge(4) device named _BRIDGE_ to the indicated interface, e.g., _eth0_. If the interface already exists, link it to the new bridge and disable LRO/TSO. If the interface does not exist, create it as an ng_eiface(4) device. This variable will be set with the **bridge** and **unbridge** subcommands.
 
 _ngbuddy\_(_BRIDGE_)\_list_
-:    A space delimited list of additional ng_eiface(4) devices that will be attached to _BRIDGE_ at startup. This variable will be set with the **create** and **destroy** subcommands.
+:    A space delimited list of additional ng_eiface(4) devices that will be attached to _BRIDGE_ at startup. This variables will be set with the **create** and **destroy** subcommands.
 
 _ngbuddy_set_mac_
-:    If set to _YES_, created ng_eiface hardware addresses will be determined only from a hash of the interface name; this ensures each interface's MAC address is stable between hosts. If set to another string, such as a host or domain name, add that seed to the MAC address generator. The default behavior will used FreeBSD's default MAC address generator, which is prone to MAC address collisions in large networks.
+:    If set to _YES_, eiface hardware addresses will be determined from a hash of the interface name, ensuring that the interfaces' MAC address are stable between hosts. If set to a string besides _YES_, that string will be added to the MAC address generator's seed.
 
 _ngbuddy_set_mac_prefix_
-:    Override the default MAC address prefix of **58:9C:FC** (the OUI of the FreeBSD Foundation). For example, you can set _ngbuddy_set_mac_prefix="02"_ to minimize the risk of collisions.
+:    Override the default MAC address prefix of **58:9C:FC** (the OUI of the FreeBSD Foundation). For example, you can set _ngbuddy_set_mac_prefix="02"_ to minimize the risk of collisions. _ngbuddy_set_mac_ must also be enabled to use this feature.
 
 _ngbuddy_set_mac_hash_
 :    Override the default hash command of **sha1** with the command indicated. The command's output will receive the seed through standard input (see _ngbuddy_set_mac_) and must return enough hexadecimal characters to complete the MAC address.
@@ -128,15 +129,12 @@ _ngbuddy_set_mac_hash_
 **/usr/local/share/ngbuddy/ngbuddy-mmd.awk**
 :    An alternative to **ngctl dot** that creates a Mermaid-JS color diagram of netgraph nodes.
 
-# NOTES
-
-These scripts were developed to assist with new netgraph features in **vm-bhyve 1.5+**, and were inspired by the **/usr/share/examples/jails/jng** example script and additional examples by Klara Systems.
-
 # EXAMPLES
+For examples and demo scripts, see **examples** at: https://github.com/bellhyve/netgraph-buddy
 
 **Example 1: Quickly deploy a VNET jail with netgraph using jail.conf.d**
 
-The following steps will configure a jail attached to the interface associated with your default route, likely your LAN, using DHCP. See the files in **examples** at: https://github.com/bellhyve/netgraph-buddy
+The following steps will configure a jail attached to the interface associated with the host's current default route, likely your LAN, using DHCP.
 
 First, set up Netgraph Buddy: \
 - **service ngbuddy enable** \
@@ -155,9 +153,9 @@ Configure the jail configuration: \
 To create more jails, you can: \
 - Copy **/jail/my_jail/** to **/jail/new_jail1/** \
 - Copy **/etc/jail.conf.d/my_jail.conf** to **new_jail1.conf** \
-- Edit the new configuration as above, chaning the word **my_jail** to **new_jail1** \
-- Run: **service jail start new_jail1**
-- And repeat as desired.
+- Edit the new configuration as above and change the word **my_jail** to **new_jail1** \
+- Run: **service jail start new_jail1** \
+- And repeat as desired. \
 
 **Example 2: An rc.conf example for a slightly more complex setup**
 
@@ -201,6 +199,10 @@ wan
   ix0 (lower): RX 46.32 KB, TX 30.92 KB
 ```
 
+# NOTES
+
+These scripts were developed to assist with new netgraph features in **vm-bhyve 1.5+**, and were inspired by the **/usr/share/examples/jails/jng** example script and additional examples by Klara Systems.
+
 # SEE ALSO
 
 jail(8), netgraph(4), ng_bridge(4), ngctl(8), ng_eiface(4), ng_socket(4), vm(8)

examples/jail_skel.conf

@@ -3,7 +3,7 @@
 # An example simple jail using Netgraph Buddy. One useful strategy is to match
 # the jail's name with the interface name and relative path name. This jail
 # uses the devfs.rules example to allow DHCP, pf, and System V IPC primatives
-# (for applications like Postgres and Zabbix).
+# (for applications like Postgres).
 #
 # When using this as a template, change $bridge, $if_name, and the jail's name
 # as desired.

examples/ten_jails.sh

@@ -0,0 +1,94 @@
+#!/bin/sh
+#
+# ten_jails.sh
+#
+# This script is intended to be used on a blank virtual machine to illustrate
+# using Netgraph Buddy and ZFS to very quickly deploy ten jails in different
+# ways. There are no requirements besides a network connection, a FreeBSD base
+# package, and Netgraph Buddy.
+#
+# Netgraph Buddy will use jail.conf to create five "public" jails associated
+# with a system's default route and 5 "private" jails which use the host as a
+# DHCP server and router.
+#
+# This script is intended for use on an empty system with ZFS, such as the
+# FreeBSD 140 ZFS VM image:
+#
+#	https://download.freebsd.org/ftp/releases/VM-IMAGES/14.0-RELEASE/amd64/Latest/FreeBSD-14.0-RELEASE-amd64-zfs.raw.xz
+#
+# WARNING: Several commands below will overwrite jail-related files and network
+# settings; please use this script with caution.
+
+JAIL_DS=zroot/jail
+PRIVATE_NET=10.2.19
+
+set -x
+pkg install -y git-lite dhcpd
+
+# Set up ngbuddy
+# TODO: Replace with ngbuddy FreeBSD package.
+git clone https://github.com/bellhyve/ngbuddy.git
+cd ngbuddy
+cp ngbuddy /usr/local/etc/rc.d/
+cp ngbuddy.8 /usr/local/share/man/man8/
+cp -r share/ngbuddy/ /usr/local/share/ngbuddy/
+cp examples/jail_skel.conf /etc/jail.conf.d/
+cat examples/devfs.rules >> /etc/devfs.rules
+cd
+service ngbuddy enable
+service ngbuddy start
+
+# Set up jail template
+JAIL_SKEL_DS=$JAIL_DS/jail_skel
+JAIL_SKEL_CONF=/etc/jail.conf.d/jail_skel.conf
+zfs create -p $JAIL_SKEL_DS
+zfs set mountpoint=/jail $JAIL_DS
+JAIL_DIR=/jail/jail_skel
+[ ! -e base.txz ] && fetch https://download.freebsd.org/ftp/releases/amd64/amd64/14.0-RELEASE/base.txz -q
+[ ! -e $JAIL_DIR/etc/rc.conf ] && tar zxf base.txz -C $JAIL_DIR
+sysrc -f $JAIL_DIR/etc/rc.conf ifconfig_DEFAULT=SYNCDHCP
+zfs snapshot $JAIL_SKEL_DS@a
+
+# Set up jails
+for j in $(jot 5); do
+	# Public
+	jname=pubjail$j
+	zfs clone $JAIL_SKEL_DS@a $JAIL_DS/$jname
+	sed s/jail_skel/$jname/ $JAIL_SKEL_CONF > /etc/jail.conf.d/$jname.conf
+	sysrc jail_list+=$jname
+
+	# Private
+	jname=prijail$j
+	zfs clone $JAIL_SKEL_DS@a $JAIL_DS/$jname
+	sed -e s/jail_skel/$jname/ -e s/public/private/ $JAIL_SKEL_CONF > /etc/jail.conf.d/$jname.conf
+	sysrc jail_list+=$jname
+done
+
+# Public jails can be started at this point;
+# no additional configuration required
+
+# Set up networking for private jails
+DNS_SERVER=$(grep -o '[0-9].*' /etc/resolv.conf)
+EXT_IF=$(netstat -rn|awk '$1 == "default"{print $4}')
+echo "nat on $EXT_IF from $PRIVATE_NET.0/24 to any -> ($EXT_IF)" > /etc/pf.conf
+cat > /usr/local/etc/dhcpd.conf << EOF
+option domain-name-servers $DNS_SERVER;
+option subnet-mask 255.255.255.0;
+subnet $PRIVATE_NET.0 netmask 255.255.255.0 {
+  range  $PRIVATE_NET.101 $PRIVATE_NET.199;
+  option routers $PRIVATE_NET.1;
+}
+EOF
+# Enable and start required network-related services for private jails
+sysrc gateway_enable=YES pf_enable=YES dhcpd_enable=YES dhcpd_flags=nghost0
+sysrc jail_enable=YES jail_parallel_start=YES
+sysrc ifconfig_nghost0="inet $PRIVATE_NET.1/24 up"
+# The next two lines are required before the next reboot for private jails
+ifconfig nghost0 inet $PRIVATE_NET.1/24 up
+sysctl net.inet.ip.forwarding=1
+# Start firewall and DHCP server for private jails
+service pf start
+service dhcpd start
+
+# Start ten jails
+service jail start