-
Notifications
You must be signed in to change notification settings - Fork 4
Virtual Machine Migration using bhyve
NOTE: The project is under development and may not be completed.
root@host # git clone https://github.com/FreeBSD-UPB/freebsd
root@host # cd freebsd
NOTE: Please choose the projects_warm_migration branch for warm migration or projects/bhyve_migration_dev branch for live migration.
root@host # git checkout projects/bhyve_warm_migration
root@host # git checkout projects/bhyve_migration_dev
root@host # make buildworld -j9
root@host # make installworld -j9
root@host # reboot
root@host # make buildkernel -j9
root@host # make installkernel -j9
root@host # reboot
Note: The project should be build and installed on both source and destination hosts. Alternatively, the project can be cloned on the shared storage (set using NFS and described in the next step) and build & installed from there on each system.
To work, the migration process needs that both (source and destination) hosts to share the same image. You can setup a shared storage using NFS following the steps described here. You can setup one of the hosts to be server and the other one to be client.
To create a virtual machine using bhyve, you can use the tutorial from here.
NOTE: It is important the ''guest.img'' created to be shared between the two host systems.
Run the previous created virtual machine using bhyve.
## load the OS
root@source_host # bhyveload -c stdio -m 512M -d guest.img guest_vm
## start the virtual machine
root@source_host # bhyve -c 1 -m 512M -H -A -P -s 0:0,hostbridge -s 1:0,lpc -s 3:0,virtio-net,tap0 -s 4:0,ahci,guest.img -l com1,stdio guest_vm
....
root@guest #
In order to warm migrate a virtual machine, it is necessarily to start an empty virtual machine on the destination system which will wait for connections from certain host on a certain port. To start the virtual machine for warm migration on the destination host, the ''-R <hostname_or_IP>,[]'' option shall be added to the bhyve command. The default port is 24983 (from "bhyve").
## load the OS
root@destination_host # bhyveload -c stdio -m 512M -d guest.img guest_vm_dst
# start the virtual machine for warm migration
# note the -R <hostname_or_IP>,[<PORT>] option
root@destination_host # bhyve -c 1 -m 512M -H -A -P -s 0:0,hostbridge -s 1:0,lpc -s 3:0,virtio-net,tap0 -s 4:0,ahci,guest.img -l com1,stdio -R source_host,24983 guest_vm_dst
root@source_host # bhyvectl --migrate=destination_host,24983 --vm=guest_vm
On the source host, the virtual machine will be closed(destroyed). On the destination host, after the warm migration process is completed, the access to the migrated virtual machine will be granted and a console will start:
root@guest #
Run the previous created virtual machine using bhyve. For live migration, it is important that the memory is wired. Currently, the live migration process fails if the guest's memory is not wired.
## load the OS
root@source_host # bhyveload -c stdio -m 512M -S -d guest.img guest_vm
## start the virtual machine
root@source_host # bhyve -c 1 -m 512M -H -A -P -S -s 0:0,hostbridge -s 1:0,lpc -s 3:0,virtio-net,tap0 -s 4:0,ahci,guest.img -l com1,stdio guest_vm
....
root@guest #
In order to live migrate a virtual machine, it is necessarily to start an empty virtual machine on the destination system which will wait for connections from certain host on a certain port. To start the virtual machine for live migration on the destination host, the ''-R <hostname_or_IP>,[]'' option shall be added to the bhyve command. The default port is 24983 (from "bhyve"). Do not forget to specify that the guest's memory is wired.
## load the OS
root@destination_host # bhyveload -c stdio -m 512M -S -d guest.img guest_vm_dst
# start the virtual machine for live migration
# note the -R <hostname_or_IP>,[<PORT>] option
root@destination_host # bhyve -c 1 -m 512M -H -A -P -S -s 0:0,hostbridge -s 1:0,lpc -s 3:0,virtio-net,tap0 -s 4:0,ahci,guest.img -l com1,stdio -R source_host,24983 guest_vm_dst
root@source_host # bhyvectl --migrate-live=destination_host,24983 --vm=guest_vm
On the source host, the virtual machine will continue running until while its state is live migrated on the destination host. It will be a little downtime on the source host at the end of the procedure and then the source guest will be destroyed. On the destination host, after the live migration process is completed, the access to the migrated virtual machine will be granted and a console will start:
root@guest #