-
Notifications
You must be signed in to change notification settings - Fork 22
Setting up OVPL on a VM using Vagrant
the OVPL (One VM per lab) project makes use of openVZ to host "Virtual Labs". Each lab is created on a separate container, hence the name one VM (container) per lab. These containers are created by interacting with openVZ, a virtualization tool for linux.
One of the problems of OpenVZ is that it can only be deployed as a kernel level patch. Since OVPL depends on openVZ, this requires one to have a compliant kernel. Therefore, one requires an OS which provides a linux kernel patched with OpenVZ (CentOS for example, does this).
The other way around this is to run an OS which provides a compliant kernel by using a VM. This tutorial willuse this method (setting up a CentOS VM) because it is far more practical.
Vagrant, a tool which allows one to create development environments. This will be used to create a VM with CentOS to run OVPL.
Vagrant internally depends on VirtualBox, a virtualization product. This also needs to be installed as it is a Vagrant dependency
Virtualization on Intel requires direct hardware support. This is usually disabled in most computers. To enable, first enter into the BIOS of your computer.
Note: the process of entering the BIOS varies from computer to computer. The most common method is to hold down F2 when the computer boots up to access the BIOS.
Once you have accessed the BIOS, search for the Visualization option and enable it. Then save and quit.
Note: If you do not have this option in the BIOS, then chances are that your hardware has this enabled by default.
Link to documentation. A TL;DR is provided here, but it is strongly recommended to go and read the documentation, since nothing beats the source.
###Install DKMS(Dynamic Kernel Module Support)###
DKMS is a framework that VirtualBox uses to build and upgrade kernel modules.
On Ubuntu:
sudo apt-get install dkms
On Fedora:
yum install dkms
On Mandriva or Mageia
urpmi dkms
###Install VirtualBox on your flavor of Linux###
Go to the VirtualBox downloads page and install VirtualBox according to the instructions laid out. Link to virtualBox downloads.
This step is not covered in the wiki since there is quite a bit of variation between different flavors of Linux [TODO: Complete this?]
You will be asked to accept the VirtualBox Personal Use and Evaluation License. answer "yes" and continue.
The installer will also try and install a suitable kernel module. Kernel modules for the most common cases are prepackaged and this should proceed without incident. If no suitable kernel module is found, the installation script tries to build a module itself.
If the build process fails, take a look at /var/log/vbox-install.log to find out how the compilation failed. You probably have to install the correct Linux headers.
Once the problem is corrected, run sudo /etc/init.d/vboxdrv setup to give it a second try.
Once again, please read the official VirtualBox documentation. It covers far more than what can be realistically covered here.
Now that VirtualBox is correctly installed, Vagrant is to be setup next.
Link to official documentation.
###Official Method###
Go to the downloads page and get the appropriate installer or package for your platform. Then install it using the package manager.
###Alternate Method###
This method depends on Ruby's package manager, RubyGems. You should first install Ruby and RubyGems on your platform.
For Ubuntu:
sudo apt-get install rubygems ruby-dev
For Fedora:
sudo yum install rubygems ruby-devel rubygem-ffi
Now that RubyGems is installed, run
sudo gem install vagrant
to install Vagrant.
Now that Vagrant is installed, the next step is to actually create a VM using Vagrant
Link to official documentation
###Introduction###
Vagrant creates a file known as a Vagrantfile when it is initialized. It is recommended to create a separate folder to hold this file. So, preferable, create a folder somewhere called CentOS for example.
###Installation###
cd into the appropriate folder.
Now, we need to actually create a VM with the appropriate OS. A 32 bit or a 64 bit version of CentOS can be used.
Caveat: There are 2 versions of CentOS which can be installed - one 32bit (i386) and one 64bit (x84_64). Some processors are unable to emulate the 64 bit version, so it is recommended to use the 32bit version. However, the 64bit version can be used too. Link to docs regarding 64 bit Guest OS
To setup the 32 bit version, run:
vagrant init chef/centos-6.5-i386
To setup the 64 bit version, run:
vagrant init chef/centos-6.5
This should print
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
Next, run
vagrant up
to actually setup the VM.
This step downloads the correct version of the OS as specified by the vagrant init chef/____ command and also installs the OS onto the VM.
Now, you have a fully functional VM which you can use! We're not done yet, but we're getting there. Onward!
Link to official documentation
Vagrant by default runs all VM's headless (with no UI at all), so they only way to access a Vagrant VM is to ssh into it.
To ssh into the Vagrant VM, run
vagrant ssh
This should place you inside a bash prompt as [vagrant@localhost ~]
Cool! we're inside a VM running CentOS :)
Right now, we have a VM with CentOS installed, but there's no OpenVZ on it. The next step is to actually install OpenVZ onto the OS.
Note: These steps should be followed inside the VM
Link to the best documentation found
Link to official documentation
This is the only step for which the unofficial docs are recommended, simply because it is very detailed.
###Gain Root Privileges###
Explanation of what sudo su - does
Vagrant by default ssh's into a normal user called vagrant. However, to install OpenVZ, we require root permissions. So, run
sudo su -
to login as root. This should change the bash prompt to [root@localhost ~]#.
###Change root password###
Change the root password using passwd.
type passwd.
This will request a new password. type a chosen password once, then retype to confim.
passwd: all authentication tokens updated successfully
Should be displayed once this is done correctly.
Note: All steps henceforth assume root privileges
###Install OpenVZ###
CentOS uses the yum package manager, which does not have OpenVZ in it's repositories by default. This needs to be added manually. To do this, run
cd /etc/yum.repos.d
wget http://download.openvz.org/openvz.repo
rpm --import http://download.openvz.org/RPM-GPG-Key-OpenVZ
###Modify OpenVZ.repo###
Open openvz.repo by using any editor you want to (vi, nano, emacs, they all work).
Example:
vi openvz.repo
In the openvz.repo file, disable [openvz-kernel-rhel5] repository (enabled=0) and enable the [openvz-kernel-rhel6] repository instead (enabled=1):
Example openvz.repo file:
openvz.repo
...
[openvz-kernel-rhel5]
name=OpenVZ RHEL5-based kernel
#baseurl=http://download.openvz.org/kernel/branches/rhel5-2.6.18/current/
mirrorlist=http://download.openvz.org/kernel/mirrors-rhel5-2.6.18
enabled=0
gpgcheck=1
gpgkey=http://download.openvz.org/RPM-GPG-Key-OpenVZ
[...]
[openvz-kernel-rhel6]
name=OpenVZ RHEL6-based kernel
#baseurl=http://download.openvz.org/kernel/branches/rhel6-2.6.32/current/
mirrorlist=http://download.openvz.org/kernel/mirrors-rhel6-2.6.32
enabled=1
gpgcheck=1
gpgkey=http://download.openvz.org/RPM-GPG-Key-OpenVZ
...
###Install the OpenVZ Kernel###
The repository contains a different OpenVZ kernels. Install a kernel using
yum install vzkernel
This will provide a choice of OpenVZ kernels. pick [TODO: Specify which kernel to pick] and install OpenVZ.
###Update GRUB###
GRUB, which the booatloader should have been updated to contain the new OpenVZ kernel in it's list of OS images.
open the file /boot/grub/menu.lst and change the value of default to 0 so that the 1st kernel (the OpenVZ kernel) boots automatically.
Example /boot/grub/menu.lst file:
/boot/grub/menu.lst
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/mapper/vg_server1-lv_root
# initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=0 #this should be made sure to be 0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS Linux OpenVZ (2.6.32-042stab020.1)
root (hd0,0)
kernel /vmlinuz-2.6.32-042stab020.1 ro root=/dev/mapper/vg_server1-lv_root rd_LVM_LV=vg_server1/lv_root rd_LVM_LV=vg_server1/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=de crashkernel=auto rhgb quiet
initrd /initramfs-2.6.32-042stab020.1.img
title CentOS (2.6.32-71.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-71.el6.x86_64 ro root=/dev/mapper/vg_server1-lv_root rd_LVM_LV=vg_server1/lv_root rd_LVM_LV=vg_server1/lv_swap rd_NO_LUKS rd_NO_MD rd_NO_DM LANG=en_US.UTF-8 SYSFONT=latarcyrheb-sun16 KEYBOARDTYPE=pc KEYTABLE=de crashkernel=auto rhgb quiet
initrd /initramfs-2.6.32-71.el6.x86_64.img
###Install OpenVZ User Tools###
Now that OpenVZ has been integrated into the kernel, we need tools to interact with it. Vzctl and Vzquota are common tools which we will be installing.
Install the tools with
yum install vzctl vzquota ploop
###Edit sysctl.conf###
There are a number of kernel parameters that should be set for OpenVZ to work correctly. These parameters are stored in /etc/sysctl.conf file.
Open /etc/sysctl.conf and make sure that you have the following settings in it:
/etc/sysctl.conf
...
# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled
net.ipv4.ip_forward = 1
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.conf.default.proxy_arp = 0
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Enables the magic-sysrq key
kernel.sysrq = 1
# We do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0
...
Run
sysctl -p
to reload it's settings.
Sample systcl -p output:
net.ipv4.ip_forward = 1
net.ipv6.conf.default.forwarding = 1
net.ipv6.conf.all.forwarding = 1
net.ipv4.conf.default.proxy_arp = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.accept_source_route = 0
kernel.sysrq = 1
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0
kernel.core_uses_pid = 1
net.ipv4.tcp_syncookies = 1
error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key
error: "net.bridge.bridge-nf-call-iptables" is an unknown key
error: "net.bridge.bridge-nf-call-arptables" is an unknown key
kernel.msgmnb = 65536
kernel.msgmax = 65536
kernel.shmmax = 4294967295
kernel.shmall = 268435456
###TODO: NOT SURE IF THIS SECTION IS REQUIRED OR NOT###
If the IP addresses of your virtual machines are from a different subnet than the host system's IP address, This step is required. Otherwise, networking will not work.
Open /etc/vz/vz.conf and set NEIGHBOUR_DEVS to all
/etc/vz/vz.conf
...
NEIGHBOUR_DEVS=all
...
###Disable SELinux###
SELinux needs to be disabled if you want to use OpenVZ.
Open /etc/sysconfig/selinux and set the value of SELINUX to disabled
/etc/sysconfig/selinux
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
# enforcing - SELinux security policy is enforced.
# permissive - SELinux prints warnings instead of enforcing.
# disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of these two values:
# targeted - Targeted processes are protected,
# mls - Multi Level Security protection.
SELINUXTYPE=targeted
###Exit the VM###
Run exit to logout as root.
Run exit again to exit the VM.
Example of exiting VM:
[root@localhost yum.repos.d]#
[root@localhost yum.repos.d]# exit
logout
[vagrant@localhost ~]$ exit
logout
Connection to 127.0.0.1 closed.
###Setup port forwarding for OVPL###
Note: Now you should be on your computer, not the VM.
OVPL by default uses two ports - 8000 and 8080. But these ports will be inside the VM not outside. To access there ports from the outside, we need to setup port forwarding.
Edit the Vagrantfile to enable port forward of ports 8000 and 8080 by entering these lines to the very end of the file
Vagrantfile:
Vagrant.configure("2") do |config|
config.vm.network "forwarded_port", guest: 8080, host: 8080
config.vm.network "forwarded_port", guest: 8000, host: 8000
end
This will port forward the VM's 8080 port to your port 8080, and the same for port 8000
###Reload the VM###
Note: Now you should be on your computer, not the VM.
Run
vagrant reload
to reboot the vm.
Example output of vagrant reload:
==> default: Attempting graceful shutdown of VM...
==> default: Checking if box 'chef/centos-6.5-i386' is up to date...
==> default: Clearing any previously set forwarded ports...
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
default: Adapter 1: nat
==> default: Forwarding ports...
default: 22 => 2222 (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
default: SSH address: 127.0.0.1:2222
default: SSH username: vagrant
default: SSH auth method: private key
default: Warning: Connection timeout. Retrying...
default: Warning: Connection timeout. Retrying...
default: Warning: Remote connection disconnect. Retrying...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
default: /vagrant => /home/bollu/tmp/vag
Failed to mount folders in Linux guest. This is usually because
the "vboxsf" file system is not available. Please verify that
the guest additions are properly installed in the guest and
can work properly. The command attempted was:
mount -t vboxsf -o uid=`id -u vagrant`,gid=`getent group vagrant | cut -d: -f3` vagrant /vagrant
mount -t vboxsf -o uid=`id -u vagrant`,gid=`id -g vagrant` vagrant /vagrant
###Troubleshooting Vagrant error###
If you receive the error
Failed to mount folders in Linux guest. This is usually because
the "vboxsf" file system is not available. Please verify that
the guest additions are properly installed in the guest and
can work properly. The command attempted was:
mount -t vboxsf -o uid=`id -u vagrant`,gid=`getent group vagrant | cut -d: -f3` vagrant /vagrant
mount -t vboxsf -o uid=`id -u vagrant`,gid=`id -g vagrant` vagrant /vagrant
This is a very well known bug with Vagrant and is nothing to be worried about. If this issue crops up, run
vagrant ssh
first to enter into the VM, and then run
sudo ln -s /opt/VBoxGuestAdditions-4.3.10/lib/VBoxGuestAdditions /usr/lib/VBoxGuestAdditions
Afterwards, exit the VM with exit and run
vagrant reload again.
###SSH into the reloaded VM###
ssh into the VM again after running reload with
vagrant ssh
###Check if everything's done right###
Note: This is inside the VM
Run
uname -r
The output should be something along the lines of
2.6.32-042stab088.4
Note: These steps should be followed inside the VM
###Switch to root###
Once again, run
sudo su -
to switch to root.
###install git###
Git is a distributed source code management system. Github is built on git, and therefore requires git to interface with Github.
Since OVPL is stored on Github, Git is a requirement for OVPL. OVPL also uses git internally to deploy virtual labs.
Run
yum install git
to install git.
Once this is done, run
git config --global color.ui auto
to enable colored git output.
Once this is done, setup your name and email id for git by running
$ git config --global user.name "your name"
$ git config --global user.email [email protected]
###Clone OVPL###
To "clone" git terminology to copy and setup an existing repository on your computer.
First, go to the /root/ directory by typing
cd /root/
Next, clone the OVPL Repository by running
git clone https://github.com/vlead/ovpl.git
Output of git clone
git clone https://github.com/vlead/ovpl.git
Initialized empty Git repository in /root/ovpl/.git/
remote: Reusing existing pack: 577, done.
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 584 (delta 2), reused 2 (delta 0)
Receiving objects: 100% (584/584), 1.53 MiB | 91 KiB/s, done.
Resolving deltas: 100% (332/332), done.
###Switch to the config branch###
Note [TODO: take the time to explain what's actually happening. For now, I've just written down the commands needed to switch to the config branch and then create a local copy. However, a proper explanation is really needed]
Enter the newly created OVPL folder by typing
cd /root/ovpl
The next step is to update the OVPL repository to the latest version. To do this, run
git checkout origin/config
git checkout -b config
On running git branch, the output should be
* config
master
Cool! We're almost done! Now that OVPL is setup and we're on the latest version of the codebase, it's time to configure OVPL!
Note: These steps should be followed inside the VM
Note: These steps assume root privileges
##Install GCC###
GCC is GNU's C/C++ Compiler, which is used to build most python packages. To install GCC, run
yum install gcc
###Install Pip, a python package manager###
OVPL as a system depends on a few python libraries. To install these libraries, we'll use pip, a package management tool for python.
To install pip, run
curl https://bootstrap.pypa.io/get-pip.py | python
Example output of curl https://bootstrap.pypa.io/get-pip.py | python
[root@localhost src]# curl https://bootstrap.pypa.io/get-pip.py | python
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1535k 100 1535k 0 0 168k 0 0:00:09 0:00:09 --:--:-- 253k
Downloading/unpacking pip
Downloading pip-1.5.6-py2.py3-none-any.whl (1.0MB): 1.0MB downloaded
Downloading/unpacking setuptools
Downloading setuptools-3.6-py2.py3-none-any.whl (547kB): 547kB downloaded
Installing collected packages: pip, setuptools
Successfully installed pip setuptools
Cleaning up...
###Install Python Dependencies###
Before installing Python's dependencies, Python has to be updated. To update, run
yum update
To install OVPL's python code dependencies, run
pip2 install tornado requests netaddr pymongo sh
###Install MongoDB###
Open /etc/yum.repos.d/mongodb.repo to inform yum of MongoDB's repository
If you're using the 64 bit version, the file should look like this:
/etc/yum.repos.d/mongodb.repo
[mongodb]
name=MongoDB Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/x86_64/
gpgcheck=0
enabled=1
If you're using the 32 bit version, the file should look like this:
/etc/yum.repos.d/mongodb.repo
[mongodb]
name=MongoDB Repository
baseurl=http://downloads-distro.mongodb.org/repo/redhat/os/i686/
gpgcheck=0
enabled=1
Next, to install MongoDB, run
yum install mongodb-org
After installation, run
service mongod start
to start mongodb
###Make a directory to store OVPL logs###
OVPL stores its logfiles in /root/ovpl/log/
So, create this directory by running mkdir /root/ovpl/log/
###Enter the src directory###
OVPL stores all of it's code inside the /ovpl/src directory. To move to this directory, run
cd /root/ovpl/src/
###Startup OVPL's services###
Startup OVPL by running
python2 CentOSVZAdapterServer.py & python2 ControllerServer.py &
This should output
[root@localhost src]# [D 140528 11:29:25 CentOSVZAdapterServer:70] CentOSVZAdapterServer: __main__() PORT=8000, CreateURI=/api/1.0/vm/create, DestroyURI=/api/1.0/vm/destroy, RestartURI=/api/1.0/vm/restart
[D 140528 11:29:25 ControllerServer:54] ControllerServer: It will run on port : 8080
Now, go to your browser (in your HOST OS) and type
http://127.0.0.1:8080/
in the address bar.
This should take you to a "Test your lab!" page.
###Congratulations!###
You've now setup OVPL. A few things remain (changing adapter locations and the like), so continue reading! But do take a minute to give yourself a pat on the back :)
###Setup OVPL Config###
###To access the lab from base machine###
1.Open Oracle VM VirtualBox
2.Goto Settings > Network
3.Under Adapter 1 tab change the Attached to: option from NAT to Briged Adapter
4.Press OK
1.Generate an SSH key using ssh-keygen and upload to bitbucket account under
Manage -> SSH Keys -> Add Key
Git pull the lab when running the service for the first time.
press y to authenticate. This process is needed only the first time we set up a lab.
##Copy custom template##
Copy the customized OS template from base4 to /vz/template/cache/
Note: Contact people who is having access to base4 machine. Tirumal or yogesh should do it for you.
##Enable Networking##
Note: this is only if you are installing CentOS directly and are not using Vagrant
1.edit iptables in /etc/sysconfig/iptables
enable forwarding to ACCEPT
-A FORWARD -j ACCEPT
2.In /etc/modprobe/openvz.conf
options nf_conntrack ip_conntrack_disable_ve0=0