Skip to content

Development of and creating tarbackup

nanch edited this page Mar 30, 2013 · 1 revision

This is a spot where I can put stuff that was going to go in the wiki but was changed so is no longer useful to the project.

None of this information pertains to tarbackup and is deprecated (NOT USED). This is for archival purposes only.

Well, I tried FreeBSD 9.0 with ZFS as the root and it failed to boot after an interrupted (CTRL+ALT+DEL) boot. The error message on boot is "can't exec getty '/usr/libexec/getty/' for port /dev/ttyv0 \n No such file or directory" I'm trying CentOS 6.3 with mdadm and a RAID6 with regular scrubbing for fault tolerance. It won't give me the end-to-end data garauntees of the ZFS, but at least I'll have something up and running.


Deprecated CentOS design

upload your data to tarbackup

curl -u username:password -i -o /tmp/output.txt -T /tmp/example.txt 'http://tarbackup.com/put.php'

Download your data from tarbackup.com via wget

wget --user=yourusername --password='yourPassword' http://tarbackup.com/fullbackup.tar.gz.enc


Old ZFS on FreeBSD design

Where the data is hosted

  • On a box running FreeBSD 9.0 To test this system on a virtual machine, I'm using:
  • 3 virtual hard drives of 4GB per drive
  • 3 storage drives of 500MB per drive
  • 1536 MB of allocated memory for the virtual system

Follow these steps to make your own:

download the FreeBSD 9.0 ISO installation CD; http://www.freebsd.org/where.html

Commands to create the server:

(referenced from: http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/RAIDZ2 and http://wiki.freebsd.org/RootOnZFS/GPTZFSBoot/RAIDZ2)

At the install prompt, choose "Shell", then type:

gpart create -s gpt ada0
gpart create -s gpt ada1
gpart create -s gpt ada2

gpart add -s 64K -t freebsd-boot -l boot0 ada0
gpart add -s 64K -t freebsd-boot -l boot1 ada1
gpart add -s 64K -t freebsd-boot -l boot2 ada2

gpart add -s 512M -t freebsd-swap -l swap0 ada0
gpart add -s 512M -t freebsd-swap -l swap1 ada1
gpart add -s 512M -t freebsd-swap -l swap2 ada2

gpart add -t freebsd-zfs -l disk0 ada0
gpart add -t freebsd-zfs -l disk1 ada1
gpart add -t freebsd-zfs -l disk2 ada2

gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada0
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada2

zpool create -o altroot=/mnt zroot raidz2 /dev/gpt/disk0 /dev/gpt/disk1 /dev/gpt/disk2
zpool list

zfs create zroot/tmp
chmod 1777 /mnt/tmp
zfs create zroot/usr
zfs create zroot/var
zfs create zroot/home
exit

continue the install; when the installer asks about partitions, choose shell and type:

exit

after install is finished, reboot and then boot to shell from Live CD on the next boot

echo 'zfs_enable="YES"' >> /mnt/etc/rc.conf
echo 'zfs_load="YES"' >> /mnt/boot/loader.conf
echo 'vfs.root.mountfrom="zfs:zroot"' >> /mnt/boot/loader.conf
zfs unmount -a
zpool export zroot
zpool import -o cachefile=/tmp/zpool.cache -o altroot=/mnt zroot

zfs set mountpoint=/ zroot
cp /tmp/zpool.cache /mnt/boot/zfs/
zfs unmount -a
zpool set bootfs=zroot zroot
zpool set cachefile= zroot
zfs set mountpoint=legacy zroot
zfs set mountpoint=/tmp zroot/tmp
zfs set mountpoint=/usr zroot/usr
zfs set mountpoint=/var zroot/var
zfs set mountpoint=/home zroot/home
reboot

(after reboot, to create the storage pool)

zpool create data raidz2 ada3 ada4 ada5

(be sure that the directory was properly created and mounted)

cd /data

(list the zpools)

zpool list
pkg_add -r apache22
echo 'apache22_enable="yes"' >> /etc/rc.conf

configure the /etc/hosts file with the IP of your server, in my case:

echo 'vtarbackup 192.168.2.177' >> /etc/hosts
reboot or service apache22 start or service apache22 restart or /usr/local/etc/rc.d/apache22 restart

create a new group

pw groupadd targroup

get tarbackup with wget

wget --no-check-certificate https://github.com/nanch/tarbackup/zipball/master -O tarbackup.zip
tar -xvzf tarbackup.zip

a new directory nanch-tarbackup-XXXXXXX has been created

alternatively, if you don't want to use wget, you can get tarbackup with git

pkg_add -r git
git clone git://github.com/nanch/tarbackup/

if you already have partitions on the disk before you start, you can delete them from the install shell with: (referenced from: http://www.wonkity.com/~wblock/docs/html/disksetup.html)

gpart delete -i 1 ada0
gpart delete -i 2 ada0
gpart delete -i 3 ada0

for editting files

I use nano, which can be installed with pkg_add -r nano and is executed with /usr/local/bin/nano filename

Reference Links for the FreeBSD tarbackup effort

http://www.geekvenue.net/chucktips/jason/chuck/994016279/index_html

https://help.ubuntu.com/community/BackupYourSystem/TAR

http://ubuntuforums.org/showthread.php?t=35087

http://www.aboutdebian.com/tar-backup.htm

https://wiki.archlinux.org/index.php/Full_System_Backup_with_tar

http://wiki.openstreetmap.org/wiki/Using_curl_to_upload_data

http://lildude.co.uk/zfs-cheatsheet

http://www.madboa.com/geek/openssl/ (openssl worked better than gpg for me; the gpg interface seems to require a user to type in their password every time)

I tried gpg but gave up. I'll keep this for reference:

gpg

to encrypt: gpg -c --output fullbackup.tar.gz.gpg fullbackup.tar.gz

to decrypt: gpg --decrypt fullbackup.tar.gz.gpg > fullbackup.tar.gz

http://linux.about.com/library/cmd/blcmdl1_diff.htm (you can use this to verify your encrypt/decrypt process works full-circle)

http://sleepyhead.de/howto/?href=crypt

http://superuser.com/questions/371469/linux-file-system-for-a-big-file-server

https://www.dan.me.uk/blog/2012/05/06/full-disk-encryption-with-zfs-root-for-freebsd-9-x/

http://unix.stackexchange.com/questions/32319/freebsd-zfs-encryption-alternatives-suggestions

http://forums.freebsd.org/showthread.php?t=4108

http://www.itsacon.net/computers/unix/growing-a-zfs-pool/