mountUbuntutoS3.txt

S3 mount link on ubuntu 16:-

Links:-
http://www.whiteboardcoder.com/2017/12/how-to-mount-s3-bucket-as-hard-drive.html
https://serverfault.com/questions/441691/how-to-make-s3fs-work-with-iam-roles

1> create an S3 Bucket, if already created ignore this step

aws s3api create-bucket --bucket testerssss --region us-east-1

2> Create IAM User or attach a role

User/Role creation with policy(Change the "bucketName")

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": ["arn:aws:s3:::bucketName"]
        },
        {
            "Effect": "Allow",
            "Action": [
               "s3:PutObject",
               "s3:GetObject",
               "s3:DeleteObject"
            ],
            "Resource": ["arn:aws:s3:::bucketName/*"]
        }
    ]
}


3> In the server install Dependencies

sudo apt-get update
sudo apt-get install automake autotools-dev \
fuse g++ git libcurl4-gnutls-dev libfuse-dev \
libssl-dev libxml2-dev make pkg-config -y

4> Configure s3fs

git clone https://github.com/s3fs-fuse/s3fs-fuse.git
cd s3fs-fuse
./autogen.sh
./configure
make 
sudo make install

5> Store the credentials if using IAM user credentials or skip this if using IAM Role

sudo vi /etc/passwd-s3fs
Access:Secret

6> Give appropriate Permissions for the credentials file

sudo chmod 640 /etc/passwd-s3fs

7> To allow non root users in system to add files to s3

sudo vi /etc/fuse.conf
(Uncomment last line saying user_allow_other)

8> Create a folder(bucket-mount) and Mount the S3 Bucket

sudo mkdir -p /s3/bucket-mount
sudo s3fs -o allow_other bucketName /s3/bucket-mount #(Change the "bucketName")
sudo s3fs -o iam_role='roleName' bucketName /s3/bucket-mount #(Change the "bucketName" and "roleName")
sudo s3fs -o iam_role='roleName' testerss /s3/bucket-mount #(Change the "bucketName" and "roleName")

cd /s3/bucket-mount
echo "The king never fails to win his destiny "> king.txt

(check in S3 the file would have been added with message as in "king.txt" file)


#Install AZ-cli if you want to migrate s3 to blob

AZ_REPO=$(lsb_release -cs)
echo "deb [arch=amd64] https://packages.microsoft.com/repos/azure-cli/ $AZ_REPO main" | \
    sudo tee /etc/apt/sources.list.d/azure-cli.list

curl -L https://packages.microsoft.com/keys/microsoft.asc | sudo apt-key add -

sudo apt-get install apt-transport-https

sudo apt-get update && sudo apt-get install azure-cli


#upload contents to azure blob
az storage blob upload-batch -d master -s /s3/bucket-mount/* --account-name sunbird1p7 --account-key azureBlobKey