Skip to content

Commit

Permalink
Merge pull request #299 from apivideo/Add-GCS-migration-guide
Browse files Browse the repository at this point in the history
Add GCS migration guide
  • Loading branch information
szekelyzol authored Oct 13, 2023
2 parents 30b629d + da68932 commit d69fd78
Show file tree
Hide file tree
Showing 9 changed files with 40 additions and 174 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

This file was deleted.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
209 changes: 36 additions & 173 deletions templates/documentation/get-started/gcs-migration.md
Original file line number Diff line number Diff line change
@@ -1,213 +1,76 @@
---
title: "Amazon S3 migration guide"
slug: "aws-migration"
title: "Google Cloud Storage migration guide"
slug: "gcs-migration"
meta:
description: This page gets users started on how to migrate from Amazon S3 to api.video.
description: This page gets users started on how to migrate from Google Cloud Storage to api.video.
---

# Amazon S3 migration guide
# Google Cloud Storage migration guide

Planning to migrate api.video from Amazon S3? We got you covered!
Planning to migrate api.video from Google Cloud Storage? We got you covered!

There are two methods at your disposal that you can use to migrate all of your video content from S3 to api.video.
Using the api.video Import tool, it only takes a couple of clicks to migrate all of your video content from Google Cloud Storage to api.video. This simple tool only requires read access to your Google Cloud Storage bucket, and that you authenticate yourself with your api.video account.

- Migrate with api.video Import tool
- Migrate using your own script
Check out the [Import Tool](https://import.api.video/) to get started.

## What's the cost?

We understand that when you want to move to a different provider, it takes effort and development resources. You also want to make sure that it is cost-efficient, especially if you are moving to api.video to save costs.

api.video gives you the ability to **migrate for free** and avoid paying anything for encoding!

## api.video Import Tool
## Granting read access to your GCS bucket

With api.video, you can use our in-house import tool to migrate all of your videos in a short time from the popular hosting and video provider.
To access your content on Google Cloud Storage, api.video needs read access. This means that you need to grant the Import Tool's external service account read access to GCS objects.

The simple tool will require you to input several parameters from the provider you are leaving and from api.video.
You can grant this access in two quick steps:

Check out the [Import Tool](https://import.api.video/) now.
* select the project and bucket that you want to migrate
* grant read access to the Import Tool

### How to get the AWS Access Key and Account Secret
### 1. Select your project and bucket

To access AWS, you will need to sign up for an AWS account.
* Go to the [Google Cloud Console](https://console.cloud.google.com/), log in to your account and select the project you want to use.
* Navigate to Cloud Storage > Buckets.

Access keys consist of an access key ID and secret access key, which are used to sign programmatic requests that you make to AWS. If you don't have access keys, you can create them by using the [IAM console](https://console.aws.amazon.com/iam/).
![Showing the Buckets menu in Google Cloud Storage](/_assets/get-started/migration-guide/gcs-migration-1.png)

We recommend that you use IAM access keys instead of AWS root account access keys. IAM lets you securely control access to AWS services and resources in your AWS account.
* Select the bucket that you want to migrate.

{% capture content %}
![Showing the list of buckets in Google Cloud Storage](/_assets/get-started/migration-guide/gcs-migration-2.png)

To create access keys, you must have permission to perform the required IAM actions. For more information, see Granting IAM User Permission to Manage Password Policy and Credentials in the IAM User Guide.
* Click on the "Permissions" tab and click on "Grant access". This enables you to add a new member to the bucket.

{% endcapture %}
![Showing the Grant access button in Google Cloud Storage](/_assets/get-started/migration-guide/gcs-migration-3.png)

{% include "_partials/callout.html" kind: "info", content: content %}
### 2. Grant read access to the Import Tool

**To get your access key ID and secret access key:**
- Open the [IAM console](https://console.aws.amazon.com/iam/).
- On the navigation menu, choose Users.
- Choose your IAM user name (not the check box).
- Open the Security credentials tab, and then choose Create access key.
- To see the new access key, choose Show. Your credentials resemble the following:
Once you have navigated to the bucket that you want to migrate, you need to add the Import Tool as a principal and assign Storage Object Viewer role:

```
![Showing the Grant access dialog box Google Cloud Storage](/_assets/get-started/migration-guide/gcs-migration-3.png)

Access key ID: AKIAIOSFODNN7EXAMPLE
* In the "Grant access" dialog box, enter `storage-service-account@video-import-tool.iam.gserviceaccount.com` in the "New principals" input field.
* In the "Role" dropdown menu, select "Storage Object Viewer".
* Save your changes.

Secret access key: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
The Import Tool's service account now has read access to the Google Cloud Storage bucket you selected. You can now import videos from this bucket.

```
## Import the videos

- To download the key pair, choose Download .csv file. Store the .csv file with keys in a secure location.

{% capture content %}

Keep the keys confidential to protect your AWS account, and never email them. Do not share them outside your organization, even if an inquiry appears to come from AWS or Amazon.com. No one who legitimately represents Amazon will ever ask you for your secret key.

{% endcapture %}

{% include "_partials/callout.html" kind: "warning", content: content %}

You can retrieve the secret access key only when you initially create the key pair. Like a password, **you can't retrieve it later.** If you lose it, you must create a new key pair.

### Importing the videos

Once you have all the credentials, you can proceed with the migration.
Once you have granted all necessary access, you can start the migration.

1. Navigate to the api.video [Import Tool](https://import.api.video/).

2. Select Amazon S3 from the list.

3. Autherize with api.video with your account. This process will link your api.video workspace to the Import tool.

4. Select the project that you would like to import the video.

5. Next, click on Sign in to Amazon S3 and fill in the AWS Access Key and AWS Secret Key you got from Amazon S3.

6. Select the S3 bucket you would like to import.

7. Now you can proceed with the import.

The process will show you the status of each video and the encoding status.

## Importing with a script

Another way to import your videos is to use your own script and leverage one of the api.video client libraries with the provider's client libraries or API (if there are any).

### Setting up

What you'll need to run the script:

- api.video API key
- AWS credentials
- Node.js

### Getting the api.video API key

In order to get your API key. Navigate to the [api.video dashboard](https://dashboard.api.video/videos) (if you are not already signed up, sign up with api.video) and continue to copy your API key from the **Overview page** or the **API Keys page**

### What the script will do?

The below script is designed to run through all of the files in a specific S3 bucket, generate a signed URL, and pass it on to api.video as the source. api.video will then take the URL and ingest the video directly from S3.

Make sure that the script is modified according to your needs.

### Preparation

If you are new to Node.js, here's how you can run this script.

Make sure that you have Node.js installed, if it doesn't exist on your machine, you can install it by following this [guide](https://kinsta.com/blog/how-to-install-node-js/).

Now that you have Node.js installed, we will have to create the path of the script and install the dependencies.

1. Create the folder for the script:
```shell
$ mkdir s3-apivideo-migration
```

2. Navigate to the folder
```shell
$ cd s3-apivideo-migration
```

3. Initialize `npm` in order to create the package.json file for the module dependacies
```shell
$ npm init
```

4. Install the modules that we will need to run
```shell
$ npm install @aws-sdk/client-s3 --save
$ npm install @aws-sdk/s3-request-presigner --save
$ npm install @api.video/nodejs-client --save
```

5. Now create an index file that you would like the script to exist in:
```shell
$ touch index.js
```

6. Copy over the below script and replace the following parameters with your own: `videBucket`, `region`, `accessKeyId`, `secrectAccessKey`, `apivideoCreds`

```javascript
import { S3Client, ListObjectsV2Command, GetObjectCommand, S3 } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import ApiVideoClient from '@api.video/nodejs-client';

// your S3 bucket name
const videoBucket = "videos-cold-storage-api-video";
// your S3 bucket region
const region = 'eu-north-1';
// your S3 credentails
const s3creds = {
accessKeyId: 'XXXXXX',
secretAccessKey: 'XXXXXXX',
}
// api.video api key
const apivideoCreds = 'XXXXXXX';
2. Select **Google Cloud Storage** from the list.

3. Log into your api.video account. This process links your api.video workspace to the Import tool.

const apivideoClient = new ApiVideoClient({ apiKey: apivideoCreds });
4. If you have multiple api.video projects in your workspace, select the project where you want to migrate your content to.

const client = new S3Client({ region,
credentials: s3creds
});
5. Next, enter the name of the Google Cloud Storage bucket that you want to migrate.

new S3({ region,
credentials: s3creds
});
6. Next, the tool retrieves the available videos from Google Cloud Storage. Select the videos you would like to import.

const bucketName = {
Bucket: videoBucket,
};
const listBucketObjectsCommand = new ListObjectsV2Command(bucketName);
7. Select the videos you want to migrate, and start the import process.

try {
const listObjects = await client.send(listBucketObjectsCommand);
if (listObjects?.$metadata?.httpStatusCode === 200 && listObjects?.Contents.length > 0) {
for (let i in listObjects.Contents) {
const objectKey = listObjects.Contents[i].Key;
bucketName.Key = objectKey;
const objectDetailsCommand = new GetObjectCommand(bucketName);
const objectDetails = await client.send(objectDetailsCommand);
if(/video/.test(objectDetails.ContentType)) {
const signedUrl = await getSignedUrl(client, objectDetailsCommand, { expiresIn: 3600 });
const videoCreationPayload = {
title: objectKey,
source: signedUrl
};
await apivideoClient.videos.create(videoCreationPayload);
console.log("upload complete")
}
}
}
} catch (e) {
console.error(e);
}
```
7. Save the file and run the script by:
```shell
$ node index.js
```
The process displays the upload status and encoding status for each video.
2 changes: 2 additions & 0 deletions templates/documentation/get-started/migration-guides.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ Moving to a different provider takes time, effort, and development resources. ap

{% include "_partials/hagrid-item.md" title: "Amazon", image: "/_assets/get-started/migration-guide/Amazon-S3-Logo.svg", subtitle: "S3", link: "./aws-migration.md" %}

{% include "_partials/hagrid-item.md" title: "Google", image: "/_assets/get-started/migration-guide/Google-Storage-Logo.png", subtitle: "Cloud Storage", link: "./gcs-migration.md" %}

{% include "_partials/hagrid-item.md" title: "Vimeo", image: "/_assets/get-started/migration-guide/vimeo.png", subtitle: "", link: "./vimeo-migration.md" %}

{% include "_partials/hagrid-item.md" title: "Wistia", image: "/_assets/get-started/migration-guide/wistia.png", subtitle: "", link: "./wistia-migration.md" %}
Expand Down
2 changes: 2 additions & 0 deletions templates/documentation/navigation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
href: /get-started/azure-migration.md
- label: Amazon S3
href: /get-started/aws-migration.md
- label: Google Cloud Storage
href: /get-started/gcs-migration.md
- label: Vimeo
href: /get-started/vimeo-migration.md
- label: Wistia
Expand Down

0 comments on commit d69fd78

Please sign in to comment.