Upgrade S3 example to use the newest SDK.

[janhesters]

Recently, the AWS SDK v3 has been released and its now the recommended way to interact with AWS.

I tried to upgrade the S3 example myself, like this:

// ... in s3.server.ts
const {
  AWS_ACCESS_KEY_ID,
  AWS_SECRET_ACCESS_KEY,
  S3_STORAGE_BUCKET,
  S3_STORAGE_REGION,
} = process.env;

invariant(AWS_ACCESS_KEY_ID, 'AWS_ACCESS_KEY_ID is required');
invariant(AWS_SECRET_ACCESS_KEY, 'AWS_SECRET_ACCESS_KEY is required');
invariant(S3_STORAGE_BUCKET, 'S3_STORAGE_BUCKET is required');
invariant(S3_STORAGE_REGION, 'S3_STORAGE_REGION is required');

// ... rest stays same

export const uploadStream = ({
  Key,
}: Pick<PutObjectCommandInput, 'Key'>) => {
  const s3Client = new S3Client({ region: S3_STORAGE_REGION });
  const pass = new PassThrough();
  return {
    writeStream: pass,
    promise: s3Client.send(
      new PutObjectCommand({
        Body: pass,
        Bucket: S3_STORAGE_BUCKET,
        Key,
      }),
    ),
  };
};

// ... rest stays same

But it doesn't work because an error is thrown:

error NotImplemented: A header you provided implies functionality that is not implemented
    at throwDefaultError (/Users/dev/remix-app/node_modules/@aws-sdk/smithy-client/dist-cjs/default-error-handler.js:8:22)
    at deserializeAws_restXmlPutObjectCommandError (/Users/dev/remix-app/node_modules/@aws-sdk/client-s3/dist-cjs/protocols/Aws_restXml.js:5782:43)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at /Users/dev/remix-app/node_modules/@aws-sdk/middleware-serde/dist-cjs/deserializerMiddleware.js:7:24
    at /Users/dev/remix-app/node_modules/@aws-sdk/middleware-signing/dist-cjs/middleware.js:14:20
    at /Users/dev/remix-app/node_modules/@aws-sdk/middleware-retry/dist-cjs/retryMiddleware.js:27:46
    at /Users/dev/remix-app/node_modules/@aws-sdk/middleware-flexible-checksums/dist-cjs/flexibleChecksumsMiddleware.js:58:20
    at /Users/dev/remix-app/node_modules/@aws-sdk/middleware-logger/dist-cjs/loggerMiddleware.js:5:22
    at uploadStreamToS3 (/Users/dev/remix-app/app/features/recording/s3-upload-handler.server.ts:106:11) {
  '$fault': 'client',
  '$metadata': {
    httpStatusCode: 501,
    requestId: 'NSK9FW5CE2TW3NAN',
    extendedRequestId: '0zWgQIB0xmMtUuIYThFgN1MdD8hXAj/VcjHUaabT3RiB7jmh6hlTNfsDbxq9M6EMwp5rgAlVfcY=',
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  },
  Code: 'NotImplemented',
  Header: 'Transfer-Encoding',
  RequestId: 'NSK9FW5CE2TW3NAN',
  HostId: '0zWgQIB0xmMtUuIYThFgN1MdD8hXAj/VcjHUaabT3RiB7jmh6hlTNfsDbxq9M6EMwp5rgAlVfcY='
}

by this line:

const file = await stream.promise;

in uploadStreamToS3().

I'm not skilled in S3, so I have no idea how to fix this.

[janhesters]

Okay, I fixed it like this, so you need to install @aws-sdk/lib-storage and @aws-sdk/client-s3:

import { DeleteObjectCommand, S3Client } from '@aws-sdk/client-s3';
import { Upload } from '@aws-sdk/lib-storage';

export const uploadStream = ({ Key }: Pick<PutObjectCommandInput, 'Key'>) => {
  const client = new S3Client({ region: S3_STORAGE_REGION });
  const pass = new PassThrough();
  return {
    writeStream: pass,
    promise: new Upload({
      client,
      params: {
        Body: pass,
        Bucket: S3_STORAGE_BUCKET,
        Key,
      },
    }).done(),
  };
};

Then you'll also have to grab the location, if the request was successful in uploadStreamToS3():

const file = await stream.promise;

if ('Location' in file) {
  return file.Location;
}

throw new Error('Upload to S3 aborted');

[janhesters]

Made a PR to add an example for the new SDK version here: #164

[github-actions]

This issue has been automatically marked stale because we haven't received a response from the original author in a while 🙈. This automation helps keep the issue tracker clean from issues that are not actionable. Please reach out if you have more information for us or you think this issue shouldn't be closed! 🙂 If you don't do so within 7 days, this issue will be automatically closed.

[github-actions]

This issue has been automatically closed because we didn't hear anything from the original author after the previous notice.