Skip to content

Commit

Permalink
S3 storage backend and bug fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
TCB13 committed Oct 24, 2019
1 parent 02e2557 commit 13d3eeb
Show file tree
Hide file tree
Showing 6 changed files with 343 additions and 12 deletions.
25 changes: 24 additions & 1 deletion Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,8 @@ In order to use **ThunderTUS you must pick a storage backend**. Those are used t

- `FileSystem`: a quick to use and understand backend for simple projects that will append uploaded parts into a file stored at the path provided on it's constructor;
- `Redis`: useful in distributed scenarios (eg. your backend serves requests from multiple machines behind a load balancer), will store uploaded parts into a Redis database;
- `MongoDB`: also for distributed scenarios, will store uploaded parts inside a MongoDB GridFS bucket.
- `MongoDB`: also for distributed scenarios, will store uploaded parts inside a MongoDB GridFS bucket;
- `Amazon S3`: an implementation of Amazon's S3 protocol for distributed scenarios. Tested compatibility with DigitalOcean's Spaces.

You may also implement your own storage backend by extending the `StorageBackend` class and/or implementing the `StorageInterface` interface.

Expand Down Expand Up @@ -108,6 +109,28 @@ $server->handle();
`````
You may later retrieve the finished upload as described above at the basic usage section.

### S3 Storage Backend
````php
$server = new \ThunderTUS\Server($request, $response);

$client = new S3Client([
"version" => "latest",
"region" => "...",
"endpoint" => "...",
"credentials" => [
"key" => "--key--",
"secret" => "--secret---",
],
]);
$backend = new S3($client, $settingsS3->bucket, $prefix);
$server->setStorageBackend($backend);

$server->setUploadMaxFileSize(50000);
$server->setApiPath("/tus");
$server->handle();
`````
You may later retrieve the finished upload as described above at the basic usage section.

## ThunderTUS & Dependency Injection

ThunderTUS was designed to be integrated into dependency injection systems / containers.
Expand Down
13 changes: 6 additions & 7 deletions src/Server.php
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ public function getStorageBackend()

/**
* Completes an upload and fetches the finished file from the backend storage.
* This method abstracts backend storage file retrivel in a way that the user doen't
* This method abstracts backend storage file retrieval in a way that the user doesn't
* need to know what backend storage is being used at all times.
* This is useful when the TUS Server is provided by some kind of Service Provider in a
* dependency injection context.
Expand All @@ -100,9 +100,9 @@ public function completeAndFetch(string $filename, string $destinationDirectory,

/**
* Completes an upload and returns the finished file in the form of a stream.
* Useful when you want to upload the file to another system without writting
* Useful when you want to upload the file to another system without writing
* it to the disk most of the time.
* This method uses PHP's tmp stream to merge the file parts. Ajust it accordingly.
* This method uses PHP's tmp stream to merge the file parts. Adjust it accordingly.
*
* @param string $filename Name of your file
* @param bool $removeAfter Remove the temporary files after this operation
Expand All @@ -119,7 +119,7 @@ public function completeAndStream(string $filename, bool $removeAfter = true)
* same backend storage you're using for the temporary part upload.
* Useful when you want to keep the finished file in the same storage backend
* you're using for the temporary part upload.
* This method uses PHP's tmp stream to merge the file parts. Ajust it accordingly.
* This method uses PHP's tmp stream to merge the file parts. Adjust it accordingly.
*
* @param string $filename Name of your file
*
Expand Down Expand Up @@ -226,8 +226,8 @@ protected function handlePOST(): ResponseInterface
}

// Create an empty file to store the upload and save the cache container
$this->backend->create($this->file);
$this->backend->containerCreate($this->file, $cache);
$this->backend->create($this->file);

return $this->response->withStatus(201)
->withHeader("Location", $this->location);
Expand Down Expand Up @@ -340,12 +340,11 @@ protected function handlePATCH(): ResponseInterface
// Detect when the upload is complete
$cache = $this->backend->containerFetch($this->file);
if ($cache->length <= $localSize) {
// Remove the cache container, we don't need it anymore
$this->backend->containerDelete($this->file);
// Extension Thunder TUS CrossCheck: verify if the uploaded file is as expected or delete it
if ($this->extCrossCheck && $this->backend->supportsCrossCheck()) {
if (!$this->backend->crossCheck($this->file, $cache->checksum->algorithm, $cache->checksum->value)) {
$this->backend->delete($this->file);
$this->backend->containerDelete($this->file);
return $this->response->withStatus(410);
}
}
Expand Down
5 changes: 4 additions & 1 deletion src/Store/FileSystem.php
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ public function completeAndFetch(string $name, string $destinationDirectory, boo
}

if ($removeAfter) {
return rename($this->uploadDir . $name, $destinationDirectory . $name);
$this->containerDelete($name);
return rename($this->uploadDir . $name, $destinationDirectory . $name);
} else {
return copy($this->uploadDir . $name, $destinationDirectory . $name);
}
Expand All @@ -88,6 +89,7 @@ public function completeAndStream(string $name, bool $removeAfter = true)
$final = fopen("php://temp", "r+");
stream_copy_to_stream($stream, $final);
fclose($stream);
$this->containerDelete($name);
return unlink($this->uploadDir . $name);
} else {
return $stream;
Expand All @@ -96,6 +98,7 @@ public function completeAndStream(string $name, bool $removeAfter = true)

public function complete(string $name): bool
{
$this->containerDelete($name);
return true;
}

Expand Down
14 changes: 11 additions & 3 deletions src/Store/MongoDB.php
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ public function completeAndFetch(string $name, string $destinationDirectory, boo
}
}

if ($removeAfter) {
$this->containerDelete($name);
}

fclose($file);
return true;
}
Expand All @@ -109,6 +113,10 @@ public function completeAndStream(string $name, bool $removeAfter = true)
}
}

if ($removeAfter) {
$this->containerDelete($name);
}

return $final;
}

Expand All @@ -129,11 +137,11 @@ public function complete(string $name): bool
}
fclose($stream);
// Delete part from mongodb
if ($removeAfter) {
$this->bucket->delete($part);
}
$this->bucket->delete($part);
}

$this->containerDelete($name);

// We now have a final tmp with the entrie file upload it to mongodb
rewind($final);
$this->bucket->uploadFromStream($name, $final);
Expand Down
3 changes: 3 additions & 0 deletions src/Store/Redis.php
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ public function completeAndFetch(string $name, string $destinationDirectory, boo

if ($removeAfter) {
$this->delete($name);
$this->containerDelete($name);
}
return true;
}
Expand All @@ -95,13 +96,15 @@ public function completeAndStream(string $name, bool $removeAfter = true)

if ($removeAfter) {
$this->delete($name);
$this->containerDelete($name);
}

return $stream;
}

public function complete(string $name): bool
{
$this->containerDelete($name);
return true;
}

Expand Down
Loading

0 comments on commit 13d3eeb

Please sign in to comment.