Skip to content

Latest commit

 

History

History
157 lines (96 loc) · 4.7 KB

README.md

File metadata and controls

157 lines (96 loc) · 4.7 KB
Raw

Wild-Storage

Wild-Storage lets you use YouTube as a Infinite Cloud Storage. It embeds any files(Images, Videos, Documents, etc) into a video. The video can be uploaded to YouTube and the file can be retrieved from the video. A expansion & compression technique is used to defeat YouTube's compression algorithm. The file can be retrieved from the video without any loss of data.

Demo Video (YouTube Link)

ezgif com-gif-maker

Virtual Environment Set Up ❌ (... Not Required Now)

Create Virtual Environment

python3 -m venv env

Activate Virtual Environment

source env/bin/activate

Install Required Packages

pip install -r requirements.txt

Compile C++ Library & Generate Python Bindings ❌ (... Not Required Now)

g++ -fPIC -shared DataProccessing/cLibs/helper.cpp -o DataProccessing/cLibs/c_lib.so

Wild-Storage CLI App ✅

Make the script executable

chmod +x cli_app

Run CLI App

Make sure GCC compiler, Python 3, pip, Python venv is installed.
Now open terminal in the project directory and run the following command to complete the necessary setup and run the app.

./cli_app

How to Use? ✅

Wait for the app to start.

Create video from files

  • Copy all your files into \data\input directory
  • Select Encode file to video option on the app
  • Wait for the process to complete
  • Generated video (op.mp4) & Metadata (metadata.json) will be available in \data\generated directory
  • Upload the video to YouTube and copy-paste the contents of metadata.json in description of the video
  • Make sure the uploaded video is public / unlisted

Retrieve files from video

  • Select Decode file from video option on the app
  • Enter the youtube video url (Make sure the uploaded video is public / unlisted)
  • Wait for the process to complete
  • The files will be available in \data\retrieved directory

Cleaning up

  • Select Clean option on the app to clean data directory

Reload Time

  • Select Reload time option to set the time to wait before reloading the app

Exit

  • Select Exit option on the app to exit

Use Dockerized CLI App ✅ ✅

You just need to have Docker installed on your system.

Build Docker Image

docker build -t wild-storage .

Run Docker Container

docker run -it wild-storage

Run Dockerized CLI App

./cli_app

Copy Input Files from Host to Container

You can copy files from host to container using the following command. You can copy folders and files.

docker cp <host_path> <container_id>:app/data/input

Copy Output Files from Container to Host

The video and metadata files can be copied from container to host using the following command.

docker cp <container_id>:app/data/generated <host_path>

Copy Retrieved Files from Container to Host

The retrieved files can be copied from container to host using the following command.

docker cp <container_id>:app/data/retrieved <host_path>

DataProcessing Working

Embed File Into Video

Retrieve File From Video

Expansion And Compression

Python Wrapped C++ Library

Wild-Storage was a complete Python Implementation. Beign a very slow interpreted language it was taking too much time to process files. Later I moved some heavy task codebase to C++ and wrapped those in Python using ctypes. This helped to process the files ~3x faster.

Features

  • Parallel Processing using multiple CPU cores is supported now.
  • C++ Library is used for heavy tasks to make the process faster.

Credits

This project is inspired by Infinite-Storage-Glitch which is a Rust implementation. I liked the concept and tried to implement in my favourite language Python. The working of both may not be same.