conda env create -f environment_xcompat.yml
conda activate parallex
brew tap mongodb/brew
brew install [email protected]
brew services start [email protected]
mongosh
brew install rabbitmq
brew services start rabbitmq
brew services stop rabbitmq
brew services restart rabbitmq
chmod +x rmq_install.sh
./rmq_install.sh
sudo systemctl start rabbitmq-server
We need to make a user with username test and password test so that remote users can reach the RabbitMQ server. guest:guest is forbidden from accessing remotely.
On linux this can be done using the rmq_setup.sh script.
rabbitmq-plugins enable rabbitmq_management
rabbitmqctl list_queues
rabbitmqadmin get queue=<name> count=<int>
You should not be developing on windows :/
- Create account at https://ipinfo.io/signup
- Go to token tab on left, and copy the token.
- Go to repository and create a folder called "access_tokens" in root directory.
- Then create a file called "ipinfo" and paste this access token from step 2 into the file.
Build Required Files/Protos Run build.sh from Parallex/ directory
Run Command Node Run start_command_node.sh from command directory
Run Provider Daemon
Inject env variable PARALLEX_PROVIDER_ID to be consistent with backend provider._id in model.
Run run_daemon.sh from provider directory.
SECRET_KEY=[key] node index.js
$env:SECRET_KEY='abcde'; node index.js
set SECRET_KEY=abcde && node index.js
Note: you must have the Parallex.pem ssh key in your root directory of this repoistory locally.
To install dependencies (RabbitMQ, MongoDB, git, conda, etc) when deploying for the first time
python setup_cluster.py -i
To simply redploy with updated code
python setup_cluster.py
conda env export > environment_xcompat.yml
Run from a dir with Python files to format all Python files according to google style guide
yapf --in-place --recursive --style="{based_on_style: google}" *.py
Ray.io For job definition and paralellization https://www.ray.io
gRPC For node communication https://grpc.io
ident.me For public IP address https://api.ident.me
psutil For system metrics https://psutil.readthedocs.io/en/latest/#
ipinfo For IP geolocation https://ipinfo.io
aio-pika For asynchronous RabbitMQ https://aio-pika.readthedocs.io/en/latest/
RabbitMQ For the distributed message broker https://www.rabbitmq.com