Cloudless Box is an opinionated Chef cookbook that allows to run one or more Ruby on Rails, Phoenix, Node, Meteor or Middleman applications on single server with databases, backup and more. It's targeted at VPS and dedicated servers that run CentOS Linux.
- Everyone who wants to have one or more Ruby on Rails, Phoenix, Node, Meteor, Middleman applications or static websites on single, blazing fast and secure web server without spending a dozen days configuring it and committing suicide on server or data failure.
- Everyone who got used to easy deployments, such as those on Heroku, but wants it faster, cheaper and under control. With Cloudless Box, deploying new app is a matter of adding few lines to server's JSON config plus you get similar env variables like DATABASE_URL.
- Everyone who is tired of configuring and executing deployment for each app separately (ie. Capistrano) - Cloudless Box can handle all deployments at once. Of course you can still opt for custom deployment strategies by omitting single configuration line.
- Everyone who don't know their way around Chef - Cloudless Box comes with complete setup guide, possible to follow by people unfamiliar with server cookery. You are assumed to have basic *NIX terminal skills and a shiny new server with CentOS on it.
- Nginx web server with Passenger for running Rack and Node apps.
- Ability to access apps with server's subdomains or with custom domains.
- Separate PostgreSQL, MongoDB and/or Redis databases for all apps.
- Automatic solution to backup all databases to S3 on daily basis.
- Per-app scalable workers with
Procfileand scaling support. - Per-app cron tasks with Whenever's expressive syntax.
- Preconfigured server essentials: SSH, IPTables and NTP.
- Preconfigured web development essentials: Git, Bower, ImageMagick and FFmpeg.
This section guides users unfamiliar with Chef through a complete process of setting up Cloudless Box on their own server conveniently managed by the Hosted Chef server. If you've already wrapped your head around Hosted Chef, Berkshelf and Knife, you can skip this section and just use this cookbook as usually.
First, visit the Get Chef page, download Chef Development Kit for your operating system and install it. Then, visit the Hosted Chef page and sign in to Chef Manage or register for a free account.
If you've just created new account, you'll have to authorize your computer with the Chef server. Visit Chef Manage and navigate to Administration » Organizations. There, you can download the Starter Kit. Do so. After downloading it, copy Chef credentials to your home directory:
mkdir ~/.chef
cp <starter-kit-path>/.chef/* ~/.chef/
You are now ready to work with your Hosted Chef server.
Next step is to upload Cloudless Box to your Hosted Chef server. Invoke the following:
git clone https://github.com/karolsluszniak/cloudless-box.git
cd cloudless-box
berks install
berks upload
Now you have Cloudless Box cookbook and all its dependencies ready on your Hosted Chef server.
Go back to Chef Manage page and create configuration for your server:
- Navigate to Policy » Environments and click Create.
- Name the environment and click Next.
- On Constraints tab, select cloudless-box and enter current version of Cloudless Box (you can find it at Policy » Cookbooks).
- Next is Default attributes tab. It's here where you write JSON that lists and configures your server's apps according to the Configuration section below.
You can leave attributes empty and start by bootstrapping Cloudless Box without any apps. It'll still take a while to install server essentials so you'll have the time to read the Configuration section and fill needed attributes.
Finally, you can bootstrap your server with the following command:
knife bootstrap <address> --ssh-user root --ssh-password <your-password> --sudo --use-sudo-password --node-name <node-name> --environment <env-name> --run-list 'recipe[cloudless-box]'
That's it! You can re-provision the server by logging into it and executing:
chef-client
This is usually done when your web apps were updated and they need re-deploy or when you have changed your server's environment attributes.
You can repeat this step for each server that you want to govern with Cloudless Box. You can manage up to 5 servers with a free Hosted Chef server.
It's recommended to disable root user after completing the bootstrap and using sudo-enabled account for future server visits. This step is not directly related to this cookbook, nor is it required, but it's strongly recommended for improving security.
In order to create new user, invoke the following:
adduser <myuser>
gpasswd -a <myuser> wheel
passwd <myuser>
Finish by entering your new user's password. Now you should be able to log into your server with new account. You can re-provision your server at any time using new account with:
sudo chef-client
Next, let's disable root user completely. Invoke the following:
passwd -d root
yum -y install nano
nano /etc/ssh/sshd_config
Move to the end of the file and add the following:
PermitRootLogin no
Hit Ctrl+O and Ctrl+X. Finish by restarting SSH:
service sshd restart
Cookbook is configured via attributes, usually set as JSON in server's environment attributes on the Chef Manage page. They may also be set as role default/override attributes or in any other fashion available in Chef.
After changing attributes, re-run chef-client on your server in order to apply new ones. If per-app attributes won't have effect after running chef-client, you may have to create empty commit in order to force re-deployment.
A complete JSON configuration may look like this:
{
"cloudless-box": {
"applications": {
"my-rails-app": {
"layout": "rails",
"ruby": "2.2.2",
"postgresql": true,
"url": "my-rails-app.com",
"repository": "[email protected]:user/repo.git",
"env": {
"aws_s3_bucket": "(...)",
"aws_access_key_Id": "(...)",
"aws_secret_access_key": "(...)"
}
},
"my-meteor-app": {
"layout": "meteor",
"mongodb": true,
"repository": "[email protected]:user/repo.git"
},
"my-node-app": {
"layout": "node",
"bower": true,
"repository": "[email protected]:user/repo.git",
"env": {
"google_analytics_id": "(...)"
},
"nginx": {
"client_max_body_size": "2M"
}
}
},
"backup": {
"bucket": "(...)",
"access_key_id": "(...)",
"secret_access_key": "(...)"
},
"secret": "my-server-secret",
"gems": {
"sass": true
}
}
}Below, you'll find a complete list of attributes available for configuring applications and other aspects of the Cloudless Box cookbook.
You should list your applications in the node["cloudless-box"]["applications"] object. Object key specifies an application name, while object value may have the following per-app attributes:
| Attribute | Description |
|---|---|
asset_cmd |
specifies custom asset precompilation command or disables default one with false |
bower |
requests Bower support; if set, Bower will be available and bower install --production will be run on app's deployment |
env |
object with custom enviroment variables; if set, variables will be added to Bash profile, .env and Passenger (read more) |
layout |
specifies the application layout; can be one of static, rails, phoenix, node, meteor or middleman; defaults to static |
migration_cmd |
specifies custom database migration command or disables default one with false |
mongodb |
requests MongoDB database for the application; if set to true, MONGO_URL environment variable will become available |
nginx |
object with custom Nginx options; if set, options will be added to application's Nginx site configuration |
postgresql |
requests PostgreSQL database for the application; if set to true, DATABASE_URL environment variable will become available |
public |
allows to change public sub-directory within app directory; defaults to public; if set to blank string, root directory will be served |
redis |
requests Redis database for the application; if set to true, REDIS_URL environment variable will become available |
repository |
enables deployment from specified repo; if unset, app deployment will not be managed by Chef; read this if your repo is private |
repository_path |
uses specified path within repository as deployment root path; if unset, repository root will be used |
ruby |
requests specific Ruby version available for the application; if unset, default system Ruby will be available |
schedule |
allows to specify Whenever file or to disable the scheduling (with false); if unset, config/schedule.rb will be used |
secret |
sets additional secret component for generating SECRET_KEY_BASE in order to improve application security |
shared_dirs |
array of additional shared directories for on-deployment creation and symlinking; defaults to log pids system (plus per-layout additions) |
sticky_sessions |
enables sticky sessions Passenger setting; by default, it's enabled only for meteor layout; this comes useful when using WebSockets in an app |
symlinks |
object with custom symlinks for on-deployment linking; defaults to .env; allows to change or remove the defaults (with false) |
url |
string or array that sets custom domain(s) for the application; if unset, app will be available at <app-name>.domain.com subdomain |
workers |
allows to add and control per-app workers; if unset, one worker process per line in app's Procfile will be configured |
All attributes are optional and have sensible defaults. If none will be set, you'll end up with a system account for static, undeployed website without any database or addon on your server.
Packages like PostgreSQL, MongoDB, Redis, Ruby version manager, Meteor or Bower will not be installed on your system until any of your apps requests them. This will keep your system as lightweight as possible.
Removing applications from the node["cloudless-box"]["applications"] list will not remove them from your server. You'll have to do that manually by removing the deploy-<app-name> account along with all its files as well as Nginx site configuration and worker services.
In addition to application configuration, you can configure some aspects of Cloudless Box that are not related to any specific app. Following attributes are available:
| Attribute | Description |
|---|---|
backup |
object with bucket, access_key_id and secret_access_key keys that enables daily PostgreSQL and MongoDB database backup |
secret |
sets additional secret component for generating various secret hashes across the server in order to improve overall security |
gems |
object with list of optional, global gems needed on the server (the value is ignored for now) |
It may prove useful to know what Cloudless Box is doing under the hood for successful maintenance and customization of the server. Below, you'll find a more detailed description and some practical tips for your server's key functionalities.
Here's what happens during the deployment:
- Clone repository to a subdirectory in
releases. - Purge shared directories from release.
- Execute
bower install(if requested). - Execute application build sub-procedure.
- Create symlinks to shared directories and files.
- Execute application release sub-procedure.
- Create
currentsymlink pointing to new release. - Restart the webserver.
- Update cron table using Whenever (if configured).
- Update and restart worker services.
Here's what happens during the application build sub-procedure:
| Layout | Procedure |
|---|---|
meteor |
call meteor build, apply Node layout |
middleman |
call middleman build |
node |
link node_modules shared directory, call npm install |
phoenix |
call mix deps.get, mix compile, brunch build and mix phoenix.digest |
rails |
call bundle install, call rake assets:precompile |
Here's what happens during the application release sub-procedure:
| Layout | Procedure |
|---|---|
phoenix |
call mix ecto.migrate |
rails |
remove database.yml, call rake db:migrate |
If some of your applications are stored on private repositories, their initial deploy will fail due to SSH access denial. In such case, you'll see error messages but the whole chef-client run will finish with remaining tasks. Among others, it will still create system accounts and SSH keys for these applications. You can display SSH key for specific app with the following command:
sudo cat /home/deploy-<app-name>/.ssh/id_rsa.pub
Add this key as a deployment key in your app's repo settings on GitHub or BitBucket and re-provision with chef-client to finish the deployment.
A number of default environment variables will be set for you in each app's:
- Bash profile
shared/.envfile- Passenger settings
- worker services
Built-in variables include:
| Variable | Example | Notes |
|---|---|---|
| APP_ROOT | /home/deploy-<app-name/(...) |
|
| DATABASE_URL | postgres://(...) |
only if PostgreSQL was requested |
| HOME | /home/deploy-<app-name> |
|
| MONGO_URL | mongodb://(...) |
only if MongoDB was requested |
| NODE_ENV | production |
only for node layout |
| RAILS_ENV | production |
only for rails layout |
| REDIS_URL | redis://(...) |
only if Redis was requested |
| ROOT_URL | http://<app-name>.domain.com |
only for meteor layout |
| SECRET_KEY_BASE | 6984cb8190(...) |
only for rails layout |
You can add your own environment variables by listing them in the node["cloudless-box"]["applications"]["<app-name>"]["env"] object. Keys in this object will be automatically converted to upper case and your variables will be set in all places mentioned above.
See example for a more clear picture of setting environment variables.
In order to execute commands from your application instance, just type:
cd $APP_ROOT
# execute rake, npm or whatever
With this, here's how the whole procedure of logging in and executing rake task may look:
ssh <my-user>@<my-server-address>
sudo su - deploy-rails-app
cd $APP_ROOT
rake db:seed
Whenever allows to write cron jobs with a human-readable, clean syntax. Support for Whenever is already built into Cloudless Box, so all you have to do is add config/schedule.rb file to your application and its cron table will get updated at the end of every deployment.
You can change the schedule file name or disable Whenever completely by setting the schedule per-app attribute to a path string or false, respectively.
Cloudless Box will process your app's Procfile and create system services for processes listed in this file. Procfile should be located in your project's root and it should look like this:
sidekiq: bundle exec sidekiq
queue: bundle exec rake queue
You can also use the workers per-app attribute to further customize your workers. Here's what you can achieve with the workers attribute.
You can add workers that are not listed in Procfile like this:
"workers": {
"sidekiq": "bundle exec sidekiq",
"queue": {
"command": "bundle exec rake queue",
"scale": 3
}
}If process with the same name appears in the Procfile, then attributes override the command.
You can also remove unneded workers from the Procfile. This may be useful if your application includes a web worker that is not needed for Cloudless Box (built in support for Nginx and Passenger will cover that for you). Here's how to do that:
"workers": {
"web": false
}And of course, you can scale workers to run more than one instance. This can be achieved only via the attribute as Procfile doesn't have syntax for this. This can be achieved in two ways:
"workers": {
"queue1": 3,
"queue2": {
"scale": 5
}
}Worker settings will only be applied at the end of every app deployment. This means that if you've only modified the attributes and not Procfile, you may need to push an empty commit to trigger the re-deployment and re-scaling of your app. Here's how to do that:
git commit --allow-empty -m 'Trigger worker re-scale'
Alternatively, you can remove the current release link to trigger the re-deployment without polluting your repository:
cd $HOME
rm -f current
Remember that this will break your app's webserver until the next deployment finishes.