Rockstor is free and open source software and we are proud of our budding developer and user community. Anyone can contribute in a number of different ways.
There is a lot you can do other than coding that is essential to the project. First, please join our community discussion forum. It's the central location where all discussions take place. If you need help, want to share an idea, suggest a new feature etc.. the forum is a great place to start. Being active on the forum is very helpful to the project and the community as a whole.
Contributions such as testing and reporting bugs are tremendously helpful and greatly appreciated. Please see our :ref:`update_channels` section and specifically the :ref:`testing_channel` sub section for information on how to track the latest fixes and features.
You can also contribute to the documentation of the project, which in many cases is crucial and often lags behind. So documentation contributions are also very much appreciated. For more information go to :ref:`contributedocs`
Finally Rockstor's continued development and distribution depends on all those choosing to subscribe to the :ref:`stable_channel` updates where all the :ref:`testing_channel` work end up.
Developing Rockstor to deliver on its unique value proposition of being a Smart, Powerful, and Open storage platform is a major effort. If you are passionate about Open Source and Storage like us, you are in the right company. We welcome you to join our community. Please begin by joining our community discussion forum.
Our development process is simple and straight forward. There is currently one main repository called rockstor-core on Github. Begin by forking it.
We use the wonderful Git for source code management and Rockstor on Github for issue tracking. We'll assume you have basic proficiency with Git and are familiar with it using a text editor or IDE of your choice. Emacs, Vim, Eclipse and PyCharm are some recommendations. We'll further assume that you have your laptop ready with git and an editor installed.
After you've forked the rockstor-core repo, clone the fork onto your laptop. The high level flow of a new contribution is as follows.
1. Search for an existing issue or start a new issue using the Issue Tracker.
2. Make code changes .
3. Submit your changes in a single pull request.
4. Use the issue tracker for discussions as necessary .
5. Close the issue when your pull request is merged .
Let's walk through this process in detail.
Since we rely on github services, you need to create a profile on github.com.
Go to rockstor-core repo and click on the "Fork" button. This will fork the repository into your profile which serves as your private git remote called origin. The next few git steps are demonstrated on a Linux terminal. They work the same on a Mac too. Your IDE may have ways to completely avoid the terminal, but using the terminal makes you look cool.
Your development machine can be a Linux system or a Mac. In my case, it's a Lenovo laptop running Ubuntu. There are other contributors using Mac, so this process works the same in both cases.
Clone your fork of rockstor-core repo onto your local development environment. You can do this in any directory, but chances are you have a project directory appropriate for this. Let's assume you have a projects directory in your home(~/projects) for the repo.
[you@your_laptop ~/projects]# git clone [email protected]:your_github_username/rockstor-core.git
The above command creates a local rockstor-core git repo in a new directory by the same name(rockstor-core). Change into it:
[you@your_laptop ~/projects]# cd rockstor-core
Configure this new git repo with your name and email address. This is required to accurately record collaboration:
[you@your_laptop rockstor-core]# git config user.name "Firstname Lastname" [you@your_laptop rockstor-core]# git config user.email your_email_address
Add a remote called upstream to periodically rebase your local repository with changes in the upstream made by other contributors:
[you@your_laptop rockstor-core]# git remote add upstream https://github.com/rockstor/rockstor-core.git
We'll assume you have identified an issue(eg: #1234) from the github issue tracker to work on. First, start with the latest code by rebasing your local repo's master branch with the upstream:
[you@your_laptop rockstor-core]# git checkout master [you@your_laptop rockstor-core]# git pull --rebase upstream master
Checkout a new/separate branch for your issue. For example:
[you@your_laptop rockstor-core]# git checkout -b issue#1234_brief_label
You can then start making changes in this branch.
We strongly encourage you to commit changes a certain way to help other developers and keep the merge process smooth. As a guiding principle, separate your changes into one or more logically independent commits.
We request that you divide a commit message into three parts. Start the message with a single line summary, about 70 characters in length. Add a blank line after that. If you want to add more than a summary to your commit message, describe the change in more detail in plain text format where each line is no more than 80 characters. This description should be in present tense. Below is a fictional example:
foobar functionality for rockstor Add a new file to implement the algorithm called recursive transaction launcher to generate transactional foobars recursively during runtime based on dependency tree of foos and bars. # Please enter the commit message for your changes. Lines starting # with '#' will be ignored, and an empty message aborts the commit. # On branch issue#1234_test # Changes to be committed: # (use "git reset HEAD <file>..." to unstage) # # new file: foobar.py #
If you'd like credit for your patch or if you are a frequent contributor, you should add your name to the rockstor-core AUTHORS file.
You need a Virtual Machine (VM) to build and test your changes. An easy solution is to create a Rockstor VM using either Oracle's VirtualBox or if you are using a Linux desktop then Virtual Machine Manager is also an option. You can find a VirtualBox Rockstor install demo on our YouTube channel and a :ref:`kvmsetup` in our documentation. It need not be a VM, but using a physical machine just for this purpose could be an overkill.
Note that when you first create the build VM, Rockstor rpm package will already be installed. The package files are located in /opt/rockstor. Further more, the Rockstor service should be running. We don't want that as it interferes with our development activity. Further down in this document, there is a buildout step. When that is run for the first time, the rpm package and it's effects are removed. Please note that this will destroy the existing Rockstor install and it's associated database details / settings.
In the following sections we use some terms in the commands; this is a short explanation of these terms:-
- laptop: This is your laptop or desktop computer.
- rockstor-core: This is a directory on your laptop containing your local rockstor-core repo. In my case, it's ~/Learnix/rockstor-core
- build_vm: IP address of your build VM. In my case, I use Virtualbox with host-only adapter and get an ip in 192.168.56.101-254 range.
- build_dir: The directory on the build VM where you like to copy the code to and build. In my case, I picked /opt/build/.
Transfer the code from your laptop to the build VM
[you@laptop ]# rsync -avz --exclude=.git /path/to/rockstor-core/ root@build_vm:/path/to/build_dir/
If you are building for the first time or like a clean build, execute the following command in your deploy directory on the VM
[root@build_vm ]# python /path/to/build_dir/bootstrap.py -c /path/to/build_dir/buildout.cfg
The next step is to build Rockstor with your new changes. This takes a long time for a clean build, but subsequent builds finish quickly
[root@build_vm ]# /path/to/build_dir/bin/buildout -N -c /path/to/build_dir/buildout.cfg
Once the buildout step above succeeds, rockstor services are automatically started and managed by systemd. You should now be able to login to the WebUI and verify your changes.
Changes fall into two categories. (1) Backend changes involving python coding and (2) Frontend changes involving javascript, html and css.
To test any change, you need to transfer files from your laptop to the VM:
[you@laptop ]# rsync -avz --exclude=.git /path/to/rockstor-core/ root@build_vm:/path/to/build_dir/
If you made any javascript, html or css changes, you need to collect static files with this command:
[root@build_vm ]# /path/to/build_dir/bin/buildout -c /path/to/build_dir/buildout.cfg install collectstatic
Then, refresh the browser to test new changes in the WebUI. It's best to have aliases setup for above commands and have it all integrated into your editor(Emacs anyone?). At the very least you should have multiple terminal tabs open; one for transferring files, one for running commands on the VM, and another for browsing through the logs.
When making backend changes, you may want to see debug logs and errors. Everything that you or any rockstor service logs goes into the following directory on your VM:
[root@build_vm ]# ls -l /path/to/build_dir/var/log total 280 -rw-r--r-- 1 root root 106912 Jun 23 19:49 gunicorn.log -rw-r--r-- 1 root root 119533 Jun 23 19:49 rockstor.log -rw-r--r-- 1 root root 25 Jun 23 19:19 supervisord_data-collector_stderr.log -rw-r--r-- 1 root root 0 Jun 23 15:33 supervisord_data-collector_stdout.log -rw-r--r-- 1 root root 0 Jun 23 15:33 supervisord_gunicorn_stderr.log -rw-r--r-- 1 root root 8 Jun 23 16:27 supervisord_gunicorn_stdout.log -rw-r--r-- 1 root root 27980 Jun 23 19:49 supervisord.log -rw-r--r-- 1 root root 0 Jun 23 15:33 supervisord_nginx_stderr.log -rw-r--r-- 1 root root 0 Jun 23 15:33 supervisord_nginx_stdout.log -rw-r--r-- 1 root root 0 Jun 23 15:33 supervisord_replication_stderr.log -rw-r--r-- 1 root root 8 Jun 23 15:33 supervisord_replication_stdout.log -rw-r--r-- 1 root root 0 Jun 23 15:33 supervisord_smart_manager_stderr.log -rw-r--r-- 1 root root 8 Jun 23 15:33 supervisord_smart_manager_stdout.log -rw-r--r-- 1 root root 0 Jun 23 15:33 supervisord_task-scheduler_stderr.log -rw-r--r-- 1 root root 8 Jun 23 15:33 supervisord_task-scheduler_stdout.log -rw-r--r-- 1 root root 0 Jun 23 15:33 supervisord_ztask-daemon_stderr.log -rw-r--r-- 1 root root 0 Jun 23 15:33 supervisord_ztask-daemon_stdout.log -rw-r--r-- 1 root root 996 Jun 23 19:49 ztask.log
rockstor.log should be the first place to look for errors or debug logs.
When making frontend changes, Developer Tools in Chrome/Firefox are your friends. You can inspect elements for html/css changes, log to the browser console from javascript code with console.log(), and use the debugger and step through javascript from your browser.
The frontend code uses third party javascript libraries such as jquery, bootstrap, d3 and many others. These are not part of the rockstor-core repository but are dynamically generated during the buildout step. They are placed in the below directory on your build VM:
[root@build_vm ]# ls /path/to/build_dir/static/js/lib backbone-0.9.2.js cocktail.js humanize.js jquery.flot.stack.js jquery.sparkline.min.js json2.js socket.io.min.js backbone.routefilter.min.js cron jquery-1.9.1.min.js jquery.flot.stackpercent.js jquery.tablesorter.js jsonform.js underscore-1.3.2.js bootstrap-datepicker.js cubism.v1.js jquery.flot.axislabels.js jquery.flot.time.js jquery.tools.min.js later.min.js bootstrap.js d3-tip.js jquery.flot.js jquery.flot.tooltip_0.5.js jquery.touch-punch.min.js moment.min.js bootstrap-timepicker.js d3.v3.min.js jquery.flot.navigate.js jquery-migrate-1.2.1.min.js jquery-ui.min.js prettycron.js chosen.jquery.js gentleSelect jquery.flot.resize.js jquery.shapeshift.js jquery.validate.js simple-slider.min.js
If you need to add a new library, place all of it's files in the lib directory(on the build VM, obviously) and continue your development process. After you open the pull request for rockstor-core repo, it's time to open a separate pull request for merging these libaries into upstream. This separate pull request must be opened for another repository named rockstor-jslibs, which mirrors the contents of the lib directory shown above. The fork and pull-request process is same as it is for this(rockstor-core repo) one.
We use PostgreSQL as the database backend for Rockstor. There are two databases, (1) storageadmin and (2) smart_manager. Depending on your issue you may need to add a Django model, delete one, or change fields of an existing model. After editing models you need to create a migration and apply it.
We used South to manage database migrations for a while, but since updating to Django 1.8, migrations are natively supported. The steps have changed only slightly. Generate the migration on your VM and copy the migration file back to your laptop and add it in git once you are satisfied.
For model changes in storageadmin application, create a migration file using
[root@build_vm ]# /path/to/build_dir/bin/django makemigrations storageadmin
The above command generates a migration file in /path/to/build_dir/src/rockstor/storageadmin/migrations/ Apply the migration with:
[root@build_vm ]# /path/to/build_dir/bin/django migrate storageadmin
For model changes in the smart_manager application, create a migration file using
[root@build_vm ]# /path/to/build_dir/bin/django makemigrations smart_manager
Run the migration with
[root@build_vm ]# /path/to/build_dir/bin/django migrate --database=smart_manager smart_manager
As you continue to work on an issue, commit and push changes to the issue branch of your fork. You can periodically push your changes to github with the following command:
[you@laptop ]# cd /path/to/rockstor-core; git push origin your_branch_name
When you finish work for the issue and are ready to submit, create a pull request by clicking on the "pull request" button on github. This notifies the maintainers of your changes. As a best practice only open one pull request per issue containing all relevant changes.
As you work on an issue in your feature/issue branch, you may have committed multiple times. Please squash all these commits into one at the very end. This will keep the master branch's history clean and makes it easier to revert, search for a change or track a regression.
Squashing commits into one is straight forward and most editors and IDEs with git support make it super easy to do so. If you've never done this before, this short how-to is helpful.
If you want to test and/or contribute starting from another user fork, you can add his/her fork (or single branch)
Adding another user forked repo to your remotes:
[you@laptop rockstor-core]# git remote add other_user_name [email protected]:other_user_name/rockstor-core.git
Fetching another user branch:
[you@laptop rockstor-core]# git fetch other_user_name remote_branch_name
After fetching other contributor branch you can checkout it and start your coding or have a complete new branch starting from it. Github pull requests then can be directly to Rockstor repo or other user branches