RESTful API Server Example

This project showcases my vision on how the RESTful API server should be implemented.

The goals that were achived in this example:

• RESTful API server should be self-documented using OpenAPI (fka Swagger) specifications, so interactive documentation UI is in place;
• Authentication is handled with OAuth2 and using Resource Owner Password Credentials Grant (Password Flow) for first-party clients makes it usable not only for third-party "external" apps;
• Permissions are handled (and automaticaly documented);
• PATCH method can be handled accordingly to RFC 6902;
• Extensive testing with good code coverage.

I had to patch Flask-RESTplus (see flask_restplus_patched folder), so it can handle Marshmallow schemas and Webargs arguments.

Here is how it looks at this point of time (live demo):

Project Structure

Root folder

Folders:

• app - This RESTful API Server example implementation is here.
• flask_restplus_patched - There are some patches for Flask-RESTPlus (read more in Patched Dependencies section).
• migrations - Database migrations are stored here (see invoke --list to learn available commands, and learn more about PyInvoke usage below).
• tasks - PyInvoke commands are implemented here.
• tests - These are pytest tests for this RESTful API Server example implementation.
• docs - It contains just images for the README, so you can safely ignore it.
• deploy - It contains some application stack examples.

Files:

• README.md
• config.py - This is a config file of this RESTful API Server example.
• conftest.py - A top-most pytest config file (it is empty, but it helps to have a proper PYTHON PATH).
• .coveragerc - Coverage.py (code coverage) config for code coverage reports.
• .travis.yml - Travis CI (automated continuous integration) config for automated testing.
• .pylintrc - Pylint config for code quality checking.
• Dockerfile - Docker config file which is used to build a Docker image running this RESTful API Server example.
• .dockerignore - Lists files and file masks of the files which should be ignored while Docker build process.
• .gitignore - Lists files and file masks of the files which should not be added to git repository.
• LICENSE - MIT License, i.e. you are free to do whatever is needed with the given code with no limits.

Application Structure

app/
├── requirements.txt
├── __init__.py
├── extensions
│   └── __init__.py
└── modules
    ├── __init__.py
    ├── api
    │   └── __init__.py
    ├── auth
    │   ├── __init__.py
    │   ├── models.py
    │   ├── parameters.py
    │   └── views.py
    ├── users
    │   ├── __init__.py
    │   ├── models.py
    │   ├── parameters.py
    │   ├── permissions.py
    │   ├── resources.py
    │   └── schemas.py
    └── teams
        ├── __init__.py
        ├── models.py
        ├── parameters.py
        ├── resources.py
        └── schemas.py

• app/requirements.txt - The list of Python (PyPi) requirements.
• app/__init__.py - The entrypoint to this RESTful API Server example application (Flask application is created here).
• app/extensions - All extensions (e.g. SQLAlchemy, OAuth2) are initialized here and can be used in the application by importing as, for example, from app.extensions import db.
• app/modules - All endpoints are expected to be implemented here in logicaly separated modules. It is up to you how to draw the line to separate concerns (e.g. you can implement a monolith blog module, or split it into topics+comments modules).

Module Structure

Once you added a module name into config.ENABLED_MODULES, it is required to have your_module.init_app(app, **kwargs) function. Everything else is completely optional. Thus, here is the required minimum:

your_module/
└── __init__.py

, where __init__.py will look like this:

def init_app(app, **kwargs):
    pass

In this example, however, init_app imports resources and registeres api (an instance of (patched) flask_restplus.Namespace). Learn more about the "big picture" in the next section.

Where to start reading the code?

The easiest way to start the application is by using PyInvoke command app.run implemented in tasks/app/run.py:

$ invoke app.run

The command creates an application by running app/__init__.py:create_app() function, which in its turn:

1. loads an application config;
2. initializes extensions: app/extensions/__init__.py:init_app();
3. initializes modules: app/modules/__init__.py:init_app().

Modules initialization calls init_app() in every enabled module (listed in config.ENABLED_MODULES).

Let's take teams module as an example to look further. app/modules/teams/__init__.py:init_app() imports and registers api instance of (patched) flask_restplus.Namespace from .resources. Flask-RESTPlus Namespace is designed to provide similar functionality as Flask Blueprint.

api.route() is used to bind a resource (classes inherited from flask_restplus.Resource) to a specific route.

Lastly, every Resource should have methods which are lowercased HTTP method names (i.e. .get(), .post(), etc). This is where users' requests end up.

Dependencies

Project Dependencies

• Python 2.7, 3.3+ / pypy2 (2.5.0)
• flask-restplus (+ flask)
• sqlalchemy (+ flask-sqlalchemy) - Database ORM.
• sqlalchemy-utils - for nice custom fields (e.g., PasswordField).
• alembic - for DB migrations.
• marshmallow (+ marshmallow-sqlalchemy, flask-marshmallow) - for schema definitions. (supported by the patched Flask-RESTplus)
• webargs - for parameters (input arguments). (supported by the patched Flask-RESTplus)
• apispec - for marshmallow and webargs introspection. (integrated into the patched Flask-RESTplus)
• oauthlib (+ flask-oauthlib) - for authentication.
• flask-login - for current_user integration only.
• bcrypt - for password hashing (used as a backend by sqlalchemy-utils.PasswordField).
• permission - for authorization.
• Swagger-UI - for interactive RESTful API documentation.

Build Dependencies

I use pyinvoke with custom tasks to maintain easy and nice command-line interface. Thus, it is required to have invoke Python package installed, and optionally you may want to install colorlog, so your life become colorful.

Patched Dependencies

• flask-restplus is patched to handle marshmallow schemas and webargs input parameters (GH #9).
• swagger-ui (the bundle is automatically downloaded on the first run) just includes a pull-request to support Resource Owner Password Credentials Grant OAuth2 (aka Password Flow) (PR #1853).

Installation

Using Docker

It is very easy to start exploring the example using Docker:

$ docker run -it --rm --publish 5000:5000 frolvlad/flask-restplus-server-example

From sources

Clone the Project

$ git clone https://github.com/frol/flask-restplus-server-example.git

Setup Environment

It is recommended to use virtualenv or Anaconda/Miniconda to manage Python dependencies. Please, learn details yourself.

You will need invoke package to work with everything related to this project.

$ pip install -r tasks/requirements.txt

Run Server

NOTE: All dependencies and database migrations will be automatically handled, so go ahead and turn the server ON! (Read more details on this in Tips section)

$ invoke app.run

Quickstart

Open online interactive API documentation: http://127.0.0.1:5000/api/v1/

Autogenerated swagger config is always available from http://127.0.0.1:5000/api/v1/swagger.json

example.db (SQLite) includes 2 users:

• Admin user root with password q
• Regular user user with password w

NOTE: Use On/Off switch in documentation to sign in.

Authentication Details

This example server features OAuth2 Authentication protocol support, but don't be afraid of it! If you learn it, OAuth2 will save you from a lot of troubles.

Authentication with Login and Password (Resource Owner Password Credentials Grant)

Here is how you authenticate with user login and password credentials using cURL:

$ curl 'http://127.0.0.1:5000/auth/oauth2/token?grant_type=password&client_id=documentation&username=root&password=q'
{
	"token_type": "Bearer",
	"access_token": "oqvUpO4aKg5KgYK2EUY2HPsbOlAyEZ",
	"refresh_token": "3UTjLPlnomJPx5FvgsC2wS7GfVNrfH",
	"scope": "teams:read users:read users:write teams:write"
}

That is it! You grab the access_token and put it into Authorization header to request "protected" resources:

$ curl --header 'Authorization: Bearer oqvUpO4aKg5KgYK2EUY2HPsbOlAyEZ' 'http://127.0.0.1:5000/api/v1/users/me'
{
    "id": 1,
    "username": "root",
    "email": "root@localhost",
    "first_name": "",
    "middle_name": "",
    "last_name": "",
    "is_active": true,
    "is_regular_user": true,
    "is_admin": true,
    "created": "2016-10-20T14:00:35.912576+00:00",
    "updated": "2016-10-20T14:00:35.912602+00:00"
}

Authentication with Client ID and Secret (Client Credentials Grant)

Here is how you authenticate with user login and password credentials using cURL:

$ curl 'http://127.0.0.1:5000/auth/oauth2/token?grant_type=client_credentials' --user 'documentation:KQ()SWK)SQK)QWSKQW(SKQ)S(QWSQW(SJ*HQ&HQW*SQ*^SSQWSGQSG'
{
	"token_type": "Bearer",
	"access_token": "oqvUpO4aKg5KgYK2EUY2HPsbOlAyEZ",
	"scope": "teams:read users:read users:write teams:write"
}

The same way as in the previous section, you can grab the access_token and access protected resources.

Tips

Once you have invoke, you can learn all available commands related to this project from:

$ invoke --list

Learn more about each command with the following syntax:

$ invoke --help <task>

For example:

$ invoke --help app.run
Usage: inv[oke] [--core-opts] app.run [--options] [other tasks here ...]

Docstring:
  Run DDOTS RESTful API Server.

Options:
  -d, --[no-]development
  -h STRING, --host=STRING
  -i, --[no-]install-dependencies
  -p, --port
  -u, --[no-]upgrade-db

Use the following command to enter ipython shell (ipython must be installed):

$ invoke app.env.enter

app.run and app.env.enter tasks automatically prepare all dependencies (using pip install) and migrate database schema to the latest version.

Database schema migration is handled via app.db.* tasks group. The most common migration commands are app.db.upgrade (it is automatically run on app.run), and app.db.migrate (creates a new migration).

You can use better_exceptions package to enable detailed tracebacks. Just add better_exceptions to the app/requirements.txt and import better_exceptions in the app/__init__.py.

Useful Links

• "The big Picture" - short yet complete idea about how the modern apps should talk.
• "Please. Don't PATCH Like An Idiot."
• "A Concise RESTful API Design Guide"
• "Best Practices for Designing a Pragmatic RESTful API"
• "My take on RESTful authentication"
• "Is it normal design to completely decouple backend and frontend web applications and allow them to communicate with (JSON) REST API?"