django-tanstack-starter

A starting point for new Python/React applications

Technology

The main pieces of technology used are Docker, Kubernetes, Django + Ninja, and TanStack Router + Query (and soon Start).

Other technologies used

Docker - Package and run software in containers
Kubernetes - Production-grade container orchestration
mypy - Static type checker for Python
Django - Python web framework
Django Ninja - Like FastAPI, but integrates seamlessly with the Django ORM
MJML - Framework that makes responsive email easy
Celery - Distributed task queue
Redis - In-memory database
- Mainly used as a message broker for Celery
- Can be used for caching
- ...
PostgreSQL - Versatile relational database
- Full text search support that might be sufficient, so that you don't need to introduce for example Elasticsearch
- NoSQL support with jsonb fields
- ...
TypeScript - JavaScript with syntax for types
React - Library for web and native user interfaces
TanStack Router - Modern and scalable routing for React applications
TanStack Query - Powerful asynchronous state management
Tailwind CSS - Rapidly build modern websites without ever leaving your HTML
Motion - A modern animation library for JavaScript and React
shadcn/ui - Build your component library
Motion-Primitives - UI kit to make beautiful, animated interfaces, faster
Storybook - Frontend workshop for building UI components, mainly used for documentation
Playwright - End-to-end testing framework
Vitest - Vite-native testing framework, used mainly for frontend unit testing
Sentry - Application monitoring software
Snyk - Check pip and npm dependencies for vulnerabilities
lockfile-lint - Lint a lockfile for security policies
...

Local development

Requirements

Docker Desktop - For creating a cluster (make sure Kubernetes is enabled)
kubectl - CLI for interacting with the cluster
Helm - Kubernetes package manager
Tilt - Kubernetes development tool
Python - For custom Tilt functionality
Node - For custom Tilt functionality

Optional

gettext - For making translations

Getting started

# Start Tilt
tilt up
# Open up the Tilt UI (and minify all groups except resources, if you prefer)
open http://localhost:10350/
# You can use the superuser "[email protected]" with password "admin" to explore
# the TanStack app, the Django admin, and the API documentation from the endpoints
# in the resources group (once everything is running)

Using VS Code and opening a specific directory based on the task provides the best developer experience:

# Docker/Kubernetes/Tilt development
code .
# Django development
code ./django
# TanStack development
code ./tanstack

Project structure

django-tanstack-starter/
├─ manifests/              # Kubernetes manifests
│  ├─ bases/               # Manifests shared across all environments
│  ├─ overlays/            # Additional manifests and patches for different environments
│  └─ platform-chart/      # Umbrella Helm chart for ingress-nginx and cert-manager
│
├─ django/                 # Root of the Django backend
│  ├─ config/              # Project-wide configuration
│  ├─ base/                # Base app containing things not suited for other apps
│  │                       # (translations, custom management commands, mail templates, ...)
│  ├─ users/               # App with everything pertaining to the user
│  └─ todos/               # Example app
│
├─ tanstack/               # Root of the TanStack frontend
│  ├─ e2e/                 # End-to-end Playwright tests
│  ├─ public/              # Files that are available on the root path /
│  └─ src/
│     ├─ api/              # API wrappers
│     ├─ components/       # React components
│     │  ├─ motion-ui/     # Motion-Primitives components
│     │  └─ ui/            # shadcn/ui components
│     ├─ hooks/            # React hooks
│     ├─ lib/              # Library of things not suited for the other categories
│     ├─ query/            # TanStack Query client, mutations, options
│     ├─ routes/           # TanStack Router routes
│     └─ types/            # TypeScript types
.
.
.

CI/CD

Static analysis & unit tests

This workflow is triggered when a new deploy to production is about to take place, and on every pull request using main as base.

If you want to use it as-is you need to add the GitHub Actions secret SNYK_TOKEN – a Snyk API token that can be retrieved from your account. Consider also referencing your organization with the --org flag.

End-to-end tests

This workflow is triggered only when a new deploy to production is about to take place, since it's fairly slow to set up a cluster in CI.

Deploy to production

Deployment to the production environment occurs when a git tag in the SemVer format X.Y.Z is created. This combined with GitHub rulesets makes for a very powerful and controlled way of deploying to the real environment.

The release.sh script is used for convenience and performs several sanity checks before proceeding with a new release:

./release.sh 0.1.0
# Created release 0.1.0 – 'git push origin main 0.1.0' to trigger the release

Set up a production environment

We will imagine that we have built a proof-of-concept (POC), before more resources are invested into the project. We are then tasked with setting up a cheap production environment for the POC.

We will be using Azure in this example, but it would mostly be conceptually the same on any other Cloud Service Provider (CSP). We will be using the North Europe region, but you can pick another region closer to you if you prefer.

If we were to go all-in on Azure, it would also be worth investigating if Azure DevOps with its version control and CI/CD, Azure Key Vault for secret handling, etc. should replace our use of GitHub. We continue using GitHub for now to make it as general as possible.

1. Azure account

Create an Azure account and ensure you have the generous free 30-day credit.

Install the CLI and sign in to your new account:

# Log in to Azure CLI.
az login
# Follow the instructions to log in through the browser.

# Put your subscription id in a shell variable for later use.
SUBSCRIPTION_ID=<your-subscription-id>

# Verify the variable.
echo $SUBSCRIPTION_ID

2. Resource Group

First, we will create a Resource Group to keep all our related resources together:

# Put the resource group name and location in shell variables.
RESOURCE_GROUP_NAME="dtss-prod"
LOCATION="northeurope"

# Create the resource group.
az group create --name $RESOURCE_GROUP_NAME --location $LOCATION

3. Virtual Network

Just as a resource group keeps all our resources grouped together, a Virtual Network (VNet) keeps the communication between these resources grouped together within the same private network.

# --address-prefix 10.0.0.0/16 means that the VNet will use 10.0.0.0 to 10.0.255.255.
# 65536 addresses will be more than enough for our use case.
VNET_NAME="vnet"
az network vnet create \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $VNET_NAME \
    --address-prefix 10.0.0.0/16

4. Cluster

Before creating our Azure Kubernetes Service (AKS) cluster, we must do an estimation of how much resources we actually need:

Kubernetes system components (kubelet, kube-proxy, ...): 350-1000m CPU, 500-1500Mi RAM
platform-chart (ingress-nginx, cert-manager): 200-400m CPU, 200-400Mi RAM
Our application (Django, Celery worker, TanStack): 600-1750m CPU, 800-1600Mi RAM (based on resource requests/limits)

That's a total of 1150m-3150m CPU, 1500Mi-4500Mi RAM. This will fit very snuggly on a cheap, burstable Standard_B2s (2 CPU, 4Gi RAM) single-node cluster.

Note
For production environments, consider using something like a general-purpose D-series VM and distributing nodes across all zones within the region (typically 3).

While this may be overkill for a POC, it's worth keeping in mind if the POC takes off.

# Register the Microsoft.ContainerService and Microsoft.Compute resource providers.
# These are necessary for creating Kubernetes clusters and virtual machines, respectively.
az provider register --namespace Microsoft.ContainerService
az provider register --namespace Microsoft.Compute

# Monitor the progress with the following commands.
# It will be done when it says "Registered".
az provider show --namespace Microsoft.ContainerService --query registrationState
az provider show --namespace Microsoft.Compute --query registrationState

# Create a subnet with an address space of 10.0.0.0/20, which is 10.0.0.0 to 10.0.15.255.
# 4096 addresses is more than enough for our POC environment,
# and leaves room for growth if needed.
AKS_SUBNET_NAME="cluster-subnet"
az network vnet subnet create \
    --resource-group $RESOURCE_GROUP_NAME \
    --vnet-name $VNET_NAME \
    --name $AKS_SUBNET_NAME \
    --address-prefix 10.0.0.0/20

# Put the subnet id in a shell variable
AKS_SUBNET_ID=$(az network vnet subnet show \
    --resource-group $RESOURCE_GROUP_NAME \
    --vnet-name $VNET_NAME \
    --name $AKS_SUBNET_NAME \
    --query id -o tsv)

# Create the AKS cluster.
#
# A new resource group will be created for the node pool and everything related to it,
# but it will manage itself, and it will be removed if the main resource group is removed.
#
# --network-plugin azure will activate Azure CNI,
# which allows our pods to communicate with external resources within the VNet later.
# This also allows pods to use IPs from the VNet directly.
#
# --enable-managed-identity will enable managed identity for the cluster,
# making it possible to use Azure services that require authentication.
# We will give it certain permissions in later sections.
#
# --service-cidr 10.1.0.0/16 changes the Service CIDR from the default 10.0.0.0/16,
# to avoid conflicts with our VNet address space,
# but still being within the private address space.
#
# --dns-service-ip 10.1.0.10 changes the default DNS service IP from 10.0.0.10,
# so that it is within the service CIDR range.
#
# --generate-ssh-keys will generate SSH keys to communicate with the cluster,
# but you could use existing keys if you prefer.
AKS_CLUSTER_NAME="cluster"
az aks create \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $AKS_CLUSTER_NAME \
    --node-vm-size standard_b2s \
    --node-count 1 \
    --network-plugin azure \
    --enable-managed-identity \
    --vnet-subnet-id $AKS_SUBNET_ID \
    --service-cidr 10.1.0.0/16 \
    --dns-service-ip 10.1.0.10 \
    --generate-ssh-keys

# Fetch the kubeconfig for the cluster.
az aks get-credentials \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $AKS_CLUSTER_NAME
# Check if the correct context is being used.
kubectl config get-contexts
# Verify that kubectl can connect to the cluster.
kubectl get nodes
# Create the production namespace, where the application will reside.
kubectl create namespace production

# Create a service principal (SP) with Contributor role, scoped to this cluster.
# This grants the SP rights to apply changes. We will use this in CI.
AKS_SP_NAME="cluster-sp-ci"
PROD_AKS_SP_CREDENTIALS=$(az ad sp create-for-rbac \
    --name $AKS_SP_NAME \
    --role Contributor \
    --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.ContainerService/managedClusters/$AKS_CLUSTER_NAME \
    --query "{clientId:appId, clientSecret:password, tenantId:tenant, subscriptionId:'$SUBSCRIPTION_ID'}" -o json)

# Note that all the echo commands in this example are for convenience.
# Be careful not to store secrets in logs.

# Store the AKS SP credentials in the GitHub Actions secret PROD_AKS_SP_CREDENTIALS.
echo $PROD_AKS_SP_CREDENTIALS
# Store the resource group name in the GitHub Actions secret PROD_RESOURCE_GROUP_NAME.
echo $RESOURCE_GROUP_NAME
# Store the AKS Cluster name in the GitHub Actions secret PROD_AKS_CLUSTER_NAME.
echo $AKS_CLUSTER_NAME

5. platform-chart

platform-chart is just an umbrella chart used to install ingress-nginx and cert-manager, with some sane defaults.

We use ingress-nginx in local development as well, to route incoming traffic to the correct service. cert-manager is a new addition that allows us to issue Let's Encrypt certificates, enabling HTTPS on our website.

# We will create a public IP for the load balancer of our cluster,
# so that the IP doesn't change unexpectedly.
# If you were to have cluster nodes in multiple zones in the region,
# it's worth using --zone 1 2 3 to make the public IP non-zonal,
# so that it doesn't share fate with the health of a particular zone.
CLUSTER_PUBLIC_IP_NAME="cluster-public-ip"
az network public-ip create \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $CLUSTER_PUBLIC_IP_NAME \
    --sku Standard

# Put the public IP in a shell variable.
CLUSTER_PUBLIC_IP=$(az network public-ip show \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $CLUSTER_PUBLIC_IP_NAME \
    --query ipAddress -o tsv)

# Put the managed identity principal id for the cluster in a shell variable.
AKS_IDENTITY_PRINCIPAL_ID=$(az aks show \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $AKS_CLUSTER_NAME \
    --query "identity.principalId" -o tsv)

# Give the cluster the rights to read the public IP resource.
az role assignment create \
  --assignee $AKS_IDENTITY_PRINCIPAL_ID \
  --role Reader \
  --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Network/publicIPAddresses/$CLUSTER_PUBLIC_IP_NAME

# Give the cluster rights to manage load balancers.
# This is needed because ingress-nginx creates a LoadBalancer service.
az role assignment create \
    --assignee $AKS_IDENTITY_PRINCIPAL_ID \
    --role "Network Contributor" \
    --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME

# Verify that the Reader and Network Contributor roles have been assigned.
# You should see non-empty tables as output when it's done.
az role assignment list \
  --assignee $AKS_IDENTITY_PRINCIPAL_ID \
  --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.Network/publicIPAddresses/$CLUSTER_PUBLIC_IP_NAME \
  --query "[].{Role:roleDefinitionName, Scope:scope}" -o table

az role assignment list \
  --assignee $AKS_IDENTITY_PRINCIPAL_ID \
  --scope /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME \
  --query "[].{Role:roleDefinitionName, Scope:scope}" -o table

# EMAIL_ADDRESS will receive warnings about certificates about to expire.
# Make sure to use a real email address, as using a domain like "example.com" will not work.
EMAIL_ADDRESS=""
helm upgrade platform ./manifests/platform-chart \
    --dependency-update \
    --install \
    --reuse-values \
    --set acme_email_address=$EMAIL_ADDRESS \
    --set ingress-nginx.controller.service.loadBalancerIP=$CLUSTER_PUBLIC_IP \
    --set ingress-nginx.controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-resource-group"=$RESOURCE_GROUP_NAME

# If the install was successful you should see two final events,
# "Ensuring load balancer" and "Ensured load balancer".
kubectl describe svc/ingress-nginx-controller --namespace ingress-nginx

# Potential errors are most likely due to slow role assignment propagation.
# You can uninstall the chart and try again.
#
# helm uninstall platform
# kubectl delete namespace ingress-nginx cert-manager

6. Domain

While Azure does not provide domain registration services, any domain registrar will work. For this setup, we will be using Cloudflare.

Buy a domain you would like to use, create a DNS A Record pointing to echo $CLUSTER_PUBLIC_IP, and redirect www to non-www.

# Add the domain you bought to a shell variable.
DOMAIN=<your-domain>
# Add the domain to the GitHub Actions variable PROD_DOMAIN.
echo $DOMAIN

7. PostgreSQL

Persistence is best handled outside of the cluster, so we will be using the cheapest managed PostgreSQL Flexible Server B1ms, which will be more than enough for our POC with its 2Gi RAM, 32Gi storage, and 7 days backup retention.

# Register the Microsoft.DBforPostgreSQL provider.
# This is necessary for creating managed PostgreSQL databases.
az provider register --namespace Microsoft.DBforPostgreSQL

# Monitor the progress with the following command.
# It will be done when it says "Registered".
az provider show --namespace Microsoft.DBforPostgreSQL --query registrationState

# Put the names for the database admin and Django user in shell variables.
PG_ADMIN_USER="db_admin"
PG_DJANGO_USER="django_user"

# Generate strong passwords for the users and put them in shell variables.
# You might want to set up a Key Vault and store these there for future reference.
# The admin password can be reset, but you might lose the Django user password.
PG_ADMIN_PASSWORD=$(openssl rand -base64 32 | tr -d '+/=')
PG_DJANGO_PASSWORD=$(openssl rand -base64 32 | tr -d '+/=')

# Create a subnet for the database.
#
# --delegations Microsoft.DBforPostgreSQL/flexibleServers ensures that the subnet
# is correctly associated with the PostgreSQL Flexible Server,
# and allows the required routing and access management.
#
# --address-prefix 10.0.16.0/24 gives us 256 IP addresses, which is more than enough.
# It's also right after 10.0.15.255, the end of the cluster address space.
PG_SUBNET_NAME="pg-subnet"
az network vnet subnet create \
    --resource-group $RESOURCE_GROUP_NAME \
    --vnet-name $VNET_NAME \
    --name $PG_SUBNET_NAME \
    --delegations Microsoft.DBforPostgreSQL/flexibleServers \
    --address-prefix 10.0.16.0/24

# Create the PostgreSQL server. This can take a long time.
#
# Problems might arise if the name isn't globally unique,
# even though the CLI doesn't say so.
# If a taken name is given, it will most likely hang until timeout.
#
# --yes is to answer "yes" to the question if we want a new private DNS zone.
PG_SERVER_NAME=""
az postgres flexible-server create \
    --resource-group $RESOURCE_GROUP_NAME \
    --location $LOCATION \
    --vnet $VNET_NAME \
    --subnet $PG_SUBNET_NAME \
    --name $PG_SERVER_NAME \
    --tier burstable \
    --sku-name standard_b1ms \
    --version 17 \
    --admin-user $PG_ADMIN_USER \
    --admin-password $PG_ADMIN_PASSWORD \
    --yes

# Since the PostgreSQL server has no public access,
# we will need a VM within the same VNet that we can connect to using SSH,
# so we can interact with it.
#
# Create a subnet for the VM in the 10.0.32.0/24 range (256 addresses).
# This allows us to fill our VNet with more relevant things between
# 10.0.17.0/24 and 10.0.31.0/24 in the future.
VM_SUBNET_NAME="vm-subnet"
az network vnet subnet create \
    --resource-group $RESOURCE_GROUP_NAME \
    --vnet-name $VNET_NAME \
    --name $VM_SUBNET_NAME \
    --address-prefix 10.0.32.0/24

# Create a public IP for the VM.
VM_PUBLIC_IP_NAME="vm-public-ip"
az network public-ip create \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $VM_PUBLIC_IP_NAME \
    --sku standard

# Create the VM.
VM_NAME="vm"
VM_USERNAME="azureuser"
az vm create \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $VM_NAME \
    --vnet-name $VNET_NAME \
    --subnet $VM_SUBNET_NAME \
    --image ubuntu2204 \
    --size standard_b1ms \
    --admin-username $VM_USERNAME \
    --public-ip-address $VM_PUBLIC_IP_NAME \
    --generate-ssh-keys

# Put the VM public IP in a shell variable.
VM_PUBLIC_IP=$(az vm list-ip-addresses \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $VM_NAME \
    --query "[].virtualMachine.network.publicIpAddresses[0].ipAddress" -o tsv)

# Store the FQDN of the PostgresQL server in a shell variable.
PG_DOMAIN=$(az postgres flexible-server show \
    --name $PG_SERVER_NAME \
    --resource-group $RESOURCE_GROUP_NAME \
    --query fullyQualifiedDomainName -o tsv)

# Connect to the VM using SSH,
# which in turn connects to the PostgreSQL server and creates the Django user.
ssh $VM_USERNAME@$VM_PUBLIC_IP <<EOF
    sudo apt update
    sudo apt install -y postgresql-client-common postgresql-client

    psql "host=$PG_DOMAIN port=5432 user=$PG_ADMIN_USER password=$PG_ADMIN_PASSWORD dbname=postgres sslmode=require" <<PSQL
        CREATE USER $PG_DJANGO_USER WITH PASSWORD '$PG_DJANGO_PASSWORD';
        GRANT ALL PRIVILEGES ON DATABASE postgres TO $PG_DJANGO_USER;

        \c postgres;

        GRANT ALL PRIVILEGES ON SCHEMA public TO $PG_DJANGO_USER;
        GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO $PG_DJANGO_USER;
        ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL PRIVILEGES ON TABLES TO $PG_DJANGO_USER;
        ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL PRIVILEGES ON SEQUENCES TO $PG_DJANGO_USER;
PSQL
EOF

# Put the database connection string in a shell variable.
PROD_DATABASE_URL="postgresql://$PG_DJANGO_USER:$PG_DJANGO_PASSWORD@$PG_DOMAIN:5432/postgres?sslmode=require"
# Store the database connection string in the GitHub Actions secret PROD_DATABASE_URL.
echo $PROD_DATABASE_URL

# Deallocate the VM. It can be useful to have a VM available,
# so you can activate it when you need to do some administrative tasks.
# Only the storage will cost when it's deallocated, which is negligible in this case.
az vm deallocate --resource-group $RESOURCE_GROUP_NAME --name $VM_NAME

8. Redis

By default, Redis is only used as a message broker for Celery. The Azure Cache for Redis Basic tier and the cheapest VM C0 with its 250Mi RAM will be more than enough for the POC.

Note
Only the Premium tier would allow us to put the Redis server within our VNet.

The publicNetworkAccess is Disabled by default, we will use a private endpoint from our VNet, TLS, and an access key, so it will still be secure, but it's worth keeping in mind for the future.

Caution
The Basic tier offers no type of disk persistence, so if it should crash for some reason, the database will be lost. We store the task results in the PostgreSQL database, but this does mean we could lose tasks that send welcome emails.

If the POC takes off, it would be a good idea to upgrade to the Standard tier for disk persistence, but it's hard to justify the cost initially.

# Register the Microsoft.Cache provider.
# This is necessary for creating Azure Cache for Redis.
az provider register --namespace Microsoft.Cache

# Monitor the progress with the following command.
# It will be done when it says "Registered".
az provider show --namespace Microsoft.Cache --query registrationState

# Create the Redis server. This can take a long time.
REDIS_SERVER_NAME="redis-server"
az redis create \
    --resource-group $RESOURCE_GROUP_NAME \
    --location $LOCATION \
    --name $REDIS_SERVER_NAME \
    --sku Basic \
    --vm-size C0

# The Redis server itself can't be in the VNet when using the Basic tier,
# but we can still set up a subnet with a private endpoint to it.
#
# It might seem overkill with a separate subnet for this,
# but this way we keep it nicely separated,
# and leave room for a potential Premium tier server in the future.
REDIS_SUBNET_NAME="redis-subnet"
az network vnet subnet create \
    --resource-group $RESOURCE_GROUP_NAME \
    --vnet-name $VNET_NAME \
    --name $REDIS_SUBNET_NAME \
    --address-prefix 10.0.17.0/24

# Put the private connection resource id in a shell variable.
PRIVATE_CONNECTION_RESOURCE_ID_REDIS=$(az redis show \
    --name $REDIS_SERVER_NAME \
    --resource-group $RESOURCE_GROUP_NAME \
    --query id --output tsv)

# Create a private endpoint for the Redis server.
az network private-endpoint create \
    --name $REDIS_SERVER_NAME-private-endpoint \
    --resource-group $RESOURCE_GROUP_NAME \
    --vnet-name $VNET_NAME \
    --subnet $REDIS_SUBNET_NAME \
    --private-connection-resource-id $PRIVATE_CONNECTION_RESOURCE_ID_REDIS \
    --group-ids redisCache \
    --connection-name $REDIS_SERVER_NAME-connection

# Private endpoints were not originally made for the Basic and Standard tiers,
# so we need to do the private DNS zone setup manually.
# If we were to setup a Premium tier server within our VNet,
# it would be just as automatic as with the PostgreSQL server.
#
# Create the Private DNS Zone.
az network private-dns zone create \
    --resource-group $RESOURCE_GROUP_NAME \
    --name "privatelink.redis.cache.windows.net"

# Link the Private DNS Zone to the VNet.
az network private-dns link vnet create \
    --resource-group $RESOURCE_GROUP_NAME \
    --zone-name "privatelink.redis.cache.windows.net" \
    --name "redis-dns-link" \
    --virtual-network $VNET_NAME \
    --registration-enabled false

# Create DNS zone group for the private endpoint.
az network private-endpoint dns-zone-group create \
    --resource-group $RESOURCE_GROUP_NAME \
    --endpoint-name $REDIS_SERVER_NAME-private-endpoint \
    --name "redis-dns-group" \
    --private-dns-zone "privatelink.redis.cache.windows.net" \
    --zone-name redis

# Put the FQDN of the Redis server in a shell variable.
REDIS_DOMAIN=$(az redis show \
    --name $REDIS_SERVER_NAME \
    --resource-group $RESOURCE_GROUP_NAME \
    --query hostName -o tsv)

# Put the Redis primary key in a shell variable.
PRIMARY_KEY=$(az redis list-keys \
    --name $REDIS_SERVER_NAME \
    --resource-group $RESOURCE_GROUP_NAME \
    --query "primaryKey" -o tsv)

# Put the Redis connection string in a shell variable.
# Note that "rediss" is not a typo – it's the Redis equivalent of https.
# Redis accepts secure connections on port 6380, instead of the regular 6379.
PROD_CELERY_BROKER_URL="rediss://:$PRIMARY_KEY@$REDIS_DOMAIN:6380/0?ssl_cert_reqs=required"
# Store the Redis connection string in the GitHub Actions secret PROD_CELERY_BROKER_URL.
echo $PROD_CELERY_BROKER_URL

9. Container Registry

Our CI build results in a couple of Docker images that we need to put on a private registry. We also need to allow our cluster to pull from this registry.

The cheapest Basic tier Azure Container Registry (ACR) will work for our use case.

Note
A container registry is a good candidate for an organization-wide resource, but we keep everything in the same resource group in this example for simplicity.

Caution
Only the Premium tier offers automatic retention policies. We have to clean up old images manually to not accumulate storage costs.

It can be done with a manual reoccurring task in the Azure Portal, or through the CLI, or by adding a step to the CI, ...

# Register the Microsoft.ContainerRegistry provider.
# This is necessary for creating Azure Container Registries.
az provider register --namespace Microsoft.ContainerRegistry

# Monitor the progress with the following command.
# It will be done when it says "Registered".
az provider show --namespace Microsoft.ContainerRegistry --query registrationState

# Put the ACR name in a shell variable.
#
# This name has to be globally unique.
ACR_NAME="dtssregistry"

# Create the ACR.
az acr create \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $ACR_NAME \
    --sku Basic

# Put the managed identity principal id of the AKS cluster in a shell variable.
AKS_MANAGED_IDENTITY=$(az aks show \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $AKS_CLUSTER_NAME \
    --query "identity.principalId" -o tsv)

# Give the cluster the rights to pull images from the ACR.
az aks update \
    --name $AKS_CLUSTER_NAME \
    --resource-group $RESOURCE_GROUP_NAME \
    --attach-acr $ACR_NAME

# Create the SP with AcrPush role scoped to the ACR.
# This grants the SP rights to push images. We will use this in the CI.
ACR_SP_NAME="acr-sp-ci"
ACR_SP_CREDENTIALS=$(az ad sp create-for-rbac \
    --name $ACR_SP_NAME \
    --role AcrPush \
    --scopes /subscriptions/$SUBSCRIPTION_ID/resourceGroups/$RESOURCE_GROUP_NAME/providers/Microsoft.ContainerRegistry/registries/$ACR_NAME \
    --query "{clientId:appId, clientSecret:password}" -o json)

# Extract the individual credential values using jq.
# sudo apt-get install jq
# brew install jq
PROD_ACR_SP_CLIENT_ID=$(echo $ACR_SP_CREDENTIALS | jq -r '.clientId')
PROD_ACR_SP_CLIENT_SECRET=$(echo $ACR_SP_CREDENTIALS | jq -r '.clientSecret')

# Store the ACR SP credentials in GitHub Actions secrets
# PROD_ACR_SP_CLIENT_ID and PROD_ACR_SP_CLIENT_SECRET.
echo $PROD_ACR_SP_CLIENT_ID
echo $PROD_ACR_SP_CLIENT_SECRET

# Put the image repositories names in shell variables.
DJANGO_REPO="dtss-django"
TANSTACK_REPO="dtss-tanstack"

# Note that we don't have to create repositories for our images manually,
# since the ACR will create repositories as new images are pushed to it.

# Store the ACR name in a GitHub Actions variable REGISTRY_NAME.
echo $ACR_NAME
# Store the Django repository name in a GitHub Actions variable DJANGO_IMAGE_NAME.
echo $DJANGO_REPO
# Store the TanStack repository name in a GitHub Actions variable TANSTACK_IMAGE_NAME.
echo $TANSTACK_REPO

10. Content Delivery Network

Our frontend Vite build results in a dist directory with 100% static files – an index.html, the files from the public directory, and a nested assets directory with the remaining build artifacts.

By putting the assets directory in a public Blob Storage container, and serving the files with a Content Delivery Network (CDN), we offload most of the requests that would have gone to our cluster, and let the CDN do what it does best.

The cheapest Blob Storage and CDN tiers will be sufficient for our use case.

Note
Both the storage account and the CDN are good candidates for organization-wide resources, but we keep everything in the same resource group in this example for simplicity.

# Register the Microsoft.Storage and Microsoft.Cdn providers.
# This is necessary for creating and managing storages and CDN profiles, respectively.
az provider register --namespace Microsoft.Storage
az provider register --namespace Microsoft.Cdn

# Monitor the progress with the following commands.
# It will be done when both say "Registered".
az provider show --namespace Microsoft.Storage --query registrationState
az provider show --namespace Microsoft.Cdn --query registrationState

# Create a storage account.
#
# This name has to be globally unique.
#
# --allow-blob-public-access true lets us make storage containers public,
# which will let the CDN read from the containers without authentication.
# This is fine, since we just want to expose public frontend assets.
STORAGE_ACCOUNT_NAME="dtssstorageaccount"
az storage account create \
    --name $STORAGE_ACCOUNT_NAME \
    --resource-group $RESOURCE_GROUP_NAME \
    --sku Standard_LRS \
    --allow-blob-public-access true

# Put the storage account key in a shell variable.
STORAGE_ACCOUNT_KEY=$(az storage account keys list \
    --resource-group $RESOURCE_GROUP_NAME \
    --account-name $STORAGE_ACCOUNT_NAME \
    --query '[0].value' \
    --output tsv)

# Allow our domain to fetch assets from the CDN.
#
# --services b allows us to read blobs.
az storage cors add \
    --account-name $STORAGE_ACCOUNT_NAME \
    --services b \
    --methods GET \
    --origins https://$DOMAIN \
    --account-key $STORAGE_ACCOUNT_KEY

# Create a container in the storage account to store the assets.
CONTAINER_NAME="dtss-prod"
az storage container create \
    --account-name $STORAGE_ACCOUNT_NAME \
    --name $CONTAINER_NAME \
    --account-key $STORAGE_ACCOUNT_KEY

# Since the container will just contain frontend assets, we can make it public,
# so the CDN can read from it without authentication.
az storage container set-permission \
    --account-name $STORAGE_ACCOUNT_NAME \
    --name $CONTAINER_NAME \
    --account-key $STORAGE_ACCOUNT_KEY \
    --public-access blob

# Create a CDN profile.
CDN_PROFILE_NAME="cdn-profile"
az cdn profile create \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $CDN_PROFILE_NAME \
    --sku Standard_Microsoft

# Create a CDN endpoint for the profile.
#
# This name has to be globally unique.
CDN_ENDPOINT_NAME="dtss-cdn"
az cdn endpoint create \
    --resource-group $RESOURCE_GROUP_NAME \
    --profile-name $CDN_PROFILE_NAME \
    --name $CDN_ENDPOINT_NAME \
    --origin $STORAGE_ACCOUNT_NAME.blob.core.windows.net \
    --origin-host-header $STORAGE_ACCOUNT_NAME.blob.core.windows.net

# Get the connection string for the storage account
STORAGE_ACCOUNT_CONNECTION_STRING=$(az storage account show-connection-string \
    --resource-group $RESOURCE_GROUP_NAME \
    --name $STORAGE_ACCOUNT_NAME \
    --query connectionString \
    --output tsv)

# Put the CDN URL in a shell variable.
#
# We could create a custom domain for the CDN, at for example cdn.<your-domain>,
# but we skip this for simplicity.
#
# A request to:
# https://$CDN_ENDPOINT_NAME.$CDN_PROFILE_NAME.azureedge.net/$CONTAINER_NAME/assets/foo.js
# will be routed through the CDN and then checked against the storage URL:
# https://$STORAGE_ACCOUNT_NAME.blob.core.windows.net/$CONTAINER_NAME/assets/foo.js
#
# The CDN will first try to serve the requested file from its edge locations.
# If the file is not cached at the edge, it will fetch it from the origin.
#
# Note that the Vite assets have cache-busting names, so as long as we don't overwrite
# existing files in CI, old releases will continue to work.
CDN_URL="https://$CDN_ENDPOINT_NAME.azureedge.net"

# Store the Blob Storage connection string in the GitHub Actions secret
# STORAGE_ACCOUNT_CONNECTION_STRING.
echo $STORAGE_ACCOUNT_CONNECTION_STRING
# Store the container name in the GitHub Actions variable PROD_STORAGE_CONTAINER_NAME.
echo $CONTAINER_NAME
# Store the CDN URL string in the GitHub Actions variable CDN_URL.
echo $CDN_URL

11. Mailgun

Just as with domain registrars, there are plenty of email sending platforms. We will be using Mailgun, but any other alternative would work nicely. Keep in mind that any other platform would involve slight code changes, since we are using a Mailgun dependency.

Create an account, add the domain we bought earlier as a custom domain, choose a domain region, and use automatic sender security.

Note
The domain region determines the geographical location of your Mailgun server. This is important because it affects latency for your email operations.

Install and verify all the listed DNS records on Cloudflare, and click Verify in Mailgun until all records are verified, and then create a Sending API key.

Lastly, add the Sending API Key as a GitHub Actions secret MAILGUN_API_KEY, a variable MAILGUN_API_URL with the value https://api.eu.mailgun.net/v3 if you chose the EU region or https://api.mailgun.net/v3 if you chose the US region, and a variable FROM_EMAIL with an email address (for example noreply@<your-domain>)

12. Sentry

There are many types of monitoring you might want to do, but being notified of unexpected backend errors is very useful for all applications. Sentry makes this easy and intuitive.

Create a Sentry account, create a Django project, and add the data source name (DSN) of the project as a GitHub Actions secret PROD_SENTRY_DSN.

13. Django secret

The Django secret key is used for cryptographic signing, and should be set to a unique, unpredictable value, for every environment.

The easiest way to generate one is to activate the virtual environment in the ./django directory and run a tiny script:

cd ./django
# Assuming you have used Tilt for local development,
# and the virtual environment already exists.
. ./env/bin/activate
python -c '
from django.core.management.utils import get_random_secret_key
print(get_random_secret_key())'
# Store the output in a GitHub Actions secret PROD_SECRET_KEY.
# In a real scenario you would want to store this in a Key Vault, since it can be lost.

14. First release

Use the release.sh script to create the first release:

./release.sh 0.1.0
# Created release 0.1.0 – 'git push origin main 0.1.0' to trigger the release
git push origin main 0.1.0

The Static analysis & unit tests and End-to-end tests workflows will run, and if they succeed, a release will take place.

If any variables or secrets are missing, the Deploy to production workflow will fail. You can add the missing variables and secrets, then manually rerun the workflow to trigger the release.

When the release is complete you can open a shell in the Django pod container and create a superuser:

# List pods in the production namespace and copy the name of the Django pod.
kubectl get pods -n production
# Open up a shell in the Django pod container.
kubectl exec -it <django-pod-name> -n production -- /bin/sh
./manage.py createsuperuser

15. Closing thoughts

You might want to avoid a tedious manual setup like this by describing your infrastructure with the help of Terraform instead (or a vendor-specific alternative like Bicep, ARM Templates, ...).

You could even go all-in GitOps and combine it with something like Argo CD or Flux.

We have opted for not including these technologies in the starter, since it's difficult to do so without creating a harder vendor lock-in. Most concepts in this setup are the same for every CSP.

# Don't forget to remove the resource group if you don't plan on using it anymore,
# so that you can use your credit for something else.
az group delete --name $RESOURCE_GROUP_NAME --no-wait

Deploy to testing

Deploying to the testing environment can be done manually, from any branch. This way we have a very serious and restricted way of deploying to production, but a very relaxed approach to deploying to the testing environment.

It can for example be done from a bugfix/ branch to make sure a potential fix actually works in a real environment, or from a feature/ branch to try database migrations before merging to main, ...

Set up a testing environment

Setting up a testing environment involves following the steps outlined in Set up a production environment, but using the testing namespace, and testing-specific variables and secrets prefixed with TEST_ instead of PROD_.

There are many ways to manage testing and staging environments, each with its own trade-offs. For example, you might set up:

A dedicated testing cluster for a single application
A shared testing cluster for multiple applications, where each application has its own namespace. You could even have shared testing PostgreSQL and Redis servers, with each application having its own testing database
One single large, organization-wide cluster with separate namespaces for different applications and environments
...

FAQ

Why the name "django-tanstack-starter"?

Django + Ninja is such a good starting point that there's no real need to look around for other backend alternatives.

With TanStack Router + Query (and soon Start), it feels the same for the frontend.

When is this not a good choice?

There are plenty of scenarios where this is not a good starting point:

Your application doesn't require a web interface (command-line tools, a backend-only service, a desktop application, ...)
You or your team prefer C# and Angular, or Ruby on Rails, or Laravel, ...
Your project has strict performance requirements that Python or React might struggle to meet
...

Does this work on Windows?

It will sadly not work on Windows for the following reasons (non-exhaustive list):

The custom Tilt functionality is written with Linux/macOS in mind
psycopg will likely have to be compiled from source in the Dockerfile instead of just installing the pip dependency psycopg[binary,pool]
...

I need server-side rendering. Can this be used with TanStack Start?

Yes, to use Start you can do the following:

Do the steps outlined in Getting Started
Configure it to use src instead of app in app.config.ts
Create a Node.js server with the build
Change the NGINX server to a Node.js server in the Dockerfile and use it in .github/workflows/deploy.yaml
Change the tanstack-deployment pod container to start the Node.js server

This starter will use Start when it leaves beta, becomes a Vite plugin instead of using Vinxi, and gets support for SPA mode. Adding SSR will then hopefully just be a matter of changing a config flag, and using a separate Dockerfile.ssr file instead.

I have no need for background work. Do I need Celery and Redis?

No, if you don't have a need for any background work and are fine with for example sending emails synchronously in the request/response cycle, you can omit both Celery and Redis. From experience, it usually doesn't take long before some type of background work is needed, so it's nice to have it in place by default.

There is ongoing discussion about creating a PostgreSQL Celery broker, which could replace Redis in the initial setup.

The Django community is working on official support for database-backed background workers and a task-based SMTP email backend. These features may eliminate the need for Celery and Redis entirely, depending on your requirements.

How can I use sensitive environment variables for development without version controlling them?

You could extend the kustomization.yaml for development to patch django-cm and django-secret with additional environment variables, and make sure you add the patches to .gitignore.

The frontend JavaScript bundle is very large. Can something be done about that?

Yes, but firstly, one large bundle for the frontend is not a problem if you are building an application that is completely hidden behind a login. If you need to reduce the bundle size you could use TanStack Router code splitting, uninstall the library Motion if you don't want the animations, ...

You might want a separate service for the public-facing parts of your application to optimize the SEO to the fullest. This will all be handled with Start in the near future.

Why is every dependency saved with an exact version?

It's a matter of trade-offs. With exact versions you minimize the chance of stuff breaking, but on the other hand you get no automatic bug fixes, etc.

It's easy to stay on top of it with the VS Code extensions Python PyPI Assistant and Version Lens, but you can of course change it if you don't agree with the trade-offs.

Why is the login functionality so basic?

The login functionality is intentionally very bare-bones, since it differs so much from application to application. You might want to expand on it with “forgot password” functionality and proper link expiration, or remove it entirely and just use social login, ...

The Django authentication system with Groups works great for many applications, but sometimes you might reach for something like Auth0.

Does this work with WebSockets?

Yes, you can use Django Channels, but you will have to swap Uvicorn for Daphne first. You could also introduce another Daphne service that handles the traffic to path prefix /ws.

You should not rely on Kubernetes pods being long-lived, so it's best to use something like lukeed's Sockette to automatically reconnect if the connection is lost, and implement logic so that data is fetched anew in case messages were missed during the downtime.

Why is this a "starting point"?

You might want to:

Add Prometheus or a vendor-specific alternative for more insightful monitoring
Move the celery beat into a separate deployment to allow for increasing the amount of replicas in the celery-worker-deployment, if you need a lot of background work
Add Horizontal Pod Autoscaling to automatically adjust the amount of replicas on your deployments to match demand, and Cluster Autoscaling to automatically scale the amount of nodes up and down
Use canary or blue-green deployment strategies
Configure cert-manager to use the DNS-01 challenge rather than the HTTP-01 challenge
Do the PostgreSQL database migration in some other way than with the default init container
Add Elasticsearch for more powerful search capabilities, if PostgreSQL's full text search isn't sufficient
Migrate the database to something like PlanetScale if your application becomes very popular
...

Good luck!

Name		Name	Last commit message	Last commit date
Latest commit History 1 Commit
.github/workflows		.github/workflows
.vscode		.vscode
django		django
manifests		manifests
tanstack		tanstack
.gitattributes		.gitattributes
.gitignore		.gitignore
.npmrc		.npmrc
.prettierignore		.prettierignore
.prettierrc		.prettierrc
README.md		README.md
Tiltfile		Tiltfile
license		license
package-lock.json		package-lock.json
package.json		package.json
release.sh		release.sh

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Repository files navigation

django-tanstack-starter

Technology

Local development

Requirements

Getting started

Project structure

CI/CD

Static analysis & unit tests

End-to-end tests

Deploy to production

Deploy to testing

FAQ

About

Releases

Packages

Languages

License

EmilTholin/django-tanstack-starter

Folders and files

Latest commit

History

Repository files navigation

django-tanstack-starter

Technology

Local development

Requirements

Getting started

Project structure

CI/CD

Static analysis & unit tests

End-to-end tests

Deploy to production

Deploy to testing

FAQ

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Packages