Add initial implementation of lidar API with FastAPI, Docker, and Kub…

…ernetes configuration

.github/PULL_REQUEST_TEMPLATE.md

@@ -8,6 +8,6 @@ Please explain the purpose and scope of your contribution.
 
 ## Checklist before requesting a review
 
-- [ ] I have performed a self-review of my code
-- [ ] When resolving a specific issue, it's referenced in the PR's title (e.g. `fix #xxx[,#xxx]`, where "xxx" is the issue number)
-- [ ] Don't forget to link PR to issue if you are solving one.
+- I have performed a self-review of my code
+- When resolving a specific issue, it's referenced in the PR's title (e.g. `fix #xxx[,#xxx]`, where "xxx" is the issue number)
+- Don't forget to link PR to issue if you are solving one.

.github/copilot-instructions.md

@@ -0,0 +1,12 @@
+Python 3 and FastAPI Guidelines:
+
+- Follow PEP 8 style guidelines for Python code.
+- Use type hints extensively for better code clarity and maintainability.
+- Structure FastAPI applications using modular design (routers, services, models).
+- Implement proper error handling with custom exception classes and middleware.
+- Use Pydantic for data validation and serialization.
+- Write comprehensive unit and integration tests using pytest.
+- Use environment variables for configuration.
+- Implement logging for debugging and monitoring.
+- Use Docker and Docker Compose for containerization and deployment.
+- Follow RESTful API design principles.

.github/workflows/README.md

@@ -0,0 +1,19 @@
+# We use the following secrets:
+
+Don't forget to check thtat everthing is working and configured on the
+https://github.com/EPFL-ENAC/AddLidar-API/settings/secrets/actions repository_secrets tab
+
+## For the Continuous Deployment pipeline (on our kube cluster):
+
+Btw the path of the config is located here
+cd $repo_org/$repo_name/overlays/$branch
+
+(cf https://github.com/EPFL-ENAC/enack8s-app-config?tab=readme-ov-file#setting-secrets for how to setup)
+CD_ORG = "epfl-resslab" # The organisation folder for the repository manifests in enack8s-app-config ie. epfl-resslab-astra-82001
+CD_REPO = "AddLidar-API_frontend" # The repository name for the manifests in enack8s-app-config ie.
+CD_TOKEN = "your-cd-token-secret"
+CD_URI = "[your-uri](https://api.github.com/repos/EPFL-ENAC/enack8s-app-config/dispatches)" # The URI of the webhook that establishes the manifest code updating
+
+## For the automatic release using google release-please github action
+
+MY_RELEASE_PLEASE_TOKEN = "your-my-release-please-token-secret"

.github/workflows/build-and-push.yml

@@ -0,0 +1,17 @@
+# https://github.com/EPFL-ENAC/epfl-enac-build-push-deploy-action#readme
+name: deploy
+
+'on':
+  push:
+    branches:
+      - dev
+    tags: ['v*.*.*']
+
+jobs:
+  deploy:
+    uses: EPFL-ENAC/epfl-enac-build-push-deploy-action/.github/workflows/[email protected]
+    secrets:
+      token: ${{ secrets.CD_TOKEN }}
+    with:
+      org: epfl-cryos # your org
+      repo: addlidar-api # your app name, usual convention is name of your repository

.github/workflows/quality-checks.yml

@@ -0,0 +1,34 @@
+name: Quality Checks
+
+on:
+  pull_request:
+    branches: [main, dev]
+  push:
+    branches: [main, dev]
+
+jobs:
+  quality:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python 3.10
+        uses: actions/setup-python@v4
+        with:
+          python-version: "3.10"
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install -r requirements.txt
+
+      - name: Lint with Flake8
+        run: |
+          flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
+          flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
+
+      - name: Format with Black
+        run: black . --check
+
+      - name: Test with Pytest
+        run: pytest

.github/workflows/release-please.yml

@@ -0,0 +1,24 @@
+on:
+  push:
+    branches:
+      - main
+
+permissions:
+  contents: write
+  pull-requests: write
+
+name: release-please
+
+jobs:
+  release-please:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: google-github-actions/release-please-action@v4
+        with:
+          # this assumes that you have created a personal access token
+          # (PAT) and configured it as a GitHub action secret named
+          # `MY_RELEASE_PLEASE_TOKEN` (this secret name is not important).
+          token: ${{ secrets.MY_RELEASE_PLEASE_TOKEN }}
+          # this is a built-in strategy in release-please, see "Action Inputs"
+          # for more options
+          release-type: node

lidar-api/.gitignore

@@ -0,0 +1,31 @@
+
+node_modules
+
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Distribution / packaging
+.Python
+env/
+venv/
+ENV/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written into a dist/ subdirectory,
+#  but make sure to ignore them if they're lying around.
+build/
+dist/
+*.manifest
+*.spec
+
+# macOS
+*.DS_Store
+
+# Environments
+.env

lidar-api/Dockerfile

@@ -0,0 +1,12 @@
+FROM python:3.9-slim-buster
+
+WORKDIR /app
+
+COPY requirements.txt .
+RUN pip install --no-cache-dir uv
+RUN uv venv
+RUN uv pip install --no-cache-dir -r requirements.txt
+
+COPY . .
+
+CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

lidar-api/Makefile

@@ -0,0 +1,14 @@
+run:
+	uvicorn src.main:app --reload --host 0.0.0.0 --port 8000
+
+install:
+	uv pip install -r requirements.txt
+
+format:
+	black .
+
+lint:
+	flake8 .
+
+test:
+	pytest

lidar-api/README.md

@@ -0,0 +1,90 @@
+# README.md
+
+# Lidar API
+
+This project is a FastAPI application that wraps the LidarDataManager CLI, allowing users to process LiDAR point cloud data via HTTP requests. The API runs the CLI inside a Docker container and mounts a volume for data access.
+
+## Project Structure
+
+```
+lidar-api
+├── src
+│   ├── main.py                # Entry point of the FastAPI application
+│   ├── config
+│   │   └── settings.py        # Configuration settings for the application
+│   ├── api
+│   │   ├── routes.py          # API endpoints definition
+│   │   └── models.py          # Data models for request and response
+│   ├── services
+│   │   └── docker_service.py   # Logic for interacting with Docker
+│   └── utils
+│       └── file_utils.py      # Utility functions for file handling
+├── kubernetes
+│   ├── deployment.yaml         # Kubernetes deployment configuration
+│   └── service.yaml            # Kubernetes service configuration
+├── Dockerfile                  # Instructions to build the Docker image
+├── requirements.txt            # Python dependencies
+├── docker-compose.yml          # Docker Compose configurations
+└── README.md                   # Project documentation
+```
+
+## Setup Instructions
+
+1. Clone the repository:
+   ```
+   git clone <repository-url>
+   cd lidar-api
+   ```
+
+2. Install the required dependencies:
+   ```
+   pip install -r requirements.txt
+   ```
+
+3. Build the Docker image:
+   ```
+   docker build -t lidar-api .
+   ```
+
+4. Run the application using Docker Compose:
+   ```
+   docker-compose up
+   ```
+
+## Usage
+
+### Endpoint
+
+**GET /process-point-cloud**
+
+Processes a LiDAR point cloud file using the CLI.
+
+### Query Parameters
+
+- `file_path` *(required)*: Path to the input point cloud file (inside mounted volume).
+- `remove_attribute`: Remove specified attribute(s).
+- `remove_all_attributes`: Remove all non-geometry attributes.
+- `remove_color`: Remove color data.
+- `format`: Output format (`pcd-ascii`, `lasv14`, etc.).
+- `line`: Export a specific line index.
+- `returns`: Max return index.
+- `number`: Max number of points in output.
+- `density`: Max density (points per m²).
+- `roi`: Region of interest.
+- `outcrs`: Output CRS (e.g., `EPSG:4326`).
+- `incrs`: Override input CRS.
+
+### Example Request
+
+```
+GET /process-point-cloud?outcrs=EPSG:4326&r=10
+```
+
+## Response
+
+- **Success (`200 OK`)**: CLI output
+- **Error (`400/500`)**: CLI error message
+
+## License
+
+This project is licensed under the MIT License.

lidar-api/docker-compose.yml

@@ -0,0 +1,28 @@
+version: '3.8'
+
+services:
+  lidar-api:
+    build:
+      context: .
+      dockerfile: Dockerfile
+    ports:
+      - "8000:8000"
+    volumes:
+      - ./data:/data
+    environment:
+      - ENV=development
+    depends_on:
+      - db
+
+  db:
+    image: postgres:latest
+    restart: always
+    environment:
+      POSTGRES_USER: user
+      POSTGRES_PASSWORD: password
+      POSTGRES_DB: lidar_db
+    volumes:
+      - db_data:/var/lib/postgresql/data
+
+volumes:
+  db_data:

lidar-api/kubernetes/deployment.yaml

@@ -0,0 +1,32 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: lidar-api
+  labels:
+    app: lidar-api
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: lidar-api
+  template:
+    metadata:
+      labels:
+        app: lidar-api
+    spec:
+      containers:
+      - name: lidar-api
+        image: your-docker-image:latest
+        ports:
+        - containerPort: 8000
+        env:
+        - name: ENV_VARIABLE_NAME
+          value: "your_value"
+        resources:
+          requests:
+            memory: "256Mi"
+            cpu: "500m"
+          limits:
+            memory: "512Mi"
+            cpu: "1"
+      restartPolicy: Always

lidar-api/kubernetes/service.yaml

@@ -0,0 +1,14 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: lidar-api-service
+  labels:
+    app: lidar-api
+spec:
+  type: ClusterIP
+  ports:
+    - port: 80
+      targetPort: 8000
+      protocol: TCP
+  selector:
+    app: lidar-api

lidar-api/requirements.txt

@@ -0,0 +1,6 @@
+fastapi
+uvicorn
+pydantic
+httpx
+docker
+python-dotenv

lidar-api/src/api/models.py

@@ -0,0 +1,23 @@
+from pydantic import BaseModel
+from typing import Optional, List
+
+
+class ProcessPointCloudRequest(BaseModel):
+    file_path: str
+    remove_attribute: Optional[List[str]] = None
+    remove_all_attributes: Optional[bool] = False
+    remove_color: Optional[bool] = False
+    format: Optional[str] = None
+    line: Optional[int] = None
+    returns: Optional[int] = None
+    number: Optional[int] = None
+    density: Optional[float] = None
+    roi: Optional[str] = None
+    outcrs: Optional[str] = None
+    incrs: Optional[str] = None
+
+
+class ProcessPointCloudResponse(BaseModel):
+    success: bool
+    message: str
+    data: Optional[dict] = None