Skip to content

Commit

Permalink
bootstap script (#145)
Browse files Browse the repository at this point in the history
## Description
Removing the git controlled `.env` file and automating the dev setup
process with `scripts/bootstrap.sh`.

## Additional Notes
Developers should be able to customize the .env file to their
environment, and not have to worry about it changing whenever they pull
new code. To make the developer setup process easy, the new bootstrap
script will create a .env file if it doesn't exist, initialize a virtual
environment if it doesn't exist, activate the virtual env and install
the development python requirements.

- Connection pooling parameters now have their default values set to
None, so it will work with sqlite:///:memory:
- The sessionmaker autocommit parameter was removed in sqlalchemy 2.0
  • Loading branch information
ericbuckley authored Dec 11, 2024
1 parent 7de3b70 commit 3a7e58b
Show file tree
Hide file tree
Showing 8 changed files with 94 additions and 15 deletions.
1 change: 0 additions & 1 deletion .env

This file was deleted.

3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,9 @@ temp/
# ignore all files ending in .DS_Store
**/.DS_Store

# Project environment files
.env

# Python-Virtual Environments
.venv
.python-version
Expand Down
24 changes: 18 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,19 +20,17 @@ The RecordLinker is a service that links records from two datasets based on a se
### Initial Setup

Set up a Python virtual environment and install the required development dependencies:
NOTE: The `-e` flag is used to install the package in "src/" in editable mode. This allows
the package to be imported in other modules without needing to re-install the package.
NOTE: Sourcing the script is recommended over simply executing the script. This allows
the virtual environment to stay active in your shell.
```bash
python -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'
source scripts/bootstrap.sh
```

### Running the API

To run the API locally, use the following command:
```bash
./scripts/local_server.sh 8000
./scripts/local_server.sh
```

The API will be available at `http://localhost:8000`. Visit `http://localhost:8000/redocs` to view the API documentation.
Expand All @@ -49,6 +47,20 @@ To run a single unit test, use the following command:
pytest tests/unit/test_utils.py::test_bind_functions
```

### Running type checks

To run type checks, use the following command:
```bash
mypy
```

### Running code formatting checks

To run code formatting checks, use the following command:
```bash
ruff check
```

For more information on developer workflows, see the [Developer Guide](docs/developer_guide.md).

## Standard Notices
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,7 @@ filterwarnings = [
"ignore:typing.io is deprecated, import directly from typing instead:DeprecationWarning"
]
env = [
"DB_URI=sqlite:///:memory:",
"INITIAL_ALGORITHMS=",
]

Expand Down
32 changes: 32 additions & 0 deletions scripts/bootstrap.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/bash

# Use this script to initialize the development environment.
#
# Usage: source bootstrap.sh
# Requires: python3

cd "$(dirname "$0")/.."

# Create a default .env file if it doesn't exist
if [ ! -f .env ]; then
echo "Creating a default .env file..."
echo "DB_URI=sqlite:///db.sqlite3" > .env
else
echo "Default .env file already exists."
fi

# Create a virtual environment if it doesn't exist
if [ ! -d .venv ]; then
echo "Creating a virtual environment..."
python3 -m venv .venv
else
echo "Virtual environment already exists."
fi

# Activate the virtual environment
echo "Activating the virtual environment..."
source .venv/bin/activate
# Install the development requirements
echo "Installing the development requirements..."
pip install --upgrade pip > /dev/null
pip install '.[dev]' > /dev/null
4 changes: 2 additions & 2 deletions src/recordlinker/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,12 @@ class Settings(pydantic_settings.BaseSettings):
)
connection_pool_size: typing.Optional[int] = pydantic.Field(
description="The number of MPI database connections in the connection pool",
default=5,
default=None,
)
connection_pool_max_overflow: typing.Optional[int] = pydantic.Field(
description="The maximum number of MPI database connections that can be opened "
"above the connection pool size",
default=10,
default=None,
)
log_config: typing.Optional[str] = pydantic.Field(
description="The path to the logging configuration file",
Expand Down
13 changes: 7 additions & 6 deletions src/recordlinker/database/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,15 @@ def create_sessionmaker(init_tables: bool = True) -> orm.sessionmaker:
"""
Create a new sessionmaker for the database connection.
"""
engine = create_engine(
settings.db_uri,
pool_size=settings.connection_pool_size,
max_overflow=settings.connection_pool_max_overflow,
)
kwargs: dict[str, typing.Any] = {}
if settings.connection_pool_size is not None:
kwargs["pool_size"] = settings.connection_pool_size
if settings.connection_pool_max_overflow is not None:
kwargs["max_overflow"] = settings.connection_pool_max_overflow
engine = create_engine(settings.db_uri, **kwargs)
if init_tables:
models.Base.metadata.create_all(engine)
return orm.sessionmaker(autocommit=False, bind=engine)
return orm.sessionmaker(bind=engine)


def get_session() -> typing.Iterator[orm.Session]:
Expand Down
31 changes: 31 additions & 0 deletions tests/unit/database/test_base.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
"""
unit.database.test_base.py
~~~~~~~~~~~~~~~~~~~~~~~~~~
This module contains the unit tests for the recordlinker.database module.
"""

import unittest.mock

from recordlinker.config import settings
from recordlinker.database import create_sessionmaker


def test_create_sessionmaker():
"""
Test the create_sessionmaker function.
"""
with unittest.mock.patch.dict(
"os.environ",
{
"DB_URI": "sqlite:///test.db",
"CONNECTION_POOL_SIZE": "10",
"CONNECTION_POOL_MAX_OVERFLOW": "20",
},
):
settings.__init__()
session = create_sessionmaker()()
assert str(session.bind.url) == "sqlite:///test.db"
assert session.bind.pool.size() == 10
assert session.bind.pool._max_overflow == 20
settings.__init__()

0 comments on commit 3a7e58b

Please sign in to comment.