Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue/9/insert cmodels in database #20

Merged
merged 52 commits into from
Apr 25, 2021
Merged
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
7e1ba19
Merge branch 'basic_api_config' of https://github.com/fenfisdi/cdslab…
Anturion Apr 10, 2021
762f731
add: requirements file
Anturion Apr 12, 2021
e703c3b
Build: collection cmodels
Anturion Apr 12, 2021
d31b7ae
pull: remote branch
Anturion Apr 13, 2021
f891880
Merge branch 'dev' of https://github.com/fenfisdi/cdslab_cmodels_api …
Anturion Apr 13, 2021
9bab012
fix: cmodels flow updated"
Anturion Apr 15, 2021
accbec1
Merge branch 'dev' of https://github.com/fenfisdi/cdslab_cmodels_api …
Anturion Apr 15, 2021
63a9a7f
build: db cmodels, insert cmodels collection method
Anturion Apr 15, 2021
ed2b520
Build: use cases cmodels
Anturion Apr 16, 2021
55cc2a1
fix:updated insert_cmodels_document
Anturion Apr 16, 2021
aada177
fix: model cmodel
Anturion Apr 16, 2021
002e25d
fix: remove unnecessary imports
Anturion Apr 19, 2021
1abb89d
fix: remove unnecessary imports
Anturion Apr 19, 2021
3bab513
build: utils cmodels file and create id cmodels method in usecases
Anturion Apr 19, 2021
981e07d
build: test of cmodels created
Anturion Apr 19, 2021
65b8443
fix: style
jearistiz Apr 20, 2021
41a2a54
.gitignore: .DS_Store
jearistiz Apr 20, 2021
49a0f07
refactor: src.interfaces.cmodel_interface.cmodel_collection -> src.db…
jearistiz Apr 20, 2021
93e9c43
refactor: crud
jearistiz Apr 20, 2021
6a42b36
refactor: compartmentl models db-related stuff
jearistiz Apr 21, 2021
842d011
fix: sonnarcloud-reported bugs
jearistiz Apr 21, 2021
76dedd9
refactor: inserting cmodels in db
jearistiz Apr 21, 2021
547c09c
fix: updated model
Anturion Apr 22, 2021
79f87f0
fix: cmodels
Anturion Apr 22, 2021
1d695ba
Merge branch 'issue/9/insert_cmodels_in_database' of https://github.c…
Anturion Apr 22, 2021
ec83c82
refactor: changed back cmodels documents creation to main.py
jearistiz Apr 22, 2021
aef427e
build: test get_collection method
Anturion Apr 23, 2021
678cd9b
build: test cmodel interface
Anturion Apr 23, 2021
887bb13
Merge branch 'issue/9/insert_cmodels_in_database' of https://github.c…
Anturion Apr 23, 2021
0541d80
fix: test_cmodel_interface
Anturion Apr 23, 2021
1298ea0
build: test delete crud method
Anturion Apr 23, 2021
ce731a8
Update src/interfaces/cmodels.py
jearistiz Apr 23, 2021
51ff8da
Update src/use_cases/cmodels.py
jearistiz Apr 23, 2021
d2e2f69
Update main.py
jearistiz Apr 23, 2021
c615607
Update src/interfaces/cmodels.py
jearistiz Apr 23, 2021
f8192ea
Update src/interfaces/cmodels.py
jearistiz Apr 23, 2021
af450a9
fix: style - sort imports
jearistiz Apr 23, 2021
91331f3
fix: interface tests
Anturion Apr 23, 2021
451a2f5
refactor: interfaces and appropiate tests
jearistiz Apr 23, 2021
65edffc
tests: cmodel use_cases
jearistiz Apr 23, 2021
3a91fa2
refactor: renamed attributes in src.interfaces test cases
jearistiz Apr 23, 2021
1a155cc
chore: updated requirements.txt
jearistiz Apr 23, 2021
7c416b0
fix: style
jearistiz Apr 23, 2021
88b970d
fix: style src/models/db/cmodels.py
jearistiz Apr 23, 2021
27dcb68
deleted src/models/db/cmodels.AllCModels (not needed)
jearistiz Apr 23, 2021
6d4d874
Merge branch 'issue/9/insert_cmodels_in_database' of https://github.c…
jearistiz Apr 23, 2021
29c836f
fix: style in tests/interfaces/test_cmodels.py
jearistiz Apr 23, 2021
aef1e22
Chore: Update Singleton Pattern
Apr 24, 2021
673260f
Merge branch 'feature/update_features' into issue/9/merge_docker_db_s…
jearistiz Apr 24, 2021
7095f08
fix: test cases run appropriately
jearistiz Apr 24, 2021
31ecb49
fix: type hint
jearistiz Apr 24, 2021
930fbdb
fix: typo and setters' logic in src.db.mongo
jearistiz Apr 25, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -151,3 +151,6 @@ cython_debug/

# Experiments
experiments/

# MacOS
.DS_Store
2 changes: 1 addition & 1 deletion Pipfile.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

7 changes: 5 additions & 2 deletions main.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
import uvicorn

from src.config import settings
from src.use_cases.cmodels import CmodelUseCases

if __name__ == '__main__':

if __name__ == '__main__':
CmodelUseCases.update_cmodels_collection()
uvicorn.run(
"src.api:app",
host=settings['HOST'],
host=settings.get('HOST'),
port=int(settings['PORT']),
reload=True,
debug=True,
Expand Down
1 change: 1 addition & 0 deletions src/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from src.config import settings
from src.routers.main import main_router


app = FastAPI()


Expand Down
16 changes: 16 additions & 0 deletions src/db/mongo.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
from typing import Generator, Tuple

from pymongo import MongoClient
from pymongo.database import Database
from pymongo.collection import Collection

from src.config import db_config

Expand Down Expand Up @@ -45,3 +47,17 @@ def get_db(
db: Database = db_connection[db_name]

return db_connection, db


def get_collection(
coll_name: str = db_config['CMODELS_COLL']
) -> Tuple[MongoClient, Collection]:
"""
Returns
-------
db_connection: pymongo.MongoClient
coll: pymongo.collection.Collection
"""
db_connection, db = get_db()
coll: Collection = db[coll_name]
return db_connection, coll
5 changes: 0 additions & 5 deletions src/interfaces/cmodel_interface.py

This file was deleted.

59 changes: 59 additions & 0 deletions src/interfaces/cmodels.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
from datetime import datetime

from pymongo.collection import Collection
from pymongo.mongo_client import MongoClient

from src.models.db.cmodels import (
CompartmentalModel,
CompartmentalModelEnum
)
from src.interfaces.crud import MongoCRUD


class CModelsInterface:

def __init__(
self,
db_connection: MongoClient,
collection: Collection
) -> None:
self.crud = MongoCRUD(db_connection, collection)

def insert_one_cmodel_document(self, model: CompartmentalModel):

cmodel_document = model.dict(by_alias=True)

existent_model = self.crud.read(model.id)

if existent_model:
pruned_existent_model = CModelsInterface._prune_db_document(
existent_model
)
pruned_current_model = CModelsInterface._prune_db_document(
cmodel_document
)
if pruned_existent_model == pruned_current_model:
# TODO: log cmodel exists f'Cmodel exists: {model.name}'
return existent_model
else:
model.updated_at = datetime.utcnow()
updated_model = self.crud.update(
model.id,
model.dict(by_alias=True)
)
# TODO log updated cmodel f'Updated cmodel: {model.name}'
return updated_model
else:
model_inserted = self.crud.insert(cmodel_document)
# TODO: log created cmodel f'Created cmodel: {model.name}'
return model_inserted

def insert_all_cmodel_documents(self):
return [self.insert_one_cmodel_document(model) for model in CompartmentalModelEnum.values()]

@staticmethod
def _prune_db_document(model_in_db: dict) -> dict:
if model_in_db:
model_in_db.pop('inserted_at')
model_in_db.pop('updated_at')
return model_in_db
102 changes: 102 additions & 0 deletions src/interfaces/crud.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
from typing import Any, Union
from bson.objectid import ObjectId

from pymongo.collection import Collection
from pymongo.mongo_client import MongoClient


class MongoCRUD():
def __init__(
self,
db_connection: MongoClient,
collection: Collection
) -> None:
self.db_connection = db_connection
self.collection = collection

def insert(self, document: dict):
"""
Parameters
----------
model
The compartmental models that will be save in the database

Return
----------
model: pymongo object
"""
with self.db_connection:
try:
document['_id']
except KeyError:
raise ValueError('self.insert(): document must have key "_id"')

existent_document = self.collection.find_one(
self._id_to_dict(document['_id'])
)

if existent_document:
raise ValueError(
f'Document with _id={document["_id"]} already exists in'
'collection {self.collection}. if you want to change the'
'fields\' values please use self.update'
)
# TODO: log

return self.collection.insert_one(document)

def read(self, _id: ObjectId) -> Union[Any, None]:
"""Search for a specific model in ``self.collection``.

Parameters
----------
query
Document's id. ``dict`` schema: ``{'_id': bson.ObjectID}``

Return
----------
model: pymongo object
Object containing the results of the search
"""
with self.db_connection:
return self.collection.find_one(self._id_to_dict(_id))

def update(self, _id: ObjectId, new_data: dict) -> bool:
"""Update document in ``self.collection``.

Parameters
----------
query
Document's id. ``dict`` schema: ``{'_id': bson.ObjectID}``
data
Updated document fields

Return
----------
False:
* If query has no information
* If is not possible to update the document's status
* If the query id doesn't match the one associated to ``new_data``
True:
If the model has valid data and its status can be updated
"""
_id_dict = self._id_to_dict(_id)
with self.db_connection:
if not _id_dict:
return False
cmodel = self.collection.find_one(_id_dict)
if cmodel:
update_model = self.collection.update_one(
_id_dict,
{"$set": new_data},
)
if update_model:
return True
return False

def delete(self, _id: ObjectId):
with self.db_connection:
return self.collection.delete_one(self._id_to_dict(_id))

def _id_to_dict(self, _id: ObjectId):
return {'_id': _id} if _id else None
5 changes: 0 additions & 5 deletions src/interfaces/simulation_interface.py

This file was deleted.

13 changes: 13 additions & 0 deletions src/models/db/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
from .cmodels import (
CompartmentalModelBase,
CompartmentalModel,
CModel,
AllCModels
)

__all__ = [
'CompartmentalModelBase',
'CompartmentalModel',
'CModel',
'AllCModels'
]
26 changes: 26 additions & 0 deletions src/models/db/base_model.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
from datetime import datetime

from pydantic import BaseModel, Field
from bson.objectid import ObjectId as BsonObjectId


class PydanticObjectId(BsonObjectId):
@classmethod
def __get_validators__(cls):
yield cls.validate

@classmethod
def validate(cls, v):
if not isinstance(v, BsonObjectId):
raise TypeError('ObjectId required')
return v


class MetadataBaseDoc(BaseModel):
id: PydanticObjectId = Field(..., alias='_id')
inserted_at: datetime = datetime.utcnow()
updated_at: datetime = datetime.utcnow()

class Config:
allow_population_by_field_name = True
extra = 'forbid'
101 changes: 101 additions & 0 deletions src/models/db/cmodels.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
from enum import Enum
from typing import Dict, List
from bson.objectid import ObjectId

from pydantic import BaseModel

from .base_model import MetadataBaseDoc

class CompartmentalModelBase(BaseModel):
"""Base Model for Compartmental Models
"""
name: str
"""Name of the compartmental model"""
state_variables: List[str]
"""Name of state variables of corresponding model"""
state_variables_units: Dict[str, str]
"""Units of each state variable. The keys are the 'state_variables' array
elements
"""
parameters: List[str]
"""Parameters of the corresponding model"""
parameters_units: Dict[str, str]
"""Units of each parameter. The keys are the 'parameters' array elements"""


class CompartmentalModel(MetadataBaseDoc, CompartmentalModelBase):
pass


class CompartmentalModelEnum(Enum):
"""Compartmental Models' Data.

Each element of this ``Enum`` class contains all the essential information
on the corresponding Compartmental Model. Each element is a
:class:``CompartmentalModelBase`` object
"""
sir: CompartmentalModelBase = CompartmentalModel(
id=ObjectId('6083175ea91f5aacea234423'),
name='SIR',
state_variables=['S', 'I', 'R'],
state_variables_units={
'S': 'persons',
'I': 'persons',
'R': 'persons',
},
parameters=['a', 'b'],
parameters_units={
'a': 'units of a',
'b': 'units of b',
},
)

seir: CompartmentalModelBase = CompartmentalModel(
id=ObjectId('6083176ca91f5aacea234424'),
name='SEIR',
state_variables=['S', 'E', 'I', 'R'],
state_variables_units={
'S': 'persons',
'E': 'persons',
'I': 'persons',
'R': 'persons',
},
parameters=['a', 'b'],
parameters_units={
'a': 'units of a',
'b': 'units of b',
},
)

seirv: CompartmentalModelBase = CompartmentalModel(
id=ObjectId('608317d0a91f5aacea234426'),
name='SEIRV',
state_variables=['S', 'E', 'I', 'R', 'V'],
state_variables_units={
'S': 'persons',
'E': 'persons',
'I': 'persons',
'R': 'persons',
'V': 'persons',
},
parameters=['a', 'b'],
parameters_units={
'a': 'units of a',
'b': 'units of b',
'c': 'units of c'
},
)

@classmethod
def values(cls) -> List[CompartmentalModel]:
return [m.value for m in cls]


class CModel(BaseModel):
model: CompartmentalModelEnum


class AllCModels(BaseModel):
models: List[CompartmentalModelEnum] = [
model.value for model in CompartmentalModelEnum
]
Loading