Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(Validata): Import achats: ajout d'appels à Validata et renvoyer les erreurs #4949

Merged
merged 7 commits into from
Feb 12, 2025

Conversation

raphodn
Copy link
Member

@raphodn raphodn commented Jan 23, 2025

Nouveau fichier common/api/validata.py

  • validate_file_against_schema_with_validata
  • process_validata_errors

@raphodn raphodn changed the base branch from staging to raphodn/backend-api-utils-import January 23, 2025 16:55
@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from 16396a7 to 17b25bd Compare January 23, 2025 16:57
self.data_schema = json.load(open("data/schemas/imports/achats.json"))
self.expected_header = [field["name"] for field in self.data_schema["fields"]]
self.schema_url = (
"https://raw.githubusercontent.com/betagouv/ma-cantine/refs/heads/staging/data/schemas/imports/achats.json"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On peut aussi en profiter pour refactor la fonction validate en passant le fichier directement
voir code compl'alim

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dans ton exemple le schema est aussi une url il me semble ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oui pardon, j'ai confondu avec le fichier

@raphodn raphodn force-pushed the raphodn/backend-api-utils-import branch from bb31e03 to 314296d Compare January 28, 2025 11:46
@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from 8a4f83b to 60764b2 Compare January 28, 2025 11:47
@raphodn raphodn force-pushed the raphodn/backend-api-utils-import branch 3 times, most recently from 0b86819 to 43b6c1c Compare January 28, 2025 14:32
@@ -31,6 +34,45 @@ def _get_file_digest(self):
file_hash.update(row)
return file_hash.hexdigest()

def _get_file_dialect(self):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

je vais basculer la logique "dialect" dans la PR précédente

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

et j'ai basculé la logique header dans une PR séparée

@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from 60764b2 to 36b538b Compare January 28, 2025 17:04
@raphodn raphodn force-pushed the raphodn/backend-api-utils-import branch 3 times, most recently from 76c1e8e to 6be0241 Compare January 29, 2025 10:11
Base automatically changed from raphodn/backend-api-utils-import to staging January 29, 2025 10:17
@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from 36b538b to 9a82b53 Compare January 29, 2025 10:23
@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from 87eb3d6 to 5ef3aa0 Compare January 30, 2025 12:08
@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from 5ef3aa0 to 9dc53a8 Compare February 3, 2025 15:39
@raphodn raphodn changed the base branch from staging to raphodn/backend-api-utils-header-check-purchases February 3, 2025 15:39
@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch 4 times, most recently from b902895 to e837692 Compare February 3, 2025 16:40
Base automatically changed from raphodn/backend-api-utils-header-check-purchases to staging February 4, 2025 12:43
@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from e837692 to bf22a83 Compare February 4, 2025 17:39
return response.json()["report"]


def process_errors(report):
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

question à trancher : est-ce qu'on pre-process les erreurs Validata ici (dans le backend), ou coté frontend ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Côté front +1 !

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes go pour le front ! du coup je renvoi un maximum d'infos dans l'objet

@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from b9210ab to aac7a7a Compare February 10, 2025 14:50
# Reset the file pointer to the beginning
file.seek(0)
response = requests.post(
"https://api.validata.etalab.studio/validate",
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

à noter que ca fait des appels API à chaque lancement de tests CI 👀

@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from 9699526 to 4b4d446 Compare February 10, 2025 16:13
@raphodn raphodn changed the title feat(API): Imports: Achats: ajout d'appels à Validata feat(Validata): Import achats: ajout d'appels à Validata Feb 10, 2025
@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from 4b4d446 to 4a21643 Compare February 10, 2025 16:32
@raphodn raphodn changed the title feat(Validata): Import achats: ajout d'appels à Validata feat(Validata): Import achats: ajout d'appels à Validata et renvoyer les erreurs Feb 10, 2025
@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from 4a21643 to c3c174f Compare February 10, 2025 16:56
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remarque mais pas bloquant, en fait on aurait aussi pu simplifier les tests en vérifiant le "title" et non "message" ?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

je vois ton message que maintenant, mais en effet maintenant qu'on a le title... je note de corriger

Copy link
Member Author

@raphodn raphodn Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

je vais laisser tel quel, vu que ailleurs (les autres erreurs) on fait les test sur le champ message aussi..
à voir comment on pourra homogénéiser tout ça

@raphodn raphodn force-pushed the raphodn/backend-api-utils-header-validata branch from c3c174f to 1560e7f Compare February 11, 2025 16:51
@raphodn raphodn merged commit 2c58007 into staging Feb 12, 2025
6 of 7 checks passed
@raphodn raphodn deleted the raphodn/backend-api-utils-header-validata branch February 12, 2025 09:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants