Skip to content

Commit

Permalink
Hl 934 ahjo auth (#2316)
Browse files Browse the repository at this point in the history
* feat: add new AhjoSetting class for storing Ahjo-related settings into the DB

* feat: add a class for retrieving and refreshing Ahjo access tokens

* feat: add a Django command for refreshing  the Ahjo access token

* feat: add dummy function as basis for further AHJO features

* feat: improve error messages

* feat: add instructions to readme

* fix: code formatting

* fix: conflicting migrations, add timestamps

* fix: migration conflict
  • Loading branch information
rikuke authored Oct 24, 2023
1 parent df16348 commit 332e471
Show file tree
Hide file tree
Showing 10 changed files with 496 additions and 2 deletions.
6 changes: 6 additions & 0 deletions .env.benefit-backend.example
Original file line number Diff line number Diff line change
Expand Up @@ -103,3 +103,9 @@ SENTRY_ENVIRONMENT=local
# for Mailhog inbox
EMAIL_HOST=mailhog
EMAIL_PORT=1025

AHJO_CLIENT_ID=
AHJO_CLIENT_SECRET=
AHJO_TOKEN_URL=
AHJO_REST_API_URL=
AHJO_REDIRECT_URL=
40 changes: 40 additions & 0 deletions backend/benefit/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,3 +213,43 @@ env variables / settings are provided by Azure blob storage:
The `local`, `development` and `testing` environments are connected to the Sentry instance at [`https://sentry.test.hel.ninja/`](https://sentry.test.hel.ninja/) under the `yjdh-benefit`-team.
There are separate Sentry projects for the Django api (`yjdh-benefit-api`), handler UI (`yjdh-benefit-handler`) and applicant UI (`yjdh-benefit-applicant`).
To limit the amount of possibly sensitive data sent to Sentry, the same configuration as in kesaseteli is used by default, see [`https://github.com/City-of-Helsinki/yjdh/pull/779`](https://github.com/City-of-Helsinki/yjdh/pull/779).

## AHJO integration
Making request to the AHJO REST api requires a Bearer token in the authorization headers.
### Retrieving the access_token

The token is retrieved following the Oauth 2.0 flow.
To retrieve the token in Django with AhjoConnector class,
the following settings need to be configured for Django:
```
AHJO_CLIENT_ID
AHJO_CLIENT_SECRET
AHJO_TOKEN_URL
AHJO_REST_API_UR
AHJO_REDIRECT_URL
```

1. The first step is to navigate via browser to (maintenance VPN enabled on local dev environment) [`https://johdontyopoytahyte.hel.fi/ids4/connect/authorize?scope=openid%20offline_access&response_type=code&redirect_uri=https://helsinkilisa/dummyredirect.html&client_id=client_id_goes_here`](https://johdontyopoytahyte.hel.fi/ids4/connect/authorize?scope=openid%20offline_access&response_type=code&redirect_uri=https://helsinkilisa/dummyredirect.html&client_id=client_id_goes_here)
- `scope=openid offline_access` is required so that the actual token call also returns a refresh token.
- `redirect_uri` is dummy according t, because it is not needed for anything after this, but it must be defined.
2. Login to the form with AD credentials (ask from fellow developer)
3. Submitting the form redirects the browser to the redirect_uri parameter address, for example
`https://helsinkilisa/dummyredirect.html?code=5510FE3A7A99D4A8D0FB69C0BAB70A31DD38243EFB1D606B1F96FE75383684E4-1&scope=offline_access&iss=https%3A%2F%2Fjohdontyopoytahyte.hel.fi%2Fids4`
- Again the `redirect_uri parameter` has no other use, so it can be dummyredirect.html
- from this return address the `code` parameter is taken, in the example above:
`5510FE3A7A99D4A8D0FB69C0BAB70A31DD38243EFB1D606B1F96FE75383684E4-1`
4. In the Django admin, on the AhjoSetting tab, set the setting ahjo_code to a JSON object:
`{"code": "5510FE3A7A99D4A8D0FB69C0BAB70A31DD38243EFB1D606B1F96FE75383684E4-1"}`
5. Now the AhjoConnector class can fetch the new token. At this stage of development, there is one dummy function for testing authentication, which can be used like this:
`$ python manage.py shell`
`$ from applications.services.ahjo_integration import dummy_ahjo_request`
`$ dummy_ahjo_request()`
6. Unless there is an error, there will be a new ahjo_access_token object (example below) in the database, which can be used for making actual requests to AHJO.
```JSON
{"expires_in": "2023-10-06T21:02:14.459161", "access_token": "eyJhbGciOiJSUzI1NiIsImtpZCI6IkY1QUMzRjhGNjNDQTdGQjc0QzgxODc1RkYyNTQ4M0YyMzI0RTNFMjNSUzI1NiIsIng1dCI6Ijlhd19qMlBLZjdkTWdZZGY4bFNEOGpKT1BpTSIsInR5cCI6ImF0K2p3dCJ9.eyJpc3MiOiJodHRwczovL2pvaGRvbnR5b3BveXRhaHl0ZS5oZWwuZmkvaWRzNCIsIm5iZiI6MTY5NjU4NTMzNCwiaWF0IjoxNjk2NTg1MzM0LCJleHAiOjE2OTY2MTUzMzQsImF1ZCI6Imh0dHBzOi8vam9oZG9udHlvcG95dGFoeXRlLmhlbC5maS9pZHM0L3Jlc291cmNlcyIsInNjb3BlIjpbIm9wZW5pZCIsIm9mZmxpbmVfYWNjZXNzIl0sImFtciI6WyJwd2QiXSwiY2xpZW50X2lkIjoiaGVsc2lua2lsaXNhIiwic3ViIjoiaGVsbGlzYWh5dCIsImF1dGhfdGltZSI6MTY5NTgwNzgzNywiaWRwIjoibG9jYWwiLCJzaWQiOiJBM0NFNkZFN0FBQkREMzQ4MUJBQTlBQzgzREVCRTZFNCIsImp0aSI6IjcyMzAyRkY0MjEwRkYxQTE4RTA1RDFFQTZCQTUwNDc3In0.iJOgkX4P1eNqDCVmXP2U198U_YIF0labba3hRP2x8oUA3DmCqCpPxvLIdMuxJ5N--xtqrW2hJw2X6XS-GQa9aSODwP5Tt5XLvzMthAzD6m4Y09uaZFoVGqvBu8Cc6oedJNQknQKTiK8vyhoHrXoG-ACOoYs1JtUBOsqR-SIgIpEZepWp3XcjlcVsSnqf1j1YTsRDl5FfoIv1lZSFTAlRmEZGirL1rRDm_2pR_HQ4y20KAaoZaBuyVoyf89duSGmvf40FlImLMXWuIcS7FkIrMUNogdgRittSJKRj5yfRnCgzjBndn0OptWtzXk5GZPfQeGERwVMJaD82X843j5UX4g", "refresh_token": "BB2AE7A54C9EB4374FCB69B21AE75484D9C33094DD92C4DADD48F5806FE726F3-1"}
```
7. In the future, it is intended that the token will be continuously refreshed with a cron job (see refreshing the token), so points 1-4 are not needed unless for some reason the token refresh fails during the 8-hour period when the token is valid

### Refreshing the token
The token retrieved the first time is valid for 30,000 seconds, or about 8 hours. A successful token call also returns the refresh_token information, which is also stored in the Django database. Django has a registered command refresh_ahjo_token which can be scheduled to perform token refresh. The command can be run manually with
`$ python manage.py refresh_ahjo_token`
7 changes: 7 additions & 0 deletions backend/benefit/applications/admin.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
from django.contrib import admin

from applications.models import (
AhjoSetting,
Application,
ApplicationBasis,
ApplicationBatch,
Expand Down Expand Up @@ -101,10 +102,16 @@ class ApplicationBasisAdmin(admin.ModelAdmin):
inlines = (ApplicationBasisInline,)


class AhjoSettingAdmin(admin.ModelAdmin):
list_display = ["name", "data"]
search_fields = ["name"]


admin.site.register(Application, ApplicationAdmin)
admin.site.register(ApplicationBatch, ApplicationBatchAdmin)
admin.site.register(DeMinimisAid)
admin.site.register(Employee)
admin.site.register(Attachment)
admin.site.register(ApplicationBasis, ApplicationBasisAdmin)
admin.site.register(ApplicationLogEntry)
admin.site.register(AhjoSetting, AhjoSettingAdmin)
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
from django.core.exceptions import ObjectDoesNotExist
from django.core.management.base import BaseCommand

from applications.models import AhjoSetting
from applications.services.ahjo_authentication import AhjoConnector


class Command(BaseCommand):
help = "Refresh the Ahjo token using the refresh_token stored in the database"

def handle(self, *args, **options):
try:
ahjo_auth_code = AhjoSetting.objects.get(name="ahjo_code").data
self.stdout.write(f"Retrieved auth code: {ahjo_auth_code}")
except ObjectDoesNotExist:
self.stdout.write(
"Error: Ahjo auth code not found in database. Please set the 'ahjo_code' setting."
)
return

ahjo_connector = AhjoConnector()
if not ahjo_connector.is_configured():
self.stdout.write(
"Error: Ahjo connector is not properly configured. Please check your settings."
)
return
try:
token = ahjo_connector.refresh_token()
except Exception as e:
self.stdout.write(f"Failed to refresh Ahjo token: {e}")
self.stdout.write(f"Ahjo token refreshed, token valid until {token.expires_in}")
26 changes: 26 additions & 0 deletions backend/benefit/applications/migrations/0043_ahjosetting.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Generated by Django 3.2.18 on 2023-10-13 12:38

from django.db import migrations, models


class Migration(migrations.Migration):

dependencies = [
('applications', '0042_reviewstate_paper'),
]

operations = [
migrations.CreateModel(
name='AhjoSetting',
fields=[
('id', models.BigAutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('created_at', models.DateTimeField(auto_now_add=True, verbose_name='time created')),
('modified_at', models.DateTimeField(auto_now=True, verbose_name='time modified')),
('name', models.CharField(max_length=255, unique=True)),
('data', models.JSONField()),
],
options={
'abstract': False,
},
),
]
7 changes: 6 additions & 1 deletion backend/benefit/applications/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from django.conf import settings
from django.core.validators import MaxLengthValidator, MinLengthValidator
from django.db import connection, models
from django.db.models import OuterRef, Subquery
from django.db.models import JSONField, OuterRef, Subquery
from django.utils.translation import gettext_lazy as _
from encrypted_fields.fields import EncryptedCharField, SearchField
from phonenumber_field.modelfields import PhoneNumberField
Expand Down Expand Up @@ -876,3 +876,8 @@ class ReviewState(models.Model):
benefit = models.BooleanField(default=False, verbose_name=_("benefit"))
employment = models.BooleanField(default=False, verbose_name=_("employment"))
approval = models.BooleanField(default=False, verbose_name=_("approval"))


class AhjoSetting(TimeStampedModel):
name = models.CharField(max_length=255, unique=True)
data = JSONField()
149 changes: 149 additions & 0 deletions backend/benefit/applications/services/ahjo_authentication.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
from dataclasses import dataclass
from datetime import datetime, timedelta
from typing import Dict, Union

import requests
from django.conf import settings
from django.core.exceptions import ObjectDoesNotExist

from applications.models import AhjoSetting


@dataclass
class AhjoToken:
access_token: str = ""
refresh_token: str = ""
expires_in: datetime = datetime.now()


class AhjoConnector:
def __init__(self, requests_module: requests.Session = requests) -> None:
self.requests_module: requests = requests_module
self.token_url: str = settings.AHJO_TOKEN_URL
self.client_id: str = settings.AHJO_CLIENT_ID
self.client_secret: str = settings.AHJO_CLIENT_SECRET
self.redirect_uri: str = settings.AHJO_REDIRECT_URL

self.grant_type_for_auth_token: str = "authorization_code"
self.grant_type_for_refresh_token: str = "refresh_token"
self.headers: Dict[str, str] = {
"Content-Type": "application/x-www-form-urlencoded",
}
self.timout: int = 10

def is_configured(self) -> bool:
"""Check if all required config options are set"""
if (
not self.token_url
or not self.client_id
or not self.client_secret
or not self.redirect_uri
):
return False

return True

def get_access_token(self, auth_code: str) -> AhjoToken:
"""Get access token from db first, then from Ahjo if not found or expired"""
token = self.get_token_from_db()
if token and not self._check_if_token_is_expired(token.expires_in):
return token
else:
return self.get_new_token(auth_code)

def get_new_token(self, auth_code: str) -> AhjoToken:
"""Retrieve the initial access token from Ahjo API using the auth code,
this is only used when getting the initial token or when the token has expired.
"""
if not auth_code:
raise Exception("No auth code")
payload = {
"client_id": self.client_id,
"client_secret": self.client_secret,
"grant_type": self.grant_type_for_auth_token,
"code": auth_code,
"redirect_uri": self.redirect_uri,
}
return self.do_token_request(payload)

def refresh_token(self) -> AhjoToken:
"""Refresh access token from Ahjo API using the refresh token of an existing token.
This should be used by, for example, a cron job to keep the token up to date.
"""
token = self.get_token_from_db()
if not token.refresh_token:
raise Exception("No refresh token")

payload = {
"client_id": self.client_id,
"client_secret": self.client_secret,
"grant_type": self.grant_type_for_refresh_token,
"refresh_token": token.refresh_token,
}

return self.do_token_request(payload)

def do_token_request(self, payload: Dict[str, str]) -> AhjoToken:
# Make the POST request
response = self.requests_module.post(
self.token_url, headers=self.headers, data=payload, timeout=self.timout
)

# Check if the request was successful
if response.status_code == 200:
# Extract the access token from the JSON response
access_token = response.json().get("access_token", "")
expires_in = response.json().get("expires_in", "")
refresh_token = response.json().get("refresh_token", "")
expiry_datetime = self.convert_expires_in_to_datetime(expires_in)

token = AhjoToken(
access_token=access_token,
refresh_token=refresh_token,
expires_in=expiry_datetime,
)
self.set_or_update_token(token)
return token
else:
raise Exception(
f"Failed to get or refresh token: {response.status_code} {response.content.decode()}"
)

def get_token_from_db(self) -> Union[AhjoToken, None]:
"""Get token from AhjoSetting table"""
try:
token_data = AhjoSetting.objects.get(name="ahjo_access_token").data
access_token = token_data.get("access_token", "")
refresh_token = token_data.get("refresh_token", "")
expires_in = token_data.get("expires_in", "")
return AhjoToken(
access_token=access_token,
refresh_token=refresh_token,
expires_in=datetime.fromisoformat(expires_in),
)
except ObjectDoesNotExist:
return None

def _check_if_token_is_expired(self, expires_in: datetime) -> bool:
"""Check if access token is expired"""
return expires_in < datetime.now()

def set_or_update_token(
self,
token: AhjoToken,
) -> None:
"""Save or update token data to AhjoSetting table"""

access_token_data = {
"access_token": token.access_token,
"refresh_token": token.refresh_token,
"expires_in": token.expires_in.isoformat(),
}

AhjoSetting.objects.update_or_create(
name="ahjo_access_token", defaults={"data": access_token_data}
)

def convert_expires_in_to_datetime(self, expires_in: str) -> datetime:
"""Convert expires_in seconds to datetime"""
return datetime.now() + timedelta(seconds=int(expires_in))
49 changes: 48 additions & 1 deletion backend/benefit/applications/services/ahjo_integration.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,14 @@

import jinja2
import pdfkit
import requests
from django.conf import settings
from django.core.exceptions import ObjectDoesNotExist
from django.db.models import QuerySet

from applications.enums import ApplicationStatus
from applications.models import Application
from applications.models import AhjoSetting, Application
from applications.services.ahjo_authentication import AhjoConnector
from applications.services.applications_csv_report import ApplicationsCsvService
from companies.models import Company

Expand Down Expand Up @@ -350,3 +354,46 @@ def export_application_batch(batch) -> bytes:

pdf_files: List[ExportFileInfo] = prepare_pdf_files(apps)
return generate_zip(pdf_files)


def dummy_ahjo_request():
"""Dummy function for preliminary testing of Ahjo integration"""
ahjo_api_url = settings.AHJO_REST_API_URL
try:
ahjo_auth_code = AhjoSetting.objects.get(name="ahjo_code").data
LOGGER.info(f"Retrieved auth code: {ahjo_auth_code}")
except ObjectDoesNotExist:
LOGGER.error(
"Error: Ahjo auth code not found in database. Please set the 'ahjo_code' setting."
)
return

connector = AhjoConnector(requests)

if not connector.is_configured():
LOGGER.warning("AHJO connector is not configured")
return
try:
ahjo_token = connector.get_access_token(ahjo_auth_code["code"])
except Exception as e:
LOGGER.warning(f"Error retrieving access token: {e}")
return
headers = {
"Authorization": f"Bearer {ahjo_token.access_token}",
}
print(headers)
try:
response = requests.get(
f"{ahjo_api_url}/cases", headers=headers, timeout=connector.timout
)
response.raise_for_status()
print(response.json())
except requests.exceptions.HTTPError as e:
# Handle the HTTP error
LOGGER.error(f"HTTP error occurred: {e}")
except requests.exceptions.RequestException as e:
# Handle the network error
LOGGER.errror(f"Network error occurred: {e}")
except Exception as e:
# Handle any other error
LOGGER.error(f"Error occurred: {e}")
Loading

0 comments on commit 332e471

Please sign in to comment.