diff --git a/docs/care/CEP/Completed/0008-pdf-generation.md b/docs/care/CEP/Completed/0008-pdf-generation.md index f592602..49ae7cb 100644 --- a/docs/care/CEP/Completed/0008-pdf-generation.md +++ b/docs/care/CEP/Completed/0008-pdf-generation.md @@ -17,7 +17,7 @@ Currently, this function is responsible for generating PDFs, and the following i # Proposed Solution -### Initial Solution Approach +## Initial Solution Approach Initially, I considered utilizing a Python library like `WeasyPrint` or `xhtml2pdf`. Both libraries offer Django-friendly code. Here are the respective documentation links for further reference: - [Xhtml2pdf](https://xhtml2pdf.readthedocs.io/) @@ -25,7 +25,7 @@ Initially, I considered utilizing a Python library like `WeasyPrint` or `xhtml2p After conducting further research and receiving feedback, I came across a recent [article](https://zerodha.tech/blog/1-5-million-pdfs-in-25-minutes/) by Zerodha that discussed PDF generation. This is where I discovered Typst. -### Final Solution (Using Typst) +## Final Solution (Using Typst) Initially, PDFs were generated from HTML using Puppeteer, which involved spawning headless instances of Chrome. According to the Zerodha article, their earlier tech stack, similar to ours, relied on headless Chrome for PDF rendering, which proved inefficient at scale. However, they transitioned to Typst, noting its efficiency and scalability benefits. This sparked my interest, leading to extensive research validating Typst's effectiveness and credibility through various references. @@ -36,7 +36,7 @@ I engaged with the Typst community and moderators on Discord, where I received p By adopting Typst for our PDF generation needs, we can significantly improve the efficiency and scalability of our discharge report generation process, ensuring lower resource consumption and enhanced performance. -# Implementation Plan +## Implementation Plan - **Step 1:** Update Docker Files to Add Typst Dependencies @@ -55,117 +55,29 @@ By adopting Typst for our PDF generation needs, we can significantly improve the ### Step 1: Update Docker Files to Add Typst Dependencies -Since we don't have an apt installation for Typst, we can download it from the official releases according to the build we are working on. Here is how we can update our Dockerfile: - -```docker -FROM python:3.11-slim-bullseye - -ENV PYTHONUNBUFFERED 1 -ENV PYTHONDONTWRITEBYTECODE 1 - -ENV PATH /venv/bin:$PATH - -ARG TYPST_VERSION=0.11.0 - -RUN apt-get update && apt-get install --no-install-recommends -y \ - build-essential libjpeg-dev zlib1g-dev \ - libpq-dev gettext wget curl gnupg git \ - && apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \ - && rm -rf /var/lib/apt/lists/* - -# Download and install Typst for the correct architecture -RUN ARCH=$(dpkg --print-architecture) && \ - if [ "$ARCH" = "amd64" ]; then \ - TYPST_ARCH="x86_64-unknown-linux-musl"; \ - elif [ "$ARCH" = "arm64" ]; then \ - TYPST_ARCH="aarch64-unknown-linux-musl"; \ - else \ - echo "Unsupported architecture: $ARCH"; \ - exit 1; \ - fi && \ - wget -O typst.tar.xz https://github.com/typst/typst/releases/download/v${TYPST_VERSION}/typst-${TYPST_ARCH}.tar.xz && \ - tar -xf typst.tar.xz && \ - mv typst-${TYPST_ARCH}/typst /usr/local/bin/typst && \ - chmod +x /usr/local/bin/typst && \ - rm -rf typst.tar.xz typst-${TYPST_ARCH} - -# use pipenv to manage virtualenv -RUN python -m venv /venv -RUN pip install pipenv - -COPY Pipfile Pipfile.lock ./ -RUN pipenv install --system --categories "packages dev-packages" - - -COPY . /app - -RUN python3 /app/install_plugins.py - -HEALTHCHECK \ - --interval=10s \ - --timeout=5s \ - --start-period=10s \ - --retries=48 \ - CMD ["/app/scripts/healthcheck.sh"] - -WORKDIR /app - -``` +Since we don't have an apt installation for Typst, we can download it from the official releases according to the build we are working on. Updated Dockerfile : [dev.Dockerfile](https://github.com/ohcnetwork/care/blob/92f717eb15120cd5d73504acc52630d33dedba19/docker/dev.Dockerfile) ### Step 2: Update Helper Functions and Create Typst Wrapper -First of all we have to create a wrapper function to allow our typ binary to compile our template. +First of all we have to create a wrapper function to allow our typ binary to compile our template. The updated function - [compile_typ](https://github.com/ohcnetwork/care/blob/92f717eb15120cd5d73504acc52630d33dedba19/care/facility/utils/reports/discharge_summary.py#L202-L240) + +After creating a wrapper we can update our helper functions. Updated helper function - [generate_discharge_summary_pdf](https://github.com/ohcnetwork/care/blob/92f717eb15120cd5d73504acc52630d33dedba19/care/facility/utils/reports/discharge_summary.py#L242-L251) -```python -def compile_typ(output_file, data): - try: - content = render_to_string("reports/example.typ", context=data) - process = subprocess.run( - ["typst", "compile", "-", output_file], - input=content.encode("utf-8"), - capture_output=True, - check=True, - ) - logging.info( - f"Successfully Compiled Summary pdf for {data['consultation'].external_id}" - ) - return True - except subprocess.CalledProcessError as e: - logging.error( - f"Error compiling summary pdf for {data['consultation'].external_id}: {e.output.decode('utf-8')}" - ) - return False - except Exception as e: - logging.error( - f"Unexpected error compiling summary pdf: {e}" - ) - return False -``` -After creating a wrpper we can update our helper functions like : -```py -def generate_discharge_summary_pdf(data, file): - logger.info( - f"Generating Discharge Summary pdf for {data['consultation'].external_id}" - ) - compile_typ(output_file=file.name, data=data) - logger.info( - f"Successfully Generated Discharge Summary pdf for {data['consultation'].external_id}" - ) -``` ### Step 3: Create Static Template for Reports -Completed it. Progress can be seen here - [Report Template](https://typst.app/project/rBahq8CbsixUHYwri4KtmO) -Now this document just cointains different components that I've used in my template. +Static Report Template progress could be seen here - [Report Template](https://typst.app/project/rBahq8CbsixUHYwri4KtmO). + +Now this document just contains different components that I've used in the template. ### Step 4: Integrate Typst with Django Templates in Our Project -Updated the previous template using HTML/CSS with `Typst`. Template can be found [here](https://github.com/coronasafe/care/blob/5d5ca4630cebd168f3ca8a75ca2dae9bdc6110fd/care/templates/reports/patient_discharge_summary_pdf_template.typ) +Updated the previous template using HTML/CSS with `Typst`. Template can be at [patient_discharge_summary_pdf_template.typ](https://github.com/ohcnetwork/care/blob/92f717eb15120cd5d73504acc52630d33dedba19/care/templates/reports/patient_discharge_summary_pdf_template.typ) ### Step 5: Create Tests Generating `PNG` of the pdf using typst and comparing using `Pillow` library. It involves have sample png images of the pdf in `care/facility/tests/sample_reports` folder which are to be compared with the newly generated pdf pngs , if identical the test cases passes, else throws error. -To Update the sample `PNG` files, we can update the [test_compile_typ() function](https://github.com/ohcnetwork/care/blob/develop/care/facility/tests/test_pdf_generation.py#L41-L100) by adding the below code to test function below line 59. +To Update the sample `PNG` files, we can update the [test_compile_typ](https://github.com/ohcnetwork/care/blob/92f717eb15120cd5d73504acc52630d33dedba19/care/facility/tests/test_pdf_generation.py#L41-L100) function by adding the below code to test function below line 59. ```python subprocess.run( ["typst", "compile", "-", sample_file_path, "--format", "png"], @@ -175,14 +87,15 @@ subprocess.run( cwd="/", ) ``` -To investigate any errors, we can remove the [finally block](https://github.com/ohcnetwork/care/blob/develop/care/facility/tests/test_pdf_generation.py#L89-L100) from our [test_compile_typ() function](https://github.com/ohcnetwork/care/blob/develop/care/facility/tests/test_pdf_generation.py#L41-L100). It'll generate the `test_output{n}.png` files in `care/facility/tests/sample_reports` folder, from where you can use a image diff checker to investigate the differences. +To investigate any errors, we can remove the [finally block](https://github.com/ohcnetwork/care/blob/92f717eb15120cd5d73504acc52630d33dedba19/care/facility/tests/test_pdf_generation.py#L89-L100) from our [test_compile_typ](https://github.com/ohcnetwork/care/blob/92f717eb15120cd5d73504acc52630d33dedba19/care/facility/tests/test_pdf_generation.py#L41-L100). It'll generate the `test_output{n}.png` files in `care/facility/tests/sample_reports` folder, from where you can use image diff checker to investigate the differences. + +if in future we decide to add more data to the test function and the number of pages increases, then one should also update the [number_of_pngs_generated](https://github.com/ohcnetwork/care/blob/92f717eb15120cd5d73504acc52630d33dedba19/care/facility/tests/test_pdf_generation.py#L68) number to the number of pages of pdf generated. -if in future if we decide to add more data to the test function and the number of pages increases, then one should also update the [number_of_pngs_generated](https://github.com/ohcnetwork/care/blob/develop/care/facility/tests/test_pdf_generation.py#L68) number to the number of pages of pdf generated. ### Step 6: Remove All Previous Dependencies and Remove Chromium and django-hardcopy -Updated all the functions utilising the older dependecies with the newer versions and removed `django-hardcopy` from pipfile and `chromium` from docker file. +Updated all the functions utilising the older dependencies with the newer versions and removed `django-hardcopy` from pipfile and `chromium` from docker file. ### Step 7: Update production files for the changes -Updated prod.Dockerfile to remove older dependencies and added newer dependecies. +Updated prod.Dockerfile to remove older dependencies and added newer dependencies. # Updates in the template are listed below @@ -224,16 +137,17 @@ Updated prod.Dockerfile to remove older dependencies and added newer dependecies ### Others -> - Removed `Symptoms` and `Diagnosis (ICD-11)` tables and `Health Status at admission` section -> - Removed `Daily Round` section -> - Created three new `templatetags` , one to `format prescription`, one to `format_to_sentence_case` and one to `handle empty data` -> - Added conditions to update fields name according to admission status +- Removed `Symptoms` and `Diagnosis (ICD-11)` tables and `Health Status at admission` section +- Removed `Daily Round` section +- Created three new `templatetags` , one to `format prescription`, one to `format_to_sentence_case` and one to `handle empty data` +- Added conditions to update fields name according to admission status - - - -# Results +## Results - Typst is 8-15 times faster in generating PDFs - Container size decreased by 30% - Typst is 10-15% more memory efficient - We eliminate the overhead associated with browser-based rendering, resulting in a more efficient and scalable process. + +## Links +- **Link to PR** - https://github.com/ohcnetwork/care/pull/2132 +- **Link to Care repo** - https://github.com/ohcnetwork/care