Improve unit test coverage

[pscheid2]

We currently have relatively low unit test coverage of our backend python code. This makes it very difficult to ensure that changes we are making don't break existing app functionality, or that new functionality behaves the way we expect.

We should add some unit tests using the django.test package, to improve test coverage.

Once we have some coverage we can work on setting up a CI pipeline that would run the tests on each PR, and could also report on overall test coverage.

Current test coverage

Task

• choose a tool to audit Python/Django project test coverage
• audit CiviWiki project code with chosen tool
• report test coverage in this thread
• determine where to focus initial testing efforts
• add tests and measure impact on test coverage audit

Initial report

I plan on following this guide, but am open to suggestions of alternative unit test frameworks.

The deliverable for this ticket is to set up the test framework and write one test. Additionally the assignee should create another issue that outlines a plan to increase unit test coverage across the project.

[brylie]

Sounds good. Since this might be a bit of an epic, let's create a checklist in the issue description to serve as a roadmap. The checklist can be split off as separate tasks, if needed.

[wassafshahzad]

Can I just start or would you assign this to me please.

[brylie]

@wassafshahzad as requested, I assigned you. Please create a checklist of models and views without test coverage so we can tackle this issue as an epic. For models, we don't need to unit test Django core functionality, such as whether field definitions work correctly, so just add models where we have added or shadowed methods. For views, use your discretion as to what needs to be covered by unit tests and what those tests should look like (i.e. what conventions we should follow.)

[wassafshahzad]

Thank you for assigning me the issue I will start working on it.

[gorkemarslan]

@brylie where can I find the progression of unit testing? Did you specify conventions for testing, as you mentioned? I want to contribute to the project with testing and coverage.

[brylie]

I believe we should follow the basic conventions outlined in the Testing in Django documentation. Perhaps we can start with a test coverage auditing tool to get a sense of where tests are most urgently needed.

[gorkemarslan]

First, we need to create community chat, as discussed in issue #921. I want to contribute to the project. Nonetheless, I see a lot of issue that is assigned to someone, but it's been inactive for a long time, just like this thread. Now I try to understand the project itself, then we can collaborate on the project. :)

[brylie]

First, we need to create community chat, as discussed in issue #921.

Let's keep these issues decoupled and discuss the community chat in #921

[brylie]

Related to this task, I added a "Task" list to this issue description with some concrete steps to move forward.

I really appreciate your interest in helping out with CiviWiki development 😃

[gorkemarslan]

I use coverage package that is also specified in the requirements.txt file. I am sharing coverage report here:

Actually, it seems a little bit confusing so if you want I can add HTML templates from coverage to examine carefully by creating a new branch to commit HTML files from coverage.

I should also add that ProPublicaAPI is not present in the requirements.txt so the whole project gives an error and does not run. I discarded test_propublica.py file to launch the project.

aloe and selenium should be also added to requirements.txt, since they are Python packages I easily installed them.

[brylie]

Thanks for sharing the audit report, @gorkemarslan. Where would you suggest initial improvements be made? Also, would you be willing to open a pull request that improves test coverage of one of those files?

[wassafshahzad]

sorry I was inactive on this task, I am currently looking over another issue. Could we divide this into multiple sub tasks on the bases of apps ?

[gorkemarslan]

@brylie I would like to get started with accounts app. As far as I see, it needs more maintenance and unit testing. Also, durability of user model is more vital than the other parts at the first step because other applications (will) utilize user model. If the user model is broken down, then other applications won't work properly.

Another point, we should separate requirements.txt file into requirements/base.txt, requirements/dev.txt, requirements/base.txt and requirements/test.txt. For example, we need flake8 package for the development and testing steps, not for the production. The issue attracts my attention because master branch is not active and because I will be working on testing and development.

Maybe we should start a new issue for the maintenance?

[brylie]

I would like to get started with accounts app.

That sounds like a great place to start.

we should separate requirements.txt file into requirements/base.txt, requirements/dev.txt, requirements/base.txt and requirements/test.txt. For example, we need flake8 package for the development and testing steps, not for the production. The issue attracts my attention because master branch is not active and because I will be working on testing and development.

Maybe we should start a new issue for the maintenance?

Yes, great idea! Please open a separate issue so we can track that work independently of the unit testing.

[gorkemarslan]

Right now, I am dealing with creating a new issue for it. Then we can discuss tasks and steps in there.

[gorkemarslan]

The last coverage report is:

[brylie]

Let's get the test coverage report to run as part of pull requests. What do you recommend?

[wassafshahzad]

We could use github actions for that
PS
Any issue that needs Working on ?

[gorkemarslan]

This link might help us.

Or we can add pre-commit to the project that gives coverage report into a text file before each commit. GitHub action tools will be much better compared to pre-commit.

[gorkemarslan]

@brylie, is it possible to pin one of my test coverage reports to the top of this issue? I can constantly upload the coverage report when a new PR improving test coverage until a desired result is achieved?

[brylie]

@gorkemarslan create a new discussion thread and paste your coverage report there:
https://github.com/CiviWiki/OpenCiviWiki/discussions

You can then edit your comment as the coverage changes.

[izzzabellay]

Hello, I would like to take a shot at this. Do we still need more test cases to increase coverage?
I am fairly new to open source. Any additional information or guidance would be appreciated.