-
Notifications
You must be signed in to change notification settings - Fork 1
Continuous Integration workflow
DevOps, which is a conjunction of Development and Operations, aims to reduce the differences between development and operations teams. It is built on the philosophy that one team should be able to build, maintain and improve the application with minimal reliance on outside expertise. This means that DevOps developers should not only know how to create code but also how to run their own code, monitor it, and know how to act if something goes wrong.
Automation is the name of the game. The more we can automate, the more time we can put into creating new and exciting functionality. Ideally, the complete path from a piece of finished code to its release into production is fully automated, including all the tests required to validate the code. Even maintenance and production issues are automatically resolved with redundant services, rolling updates, and automated fail-overs.
Continuous Integration is the part where we produce a piece of new functionality and integrate it within the application. We could just add it to the code, but who will tell us if the quality is up to par, if there are no bugs in it, and how do we keep the code bases consistent? These issues are solved by Continuous Integration. Before a piece of code is accepted, it goes through several steps, each checking if the code is of sufficient quality to be accepted. If any of these steps fail, the developer needs to fix the issues before they can resubmit their code. For the implementation of these steps, we use GitHub Actions to define the triggers and steps we need to enforce.
The following pipeline is enabled through GitHub actions. See our example workflow file to follow along.
The first step is relatively obvious: is the code buildable, can a compiler build the code without running into any issues? Besides being buildable, we also validate that all unit and integration tests run successfully. All logic should be tested by unit tests, both the expected flow (happy flow) as the corner cases. Generally, we expect a unit test coverage of 80% of the logic.
Buildable and tested code is not necessarily good quality code. Minimizing time spent on fixing bugs and technical debt is one of the goals of DevOps. One of the ways we reduce this is by enforcing strict code quality rules on newly committed code. We use SonarCloud for this, which contains hundreds of validation rules.
Each time a developer wants to merge new code into the trunk, it will check if the quality of the code is up to par. It immediately gives feedback on potential bugs or security issues. Only when all issues are resolved can the code be merged into the trunk and be released. By enforcing this at the door, we can be sure we produce high quality, secure code.
Each application within the DiSSCo infrastructure is a container. The code we produce is packaged into a container image, a blueprint of a container. Within our CI process, we test if our code can also be packaged as a container image.
The image can then be scanned by Trivy, which checks our code for any known vulnerabilities and exposures. While not every vulnerability may be immediately addressed, it is important to always be aware of issues and add user stories for any unresolved vulnerability.
While we use automated code review, peer reviews are essential aspects of our workflow. Each new piece of code is checked by another developer in the team. The peer reviewer checks if the code is consistent with the rest and if there are any issues present. The reviewer also checks if the build functionality conforms with their expectations. The developer could have interpreted the functionality differently; keeping expectations aligned is essential within the team. Peer reviews are also important to distribute the knowledge of the code within the team. Developers might rotate over pieces of functionality, so it is important that knowledge is shared within the team.
When all previous steps have been successfully completed, we can merge our new code into the trunk. We push the latest version of the code into our container image repository. The new image gets two tags: one indicating that this image contains the latest version of the code, and one tag based on the git commit (SHA) of the code. While the latest version will shift over time (each new merge to master creates a new latest version), the SHA tag is fixed to a moment in the trunk’s timeline.
The last step is connected to the image pushing. We only do this when the pull request is merged into the trunk. As we have now pushed two container images to our repository, we tag the trunk with the SHA of the container image. This way, we can always find which version of our code is in the container image. This is important if we want to release specific versions of our code or if we need to do a quick rollback to an earlier version.
This ends the Continuous Integration pipeline. We have checked our code through automated testing, code quality checks, and peer reviewers. It should now be high quality and conform to expectations within the team. The new functionality is now ready to be released.