DevOps is a methodology in software development and release that aims to increase the speed and efficiency at which software is released. This is achieved by making sure that the developers and the operations team work together, automate away whatever possible and through shared responsibilities.
Before we even begin to talk more about DevOps, we need to know what was there before DevOps. What are the problems with the methods of the past
They write code. These includes, but not limited to:
- Bug fixes
- New features
- Security updates
They manage the servers where the app is running.
- They make sure that the servers are up and running
- They patch the server, update the OS, manage firewall etc
- They release new code given by the developers into the servers
- There are different phases in this model, each dependent on the output of the previous
- Once you are past a phase, you cannot go back, like a waterfall
- For example, you cannot change the design in the implementation stage
- This meant that there had to be precise planning and the requirements has to be perfect
- A good analogy is constructing a house. You cannot change the design once you're half way with building the roof, right?
Absolutely not flexible
Let's just say that there were a bunch of different models that tried to fix the issues with the waterfall model. For example
- Rapid Application Development
- Dynamic Systems Development Method
- Extreme Programming But let's not concern ourselves with these
Read the agile manifesto here
The TL;DR was that, the development process should be flexible such that "good" changes are always welcome. If the team came up with a really great idea, even towards the end of the "development", it was to be welcomed
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, according to Scott Ambler:
- Tools and processes are important, but it is more important to have competent people working together effectively.
- Good documentation is useful in helping people to understand how the software is built and how to use it, but the main point of development is to create software, not documentation.
- A contract is important but is no substitute for working closely with customers to discover what they need.
- A project plan is important, but it must not be too rigid to accommodate changes in technology or the environment, stakeholders' priorities, and people's understanding of the problem and its solution.
To keep things simple and in context, let's just take two scenarios. Pre-DevOps and DevOps
Don't quote me on it, I am just trying to explain what "DevOps" is fixing
Imagine that there is a
gitrepository with the code and themasterbranch is the mainline And the developers work on their own branches when making a change
- The Developer wrote the app
- They do some tests locally
- The changes are NOT merged to the master until a certain point in time. They are usually large changes
- The integration to the main branch is done on designated time (let's say we do it once a week)
- The developer has to make sure that their changes do not have any conflict, fix any.
- After a lot of mental gymanstics, the code is merged
- Now we wait until the "deploy day"
- The operations team has to make sure that the new code will work on production. They may need to update dependencies etc
- Finally the deploy day comes and the operations team deploys it manually. By running some hacked together deploy script
- If there are issues, the Operations team has to either rollback the change
- The operations team has to make sure that the service is healthy, add more servers if needed etc
- Integrating large changes once a while is more error prone. Imagine a scenario where the
developer
D1started working on a feature if done fully adds 10,000 new lines of codes. Meanwhile, developerD2is working on another bug fix that changed a lot of lines. Towards the integration of both of these to the master, it is going to be such a pain in the neck to make sure that the changes do not break things - The developer's environment is different from the production, which causes:
- Lots of errors make their way into production
- The deployments are error prone
- Everything is error prone because humans are unreliable
Now, I am gonna talk about an ideal DevOps scenario. One can dream, right?
- Developer writes code and ther are tests written for all the major functionalities. Ideally, ther are tests for all the functionalities
- Developer makes smaller, incremental changes and pushes to their branch
- A "tool" runs some automated tests to make sure that the developer did not break anything
- If the "tool" says all good, the code is merged to master.
- This means, multiple changes are merged to the master regularly (Continuous Integration)
- The "tool" builds the software and is ready to deploy any time (Continuous Delivery)
- In some cases, the "tool" does not wait for any particular time, instead it deploys to production regularly (Continuous Deployment)
- While in production, the servers can scale themselves based on the traffic pattern, which means less headache for the "Ops"
- Considerably easier integration (due to frequent integrations)
- We catch much more bugs before they get to production (due to automated tests)
- If something breaks, it is easy to identify and fix because the amount of change is smaller
- No more "deploy day headaches" for the ops team.
- New features gets released to the customers much faster and the team can get feedback on the changes and fix/update as needed with minimal delay
- Developers merge their changes back to master as soon as possible. This often means several "merge to master" per day.
- Makes use of automated tests. That is, the change is merged to the main branch only if all the tests succeed
It means, we build our software as soon as it is merged and we are ready to deliver the software at any time
We actually go ahead and deploy it as soon as the software is built by the tool
- Everything should be in
git
- Automated code testing (Selenium, Jenkins)
- Automated integration (Jenkins)
- Automated deployments (Jenkins
- Automate launching of infrastructure (Terraform)
- Automate the servers themselves (Ansible, Puppet, Chef)
- Whatever is possible to automate, should be automated (ideally)
- All new changes are made in small increments that are tested and integrated before continuing
No more works on my machine excuse
- Metrics like response time helps us in knowing if a change made our app slower and we can look into it to fix it (Prometheus, Grafana)
- Monitor logs (ELK)
- Thirdparty application performance monitoring tools (NewRelic, Datadog)