This is the second part of my blog series on CI/CD best practices. For those of you who are new to this blog, please refer to Part 1 of the same series and for those who want to see the first 10 best practices. Also, I want to give a big thank you for all the support and feedback! In my last blog, we saw the first ten best practices when working with Continuous integration. In this blog, I want to touch on some more best practices. So, with that, let’s jump right in!
Best Practice 11 – Run Your Fastest Tests First
The CI/CD system serves as a channel for all changes entering your system and hence discovering failures as early as possible is important to minimize the resources devoted to problematic builds. It is recommended you to prioritize and run your fastest tests first and save the complex, long-running tests for later. Once you have validated the build with smaller, quick-running tests and have ensured that the build looks good initially, you could test your complex and long-running test cases.
Following this best practice will keep your CI/CD process healthy by enabling you to understand the performance impact of individual tests as well as complete most of your tests early. It also increases the likelihood of fast failures. Which means that problematic changes can be reverted or fixed before blocking another members’ work.
One of the version control best practices is integrating changes into the parent branch/shared repository early and often. This helps avoid costly integration problems down the line when multiple developers attempt to merge large, divergent, and conflicting changes into the main branch of the repository in preparation for release.
To take advantage of the benefits that CI provides, it is best to limit the number and scope of branches in your repository. It is suggested that developers merge changes from their local branches to remote branch at least once a day. The branches which are not being tracked by your CI/CD systems contain untested code and it should be discarded as it is a threat to the stable code.
Another best practice (and it is related to my earlier point about discovering failures early) is that teams should be encouraged to run as many tests locally prior to committing to the shared repository. This ensures that the piece of code you are working on is good and it will also make it easy to recognize unforeseen problems after integrating the code to the master code.
To be successful with CI/CD one needs a test suit to test every code which gets integrated. As a best practice it is recommended to develop a culture of writing tests for each code change which is integrate into the master branch. The test cases should include unit, integration, regression, smoke tests or any kind of test which covers the end to end project.
This is probably the most important part of any implementation. As a next best practice, always think of an easy way to roll back the changes if something goes wrong. Normally I have seen organizations doing a rollback via redeploying the previous release or redoing the build from the stable code.
I have seen organizations having multiple ways of CI/CD pipeline. It varies from using multiple tools to multiple mechanisms. Though not a very prominent one but as a best practice, I would say deploy the code same way every time to avoid unnecessary issues like configuration and maintenance across environments. If you are using same deploy method every time, then the same set of steps are triggered in all environments making it easier to understand and maintain.
Although the manual build and deployment can work, I would recommend you automate the pipeline. Automation to start with eliminates all the manual errors but further to that ensures that the development team only works on the latest source code from the repository and compiles it every time to build the final product. With lots of tools available like Jenkins, Bamboo, BitBucket etc it is very easy to automate the create the workspace, compile the code, convert it to binaries and publish it to Nexus.
It is a good practice to track the CI/CD pipeline success rate. You could measure these both ways, before & after you start automation and compare the results. Although the metrics for evaluating CI/CD success rate depends on organizations, some of the points to be considered are:
- Number of jobs deployed monthly/weekly/daily to Dev/TEST/Pre-PROD/PROD
- Number of Successful/Failure Builds
- Time is taken for each build
- Rollback time
Best Practice 19 – Backup
Your CI/CD pipeline has lot of process/steps involved and as the next best practice I recommend you take periodic backup of your pipeline. If using Jenkins, this can be accomplished via Backup Manager as shown below.
Best Practice 20 – Clean up CI/CD environment
Lots of builds can make your CI/CD system too clumsy and over a period of time and it might impact the overall performance. As the next best practice, I recommend you to clean the CI/CD server periodically. This cleaning could include CI pipeline, temporary workspace, nexus repositories etc.
And with this, I come to an end of the two-part series blog. I hope these best practices are helpful and you would embed these while working with CI/CD.