May 14, 2015 // By Raja Gollamudi
This blog was written by Raja Gollamudi with contributions from Jeff Xiong.
SUMMARY: Fast-paced software development and delivery is the key for success in the current consumer driven era of IT. Agile processes, Continuous Integration (CI), and Continuous Deployment (CD) are major contributors in achieving fast-paced delivery. Maintaining a high quality is the most important piece of delivery. QA assistance with early automated tests, in CI, is the solution for achieving superior quality, called QACI.
Most development tools and frameworks improve developers’ efficiency by implementing automated processes, as well as imposing checks on the code. Continuous integration takes it a step further by finding any code integration issues by compiling the code as soon as the developer checks it in, and running unit tests. Build status and unit test results indicate the quality of the build. Continuous delivery automatically deploys the compiled code when all the unit tests are passed. Even though unit tests catch the initial errors in the code during the compiling, those tests are insufficient to catch the deployment and integration related errors. Quality Assurance in Continuous Integration (QACI) bridges the quality gap in the development pipeline.
The below diagram, of traditional CI, shows the flow chart of the development cycle. Traditionally, when developers check-in code, CI starts a build on a build server, and verifies build quality. The assessments of build quality, deployment to the QA environment, and to other environments are manual steps.
During the traditional CI approach, defects are not uncovered until a build is deployed to an environment. This means that defects may exist in the code for days or weeks before they are uncovered through manual testing.
The QACI approach shown in the diagram below extends the build process to automatically deploy on the QACI server and begin automated tests immediately.
QACI automated tests ensure the quality of a build, by running configured smoke and integration tests on a deployed environment. Unlike unit tests, QACI tests use a realistic test environment, database and other integrated components. QACI tests help find the defects as soon as they are introduced. Any bugs found during the QACI tests can be addressed by the development team. The code then follows the QACI path again. A build log gets updated with test results, indicating whether or not the build is ready for further exploratory tests.
QACI starts with simple smoke tests in the beginning of the project, and evolves to several test suites, touching the high risk areas of an application. As long as time is available, there is no limit to the amount of QACI tests; these can be expanded to a whole battery of test suites to cover all of the application. In fact, a daily run of a complete regression suite is recommended overnight or during off hours of development.
QACI Automated tests
QACI Automated test are the most important part of any CI/CD process. Efficient and effective automated tests which provide an excellent coverage, without false positive results, are the key factors in successful CI/CD process.
Automated tests are usually divided into multiple “suites”, each with their own objective. The list below gives a small overview.
- Unit tests: This is the suite that is run first, often by developers, before they add their changes to the repository. Unit tests normally test individual classes or functions. When those classes or functions need access to external resources, those resources are often provided as “mocks” or “stubs”.
- Integration tests: This is the next level up from the unit tests. Integration tests ensure all the modules of an application work properly with each other. They are normally run in a production like environment. Often times it is a clone of the production environment using similar database and integration endpoints.
- Regression tests: These are the type of software tests that seek to uncover new software bugs, or regressions, in existing functional and non-functional areas of a system after changes such as enhancements, patches or configuration changes, have been made to them.
QACI Test Environment
QACI environment is equally important as the quality of the tests:
In order to avoid erroneous test results, automated tests should run in clean environments, every time. Ideally, tests should start from a clean slate by resetting the test environment, restoring a virtual machine snapshot, restoring backups of databases, etc.
QACI tests should run in an environment that is as close to production as possible. Ideally, a clone of the production environment is preferred.
Conclusion
Implementing QACI enables software teams to get information more quickly and achieve their quality goals faster. Selection of the right tools and tests as well as optimizing test suites are key elements in success of QACI.
Do you want to know more about QACI? Want to setup QACI for your projects? We’d love to chat! Give us a call at 877-277-1044 or email us!