Test Management

There are multiple issues that we are faced with while testing and verifying the application in an environment that involves parallel development. First and foremost is the availability of the test environments. Another issue is regarding the scope of testing to be done in the different releases.

A usual testing infrastructure would provide one system testing environment that mirrors the production environment as close as possible. But, here, in the case of parallel development, we could be faced with the task of testing different versions at the same time. This could occur due to the long testing times for adjacent releases. i.e. it could so happen that the next release enters the system testing phase before the previous one has left the ST phase. This could also come in the case when a hotfix is required to be developed and tested when a development release is under system testing. So, the main thing from an infrastructure point of view is to plan for multiple system testing environments. This would usually be number of releases that potentially could be in system testing phase in parallel plus one environment for verifying immediate hotfixes that need to be sent to production without waiting for any of the planned releases. So, at any point in time, there will be one environment in sync with the production version – ready for verifying any SOS deployments.

In the case of hotfixes disturbing the release schedule, it is important to have a strategy to test and verify the changes in the hotfixes as well as ensure the overall stability of the application. From the name itself, it is evident that the hot fix can’t wait for an elaborate regression testing exercise. First level of test planning would involve identifying / creating test cases / scenarios that are relevant to the functionality being modified / developed in the hotfix. Next would be identifying test cases from impacting modules to be added to the test plan of the hotfix. A dependency matrix could be of help in identifying these modules. It is advisable to maintain a threadbare test case that ensures the sanity of the build from the main functionalities perspective. This can be executed in lieu of the regression test suite to enable a quick verification of the overall stability of the product.

A strategy that can be employed in case of too many hot fixes (which inherently is not good though) is to ensure that all the hotfixes are tracked to closure only after a proper testing (including regression testing) of the codebase is done in a subsequent release to the hotfix deployment. This is beneficial not only from the perspective of the stability of the product, but also helps in tracking the quality of hot fixes that go into the codebase. There might be situations where the urgency of the fix forces us to release a kludge. Having a mechanism to track the hotfixes to a closure gives an opportunity to verify the hotfix made also.

There are various situations in which we, as development teams, are faced with the challenges around parallel development. In this post, let us look at some of the challenges around such a situation and how to tackle them efficiently.

One situation that causes parallel development work is supporting a product / application that has planned frequent releases. In such a case, there is a possibility that development of one release overlaps another. Another situation is that of addressing maintenance issues in the midst of an enhancement work. In such cases, there could be hotfix releases appearing unexpectedly in between the enhancement release plan.

The main challenges that come up are :

1. How do I maintain the source code for the two versions of the application being worked upon?
2. How do I test multiple versions of the application?

Code Management

The problems related to maintenance of different versions of source code can be addressed by the branching functionality available in most of the modern configuration management tools. This functionality provides the ability to maintain different versions of the codebase and manage changes in them. Typically, this feature enables maintaining a main version of the code named trunk and have branches to maintain other versions. However, just the choice of a configuration management tool that supports branching won’t help us. We need to anticipate parallel development and be ready for it.

We should be maintaining code corresponding to production version (or any other primary environment that we will be supporting) in the trunk and do any development work only in a new branch. This would always give the team quick access to the correct codebase in the trunk if at all there is need to work on any production hotfix. The common issue that we usually end up in is that the development is done on the trunk itself, anticipating no releases would go in before the one being worked on. A hot fix development would then lead to fetching old code from a label in the trunk or, picking up source code from a release package or, even worse, fishing out code from production to start work on the hotfix. So, the suggested way is to always have the trunk mapped to the version in production and perform development work in branches.

So, we branch out and do our development in the new branch and are prepared for parallel development. Now, if we need to do a hotfix, the team can fetch the latest from the trunk and work on it and checkin into the trunk itself and release a hotfix. We would do it this way unless the hotfix work itself is large enough and we anticipate a hotter fix to come around and get ahead of the current hotfix!!! In such a situation, we would need to do the hotfix also in a new branch and keep the path for the potential hotter fix as the trunk.

A question which naturally arises is what happens ultimately when the release being developed in the branch goes into production. There would be changes made to the version in trunk due to the hotfixes and changes to the version in the branch due to the development. Thus, we are faced with the challenge of creating a version to be sent to production which contains the hotfixes as well as the development done in the branch. This is done by merging the branch onto the trunk. There are merge features within CM tools to help us perform this. However, a manual intervention is required in the merge process to resolve conflicts that occur i.e. we need to manually make decisions about the version to be retained after merge in cases where same files have been changed in the trunk as well as the development branch. This is a critical process that needs to be given due importance and done very carefully. Irrespective of the number of conflicts a merge has generated, the newly merged version has to be tested afresh to verify the stability of the application.

To reduce the number of conflicts that would arise during a merge, it is a good practice to merge the trunk onto the active development branches periodically. i.e. the development branch is periodically updated with the changes happening in the trunk so that there are lesser issues while merging the branch during the development release into the trunk. A corollary of this good practice is to merge the trunk onto any other active development branches after a development branch has been merged to the trunk. The diagram below summarizes this.

In the next post, we will have a look at how to manage testing in a parallel development situation.

Take the case of a layout having a header, content, right area, left area, footer section etc. There are several methods to make each section dynamic within the same page. One such method is by using Categories & Posts feature of WordPress.

Home Page

Steps for creating a dynamic page

1. Create a page template : Create a template file say, “home.php” in your “/wp-content/themes/yourtheme directory”.
2. Create the page : Create a page without any content and assign the template created in the above step to the page.
3. Create category : Create a category for each section in the layout. For example, referring to the above image, create “Header” category for header contents, “Left-Content” category for left block contents, etc.
4. Create posts : Create content for each section using posts. For example, say the header section has a logo and a mission/vision text. Create a post having a title, say, “Header-Logo” and paste the logo image in the content area. Then assign the category “Header” to this post. Similarly a post title “Header-Text” can be created. You must note the “PostID” of each post you are creating.

After creating all the posts(content) for a particular page, you need to retrieve the post contents in your template file(home.php) created in the first step. For this, use the code below :

For the header section :

For the left content section :

Similarly retrieve content for right-section and footer area.

Drawback

The only difficulty with this method is with your site's blog functionality. Blog index page will show all the posts created irrespective of the categories. So if you are implementing a blog in your site, you must exclude the categories created for adding page contents from the list of blogs. Otherwise all posts in those categories will also be indexed in your blog page.

How to exclude multiple categories from blog pages (index.php, single.php, search.php, archive.php, etc)?

Use the code :

Benefits

• Continuous integration ensures lesser time to integrate the modifications by different developers.
• Clean central build / deployment / integration testing infrastructure ensures that there are no footprints of development machine involved in the product being tested
• Versioned and backed up codebase enables disaster recovery.
• Regular versioning helps in managing maintenance tasks and enhancements in parallel.
• Coding guidelines improves readability and enhances maintainability, in addition to identifying obvious low-performance code snippets.

System testing process

System testing is performed on an environment that is built as close as possible to the production environment using virtualization. System test cases are developed executed by test engineers on a “build” delivered by the development team. The bugs detected are logged, fixed and tracked to closure through subsequent builds from the development team. Deployment instructions are created and tested during deployment of system testing environment.

Benefits

• Early testing in an environment close to production helps unearth and solve issues early on.
• A virtualized ST environment helps in setting up the ST environment for maintenance purposes and testing the deployment instructions quickly.
• Formal release of “builds” by the development team to the testing team ensures delivery of a "tested build" that is traceable through version control.

Release management and deployment

A package that is already tested and certified by the testing team is chosen for release. Release log and release checklist ensures standard operating procedures are followed for the release and is traceable to a build and a label in the codebase repository.

Benefits

• Since release package has already been tested on the ST environment which is close to the staging / production environment, there is minimal risk of issues with deployment.
• Release packaging with deployment instructions and release notes makes the release self-sufficient and enables separation of development and deployment activities.
• With self-sufficient release packages, we can have common secured release location for multiple projects.
• Maintenance of release log ensures traceability of releases to codebase versions.
• Release checklist ensures basic features of a release package.

Development process and methodology