Approaches to test content

Test content

Testers use test content to test the releases of a Bloomreach Experience Manager application. This article describes the different approaches to creating and storing test content in a test environment. 

Bootstrap test content

A simple but effective approach is to use a separate bootstrap Maven module in your project. The bootstrap content is built using a Maven profile. The test data is bootstrapped into the development or test environment. This works only if the test environment does not contain content and configuration. This approach is typically used during the sprint phases of new projects. It allows project teams to deploy rapidly without complicated updates and upgrades. When testers create new test content in a test environment, the test content is exported using the CMS console and merged back into the bootstrap test content module. This way the test content is released together with the new release. 

Test database

Another approach is to make a test database with a good coverage of document types and content. This give full control over the tests and gives reliable results. The testers update the test database after a new release is deployed. This approach is useful for the maintenance of existing projects. The advantage of this approach is that updates and upgrades of Bloomreach Experience Manager releases can be tested.  

Using a backup 

At the start of each sprint or test cycle a backup of the test database is made. Each deployment during that sprint uses a copy of that database backup. After the sprint a new backup is made of the database. 

The steps are

1. At the start of the sprint make a backup of the test database.  
2. During the sprint several deployments are executed. Each deploy uses a copy of the backup.

This approach is important for testing update and upgrade scenarios because it mitigates the risks associated with so-called incremental releases. What are these risks ?

Incremental releases

First developers develop with local test data in a development environment. At the end of the development cycle, the development team makes a release, which is deployed on the test server. The release is verified and tested and usually several fix releases are made with bug fixes on test.  After the approval of the release, the release is deployed in the acceptance environment and again tested and verified, usually by customers and business users. Then the process of bug fixing is repeated and usually one or more releases are made and deployed in both the test and acceptance environments. After final approval the release is deployed to the production environment.

This process of incremental releases is useful for developers and testers to iterate fast during a sprint. But the development team can lose track of the exact configuration changes that were introduced during a sprint. This is especially true if several sprints only used the test environment before a deployment is done in the acceptance environment. It is difficult to spot dependencies between the successive releases. When the acceptance environment receives a new release which spans several intermediate or incremental releases, the deployment can have unexpected results.

The remedy to this risk is to create a snapshot database for each new version in the acceptance environment, and use that database as start for each incremental release in the test environment. This assures that releases are deployed and tested in an environment with the same release span as in the acceptance environment.

What if a test environment contains valuable data?

With the approaches explained in this chapter content introduced by users or testers will be overwritten after the deployment. The test database or bootstrap content overwrites the content.

Examples:

• Testers use test content which need to be preserved
• Trainings or demonstration are done in acceptance environments
• Environments are used for security or other audits
• Environments are used for content migration

How to handle this ?

It is recommendable to use test and acceptance environments only for testing of deployments. If possible create a new, stable environment for training or demonstration or other purposes.

Test content of testers must follow the practices described. The content created by testers can be exported from the console and stored into the test database or bootstrapped as test content.