Niels van Kampenhout & Eduardo Perez

Sep 24, 2019

Set up Continuous Deployment of your Experience Manager Project in Bloomreach Cloud using Jenkins

With the Bloomreach Cloud API, setting up continuous deployment of your brXM implementation is a straightforward process. This tutorial will get you on your way.

 

Goal

Set up a pipeline in Jenkins that will continuously deploy a brXM project to a brCloud environment anytime changes are pushed into a specific branch in the project’s Git repository.

 

Approach

You will add a Jenkins deployment script to the root of your brXM project’s Git repository and set up a pipeline in Jenkins that pulls the script from Git and executes it. The script uses Maven to compile, test, and package the application, and uses the brCloud API to upload and deploy the distribution.

The brCloud API documentation is available on your stack at https://api-<stack>.onehippo.io/v3/docs. (replace <stack> with the name of your stack).

 

Prerequisites

  • A brXM enterprise project that was created using the Maven archetype and is stored in a Git repository

  • Read access to the brXM enterprise Maven repository

  • Admin access to a brCloud stack with an environment already created

  • Admin access to a Jenkins instance

 

Configure Jenkins

Make sure the following plugins are installed (Manage Jenkins > Manage Plugins):

  • Pipeline Utility Steps

  • Pipeline Maven Integration

  • Config File Provider

Make sure you have configured a Maven installation (Manage Jenkins > Global Tool Configuration > Maven) with a suitable Name (in our example, we assume ‘M3’) and the correct MAVEN_HOME path.

Add credentials (Credentials > System > Global Credentials) for:

  • Your brCloud stack (use your username and password and set ID to ‘brCloud_admin’ (the deployment script will reference this ID)

  • Your Git repository

Add a new config file (Manage Jenkins > Manage files > Add a new Config). Choose Global Maven settings.xml. Enter the ID ‘maven-id-bloomreach-cloud’ (the deployment script will refer to this ID), a name, and in the Content field enter your Maven configuration to make use of the brXM Enterprise Maven Repository.

 

Add a Jenkins Pipeline

Create a new pipeline in Jenkins (New Item > Pipeline). Choose a name without any spaces, for example "MyProject".

On the Pipeline tab, select ‘Pipeline script from SCM’ as Definition and ‘Git’ as SCM. Specify the Git repository URL, the credentials you created in the previous step, and the branch to pull from.

In the Script Path field, enter ‘Jenkinsfile.deploy’. This will be the deployment script.

 

Write a Basic Script to Access the brCloud API

Add a file called ‘Jenkinsfile.deploy’ to the root of your brXM project. Before you write the full deployment script, start with some basic steps to make sure you can access the brCloud API:

import groovy.json.JsonSlurper

pipeline {
   agent any

   environment {

       // Setup variables for the deployment
       brc_stack = "mystack"
       brc_environment = "myproject"
       brc_url = "https://api-${brc_stack}.onehippo.io"

   }

   stages {
       stage('Test Connection') {
           steps {
               script {
                   // Login to get the access token
                   echo "Login to brc and obtain token:"
                   withCredentials([usernamePassword(credentialsId: 'brCloud_admin', passwordVariable: 'brc_password', usernameVariable: 'brc_username')]) {
                       def json = "{\"username\": \"${brc_username}\", \"password\": \"${brc_password}\"}"
                       loginResult = post("${brc_url}/v3/authn/access_token", json)
                   }
                   echo "Login result ${loginResult}"
                   String access_token = "Bearer " + parseJson(loginResult).access_token

                   // Get the environment ID
                   echo "Get the environments"
                   environments = get("${brc_url}/v3/environments/", access_token)

                   // We require an existing environment. Alternative is to delete/create one
                   def environmentID = getEnvironmentID(environments, brc_environment)
                   echo "Environments result: ${environments}"
                   echo "Environment ID: ${environmentID}"
               }
           }
       }
   }
}

@NonCPS
private String get(url, access_token = null) {
   return curl("GET", url, access_token)
}

@NonCPS
private String post(url, json, access_token = null) {
   return curl("POST", url, access_token, json)
}

@NonCPS
private String curl(method, url, access_token, json = null, fileName = null, file = null, extraParams = null, contentType = "application/json") {
   return sh(script: "curl ${extraParams?:""} \
           -X ${method} '${url}' \
           ${access_token?"-H 'Authorization: ${access_token}'":""} \
           -H 'Content-Type: ${contentType}' \
           ${json?"-d '${json}'":""} \
           ${(fileName && file)?"-F '${fileName}=@${file}'":""}",
           returnStdout: true)
}

@NonCPS
def parseJson(text) {
   return new JsonSlurper().parseText(text)
}

@NonCPS
def getEnvironmentID(environments, brc_environment) {
   result = null
   parseJson(environments).items.each() { env ->
       if(env.name.toString() == brc_environment) {
           result = env.id
       }
   }
   return result
}

Make sure that the following variables in the script match your situation:

  • brc_stack: the name of your brCloud stack.

  • brc_environment: the name of your brCloud environment

  • credentialsId: the ID of the brCloud credentials that you created in the Configure Jenkins section.

The script contains some helper methods and performs the following steps:

  1. The withCredentials statement gets the username and password of the credentials you created and injects them as variables in the script.

  2. Using those variables, a JSON object is created to use as the payload for the authentication endpoint:
    post("${brc_url}/v3/authn/access_token", json)

  3. The token is extracted from the response and stored in a variable.

  4. Using the token for authentication, a list of environments in the current stack is retrieved:
    get("${brc_url}/v3/environments/", access_token)

  5. The ID of the environment we are interested in is extracted from the response:
    getEnvironmentID(environments, brc_environment)

Execute the pipeline (Build Now) and let it run. Jenkins will retrieve and execute the script. Once the build completes, you should be able to find the environment ID in the build log.

 

Extend the Script to do a Full Deployment to brCloud

Now that you can access your brCloud environment through the API, you can complete your deployment script.

Divide the pipeline into the following stages:

  1. Compile

    stage('Compile') {
       steps {
           // Run the maven build
           echo "Build the project:"
           withMaven(
                   maven: 'M3',
                   options: [artifactsPublisher(disabled: true)],
                   mavenSettingsConfig: mavenSettingsID) {
               sh '$MVN_CMD clean compile -Pdefault'
           }
       }
    }

     

  2. Test

    stage('Unit test') {
       steps {
           // Run the maven build
           echo "Execute tests:"
           withMaven(
                   maven: 'M3',
                   options: [artifactsPublisher(disabled: true)],
                   mavenSettingsConfig: mavenSettingsID) {
               sh '$MVN_CMD test -Pdefault'
           }
       }
    }

     

  3. Package

    stage('Package') {
       steps {
           // Run the maven build
           echo "Package the distribution:"
           withMaven(
                   maven: 'M3',
                   options: [artifactsPublisher(disabled: true)],
                   mavenSettingsConfig: mavenSettingsID) {
               sh '$MVN_CMD verify && $MVN_CMD -P dist'
           }
       }
    }

     

  4. Upload

    stage(Upload) {
       steps {
           script {
               withCredentials([usernamePassword(credentialsId: 'brCloud_admin', passwordVariable: 'brc_password', usernameVariable: 'brc_username')]) {
                   loginResponse = login("${brc_url}/v3/authn/access_token", brc_username, brc_password)
               }
    
               access_token = "Bearer " + parseJson(loginResponse).access_token
               refresh_token = parseJson(loginResponse).refresh_token
    
               String projectName = readMavenPom(file: "${workspace}/pom.xml").getArtifactId()
               String projectVersion = readMavenPom(file: "${workspace}/pom.xml").getVersion()
               String distribution = "target/${projectName}-${projectVersion}-distribution.tar.gz"
               echo "Upload the distribution ${distribution}"
               uploadResult = postMultipart("${brc_url}/v3/distributions/", "dist_file", "${workspace}/${distribution}", access_token)
               echo "Upload result: ${uploadResult}"
               distID = parseJson(uploadResult).id
               echo "distID: ${distID}"
           }
       }
    }

     

  5. Deploy

    stage('Deploy') {
       steps{
           script{
               if (refresh_token) {
                   access_token_valid = verify_token("${brc_url}/v3/authn/verify_token", access_token)
                   if (!access_token_valid) {
                       access_token = refresh_token("${brc_url}/v3/authn/refresh_token", refresh_token)
                   }
               } else {
                   withCredentials([usernamePassword(credentialsId: 'brCloud_admin', passwordVariable: 'brc_password', usernameVariable: 'brc_username')]) {
                       loginResponse = login("${brc_url}/v3/authn/access_token", brc_username, brc_password)
                   }
                   access_token = "Bearer " + parseJson(loginResponse).access_token
                   refresh_token = parseJson(loginResponse).refresh_token
               }
    
               echo "Get the environments"
               environments = get("${brc_url}/v3/environments/", access_token)
               echo "environments: ${environments}"
    
               def environmentID = getEnvironmentID(environments, brc_environment)
               echo "Environment ID: ${environmentID}"
    
               // Deploy the distribution to the environment
               echo "Deploy distribution"
               json = "{\"distributionId\": \"${distID}\", \"strategy\": \"stopstart\"}"
               deployResult = put("${brc_url}/v3/environments/${environmentID}/deploy", json, access_token)
               echo "Result of deploy: ${deployResult}"
           }
       }
    }

     

Note how in the Deploy stage, the validity of the access token is verified and, depending on the result, the token is refreshed or a new authentication is performed. This is just in case the upload takes longer than the time it takes for the token to expire (10 minutes).

Putting all the pipeline stages together with the helper methods, the complete script looks like this:
 

import groovy.json.JsonSlurper

pipeline {
   agent any

   environment {
       mavenSettingsID = 'maven-id-bloomreach-cloud'

       // Setup variables for the deployment
       brc_stack = "mystack"
       brc_environment = "myproject"
       brc_url = "https://api-${brc_stack}.onehippo.io"

       deployed_site = "https://${brc_environment}-${brc_stack}.onehippo.io/site/"
       deployed_cms = "https://${brc_environment}-${brc_stack}.onehippo.io/cms/"

   }

   stages {
       stage('Prepare') {
           steps {
               echo "=== Build Environment ================"
               sh "java -version && javac -version"
               echo "======================================"
           }
       }
       stage('Compile') {
           steps {
               // Run the maven build
               echo "Build the project:"
               withMaven(
                       maven: 'M3',
                       options: [artifactsPublisher(disabled: true)],
                       mavenSettingsConfig: mavenSettingsID) {
                   sh '$MVN_CMD clean compile -Pdefault'
               }
           }
       }
       stage('Unit test') {
           steps {
               // Run the maven build
               echo "Execute tests:"
               withMaven(
                       maven: 'M3',
                       options: [artifactsPublisher(disabled: true)],
                       mavenSettingsConfig: mavenSettingsID) {
                   sh '$MVN_CMD test -Pdefault'
               }
           }
       }
       stage('Package') {
           steps {
               // Run the maven build
               echo "Package the distribution:"
               withMaven(
                       maven: 'M3',
                       options: [artifactsPublisher(disabled: true)],
                       mavenSettingsConfig: mavenSettingsID) {
                   sh '$MVN_CMD verify && $MVN_CMD -P dist'
               }
           }
       }
       stage(Upload) {
           steps {
               script {
                   withCredentials([usernamePassword(credentialsId: 'brCloud_admin', passwordVariable: 'brc_password', usernameVariable: 'brc_username')]) {
                       loginResponse = login("${brc_url}/v3/authn/access_token", brc_username, brc_password)
                   }

                   access_token = "Bearer " + parseJson(loginResponse).access_token
                   refresh_token = parseJson(loginResponse).refresh_token

                   String projectName = readMavenPom(file: "${workspace}/pom.xml").getArtifactId()
                   String projectVersion = readMavenPom(file: "${workspace}/pom.xml").getVersion()
                   String distribution = "target/${projectName}-${projectVersion}-distribution.tar.gz"
                   echo "Upload the distribution ${distribution}"
                   uploadResult = postMultipart("${brc_url}/v3/distributions/", "dist_file", "${workspace}/${distribution}", access_token)
                   echo "Upload result: ${uploadResult}"
                   distID = parseJson(uploadResult).id
                   echo "distID: ${distID}"
               }
           }
       }
       stage('Deploy') {
           steps{
               script{
                   if (refresh_token) {
                       access_token_valid = verify_token("${brc_url}/v3/authn/verify_token", access_token)
                       if (!access_token_valid) {
                           access_token = refresh_token("${brc_url}/v3/authn/refresh_token", refresh_token)
                       }
                   } else {
                       withCredentials([usernamePassword(credentialsId: 'brCloud_admin', passwordVariable: 'brc_password', usernameVariable: 'brc_username')]) {
                           loginResponse = login("${brc_url}/v3/authn/access_token", brc_username, brc_password)
                       }
                       access_token = "Bearer " + parseJson(loginResponse).access_token
                       refresh_token = parseJson(loginResponse).refresh_token
                   }

                   echo "Get the environments"
                   environments = get("${brc_url}/v3/environments/", access_token)
                   echo "environments: ${environments}"

                   def environmentID = getEnvironmentID(environments, brc_environment)
                   echo "Environment ID: ${environmentID}"

                   // Deploy the distribution to the environment
                   echo "Deploy distribution"
                   json = "{\"distributionId\": \"${distID}\", \"strategy\": \"stopstart\"}"
                   deployResult = put("${brc_url}/v3/environments/${environmentID}/deploy", json, access_token)
                   echo "Result of deploy: ${deployResult}"
               }
           }
       }
   }
}

private String login(url, brc_username, brc_password) {
   echo "Login and obtain access token:"
   def json = "{\"username\": \"${brc_username}\", \"password\": \"${brc_password}\"}"
   loginResult = post(url, json)
   echo "Login result ${loginResult}"
   return loginResult
}

private boolean verify_token(url, access_token) {
    if (access_token) {
        echo "Verify access token:"
        verifyResult = get(url, access_token)
        echo "Verify result ${verifyResult}"
        if (parseJson(verifyResult).error_code) {
            echo "Token is invalid"
            echo "Error code: " + parseJson(verifyResult).error_code
            echo "Error detail: " + parseJson(verifyResult).error_detail
            return false;
        }
        echo "Access token is valid"
        return true;
    } else {
        echo "Access token is null"
        return false;
    }
}

private String refresh_token(url, refresh_token) {
    echo "Refresh access token:"
    def json = "{\"grant_type\": \"refresh_token\", \"refresh_token\": \"${refresh_token}\"}"
    refreshResult = post(url, json)
    echo "Refresh result ${refreshResult}"
    return "Bearer " + parseJson(refreshResult).access_token;
}
 

@NonCPS
private String get(url, access_token = null) {
   return curl("GET", url, access_token)
}

@NonCPS
private String post(url, json, access_token = null) {
   return curl("POST", url, access_token, json)
}

@NonCPS
private String postMultipart(url, String fileName, file, String access_token = null) {
   return curl("POST", url, access_token, null, fileName, file, null, "multipart/form-data")
}

@NonCPS
private String put(url, json, String access_token = null) {
   return curl("PUT", url, access_token, json, null, null, "-i --http1.1")
}

@NonCPS
private String  delete(url, access_token = null) {
   return curl("DELETE", url, access_token, null, null, null, "--http1.1")
}

@NonCPS
private String curl(method, url, access_token, json = null, fileName = null, file = null, extraParams = null, contentType = "application/json") {
   return sh(script: "curl ${extraParams?:""} \
           -X ${method} '${url}' \
           ${access_token?"-H 'Authorization: ${access_token}'":""} \
           -H 'Content-Type: ${contentType}' \
           ${json?"-d '${json}'":""} \
           ${(fileName && file)?"-F '${fileName}=@${file}'":""}",
           returnStdout: true)
}

@NonCPS
def parseJson(text) {
   return new JsonSlurper().parseText(text)
}


@NonCPS
def getEnvironmentID(environments, brc_environment) {
   result = null
   parseJson(environments).items.each() { env ->
       if(env.name.toString() == brc_environment) {
           result = env.id
       }
   }
   return result
}

Execute the pipeline (Build Now) and let it run. Once it finished successfully, log in to your brCloud stack’s Mission Control app to verify that the distribution is successfully deployed in the environment.

 

Configure the Pipeline for Continuous Deployment

Now that your pipeline is able to do a full deployment, all that is left to do is set up automatic continuous deployment. There are several different ways to do this, one of them is SCM polling:

Open your pipeline configuration (MyProject > Configure).

On the General tab, check GitHub project and enter the project URL.

On the Build Triggers tab, check Poll SCM and enter the following cron expression:

H/15 * * * *


 

Jenkins will now poll your Git repository every 15 minutes and execute the deployment pipeline if there are any changes.

 

Summary

You set up a basic Jenkins pipeline to build a brXM project and deploy it in a brCloud environment. In your deployment script, you pull the project source code from Git, use Maven to build, test, and package the project, and use the brCloud API to upload and deploy the distribution.

 

Next Steps

The deployment script you wrote is quite basic but it’s a good base to build additional functionality into your pipeline. For example, it could be adapted to incorporate functional testing or use blue-green deployment. Explore the brCloud API documentation (available at https://api-<stack>.onehippo.io/v3/docs) for all the possibilities.