Automation + VSO + Octopus =LOVE

Ciprian Filipaș
Senior Software Developer @ VE Interactive

PROGRAMMING

Awhile ago a dream was born. A dream forged from the deepest fears of the software development world. The dream of delivering pure, clean, quality code on TIME.

As the community was growing and the idea got seeded in the hearts of the people, a movement arose and so the CONTINUOUS DELIVERY concepts took shape.

Now, in all seriousness, as an aspirant software developer, getting you code to market in the shortest period possible, must be one of your main goals.

It may seem simple at first, you automate as many processes as necessary to reduce the time you wait for your colleagues to validate the "masterpiece" you wrote and to decide whether it will see the light of production.

Well, it may become a hard and frustrating piece of work if it's not done properly from the beginning.

The story

Keeping our mind on the goal and with our hearts filled with enthusiasm we started the race towards a better future. A process for Octopus was created to automate the deployment. With that one out of our way we wanted to tackle the challenge of running our Selenium based automated end-to-end test suite automatically.

Proposed Solution

Octopus Level

To achieve the desired solution a new step was created in our current deployment process (in Octopus) for triggering a new VSO Build. For this we used Microsoft's Visual Studio Team Services API.

All of this was achieved through a powershell script.

$basicAuth = ("{0}:{1}" -f $BuildUsername,$BuildPassword)
$basicAuth = [System.Text.Encoding]::UTF8.GetBytes($basicAuth)
$basicAuth = [System.Convert]::ToBase64String($basicAuth)
$headers = @{Authorization=("Basic {0}" -f $basicAuth)}

$uriQueue = 'https://veinteractive.visualstudio.com/' 
  +'DefaultCollection/'
  + $ProjectName
  + '/_apis/build/builds?api-version=2.0'

$definition = @{
  id = $BuildDefinitionId
}

$bodyQueue = @{
  definition = $definition
  sourceBranch = "refs/heads/" + $branchName
}

$jsonBodyQueue = ConvertTo-Json -InputObject $bodyQueue

$response = Invoke-RestMethod -Uri $uriQueue -ContentType 
  'application/json'
   -Body $jsonBodyQueue -Headers $headers -Method Post)

if($response)
{
  $buildId = $response.id

  $uriBuildStatus = 'https://veinteractive.visualstudio.com'
    + /DefaultCollection/' + $ProjectName + '/_apis/build/builds/'
    + $buildId + '?api-version=2.0'

  $buildStatus = Invoke-RestMethod -Uri $uriBuildStatus
    -Headers $headers | Select status, result
    -ErrorAction Stop

  if($buildStatus."status")
  {
    while ($buildStatus."status" -ne 'completed')
    {
      Start-Sleep -s 10
      $buildStatus = Invoke-RestMethod -Uri $uriBuildStatus
        -Headers $headers | Select status, result
        -ErrorAction Stop

    }
  }
  else
  {
    throw "Getting the Build Status Failed"
  }

if($buildStatus."result" -ne 'succeeded')
{
  write-output "WARNING: One of the build tasks failed. The release" 
    +"candidate is not valuable. Please check the output of the build."
}
else
{
  write-output 'EPIC SUCCESS - release candidate valid'
}
}

else
{
  throw "Error on build queue"
}

The template consists in a parametrized PowerShell script that triggers the build. The parameters are:

BuildDefinitionId - your build definition Id. In order to obtain this ID go on the build definition in VSO, hit edit, check the variables tab and you should find it under 'system.definitionId'
BuildUsername - the username for queueing the build. You must use alternate credentials forthis
BuildPassword - the password for theBuildUsername
ProjectName - the name of the VSOproject
BranchName - the name of the branch used forbuilding

Using these parameters, the script is creating a JSON object for the body and a header. These are then used for a REST Call towards the VSO API.

The REST Call is done via the Invoke-RestMethod cmdlet.

VSTS Level

The VSTS Build definition looks like this.

We build the testing solution sourcecode
We start the setup for theagent
- Install Chocolatey using a Command Line step using the following script:

@poweshell
-NoProfile
-ExecutionPolicy unrestricted
-command "(iex ((new-bject net.webclient)
    .DownloadString('https://chocolatey.org/install.ps1'))) >$null 2>$1"
    && SET PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin

Install Firefox via an inline Powershellscript

choco install firefox -version 46.0 -y --acceptlicense --force

Install NUnit via an inline Poweshellscript

choco install firefox -version 46.0 -y --acceptlicense --force

3.Running the tests using NUnitcommand. For this we implemented a parametrized Powershell script that calls the proper command with parameters provided at build time via VSTS.

Param(
  [string]$NUnitTestAssemblies, 
  [string]$Tags,
  [string]$NUnitLocation
)

$runNUnit = "& $NUnitLocation /labels /out=TestResult.txt /"
  +"xml=TestResult.xml /trace=Error /include='$Tags'"

$NUnitTestAssemblies.Split(",") | ForEach {
  $asm = $_.Trim()
  $runNUnit += " $asm"
}

Write-output $runNUnit

iex $runNUnit

The parameters are:

NUnitTestAssemblies - The assemblies for NUnit toconsider
Tags - The desired tags for NUnit to consider when selecting the tests to run

4.Publish the Test results via a Public Test Results VSTSStep

5.Publish the captured screenshots. This applies for failed testsonly.

6.Notify the users via ane-mail

For this we implemented a parametrized Powershell script.

Param([string[]]$DistributionList, [string]$BuildUrl, [string]$BuildNumber, [string]$UserSmtp, [string]$SmtpServerParam, [string]$StmpPword, [string]$Environment, [int]$SmtpPortParam)

Write-Output 'The Distribution list is: ' $DistributionList
Write-Output 'The Build Url is: ' $BuildUrl

$From = $UserSmtp
$To = $DistributionList
$Subject = 'Test Results for ' + $BuildNumber + ' ' + $Environment

#--------------------------------------------------------

$Body = 'This is my mail, my mail is amazing!'

#--------------------------------------------------------

$SMTPServer = $SmtpServerParam
$SMTPPort = $SmtpPortParam
$pword = ConvertTo-SecureString -String $StmpPword -AsPlainText -Force
$user = $UserSmtp
$Credential = New-Object -TypeName 
  System.Management.Automation.PSCredential($user,$pword)

Write-Output 'The subject is ' $Subject
Write-Output 'The body is ' $Body

Send-MailMessage -From $From
-To $To
-Subject $Subject
-BodyAsHtml
-Body $Body
-SmtpServer $SMTPServer
-port $SMTPPort
-UseSsl
-Credential $Credential

The parameters are:

DistributionList - The list of emails to witch the e-mail issent
BuildNumber - The buildnumber
SmtpUser - The SmtpUser
StmpPword - The password of theSmtpUser
SmtpServer - The Smtp ServeraddressEnvironment
SmtpPort - The SmtpPort

Conclusions

This solution is far from being perfect, however it solved some of our issues.

Thanks to this approach we do not need dedicated machines for test running. This results in a cost reduction and a more efficient use of resources since multiple agents can be hosted on the same machine and their management is governed by the VSO instance.

By using agents, we have a more consistent environment while the build definition is easy to configure and it's extremely flexible.

In terms of improvements, we want to notify the Octopus server when the build was done and based on the status we should either promote the deploy or revert it.