Getting some decent test reporting can be painful when working with Visual Studio and particularly with MS Test as the framework as newer versions of Visual Studio don’t appear to produce a *.trx file anymore. Using NUnit we can generate some fancy graphs and stats using the Allure framework with NUnit. Firstly we need to download and install NUnit: www.NUnit.org Secondly we need the Al...
This article is a follow up from my previous article in setting up Allure locally with NUnit. I made a brief mention of the Allure TeamCity plugin but encountered difficulties in getting it to work and didn’t have much time to look into getting around it.
Lets get stuck in with a brief recap/ some initial steps to take.
- Install NUnit 2.6.4 and the compatible NUnit adapter into your allure bin/addins folder
- Edit the config.xml in your bin/addins folder to write out the XML to a known location (I’m using C:\AllureXML)
- Run some tests in the GUI or NUnit Console and generate some XML, you should check XML is being generated before continuing
- Generate a report with the Command Line Interface (this will verify your JAVA_HOME system variable is present and correct)
Allure TeamCity Setup and Configuration
- Install the plugin and restart the build server
- Copy the allure-commandline.zip from latest release to the <TeamCity Data Directory>/plugins/.tools – No server restart needed for this step.
- Your build needs to be generating XML, we tested this earlier
- Open your build config settings, create a new build step for the Allure Report
- Specify a relative path to your XML folder (in this example it is ../../../AllureXML)
- Use the given example for test reports or similar
- Edit your build step running the tests – I have not been able to get it to work with the NUnit runner type (it just won’t generate any XML) however, we can run a script to spin up the NUnit Console and run it that way.
- Choose PowerShell as the runner, x86 bitness
- Script: Source code
- The script:
It is a good idea to test the script on the build machine locally so you can ensure it is correct, before putting it into your TeamCity build step.
Notice a couple of things:
- We run the x86 bit console as specified in the powershell run mode in the build step
- We have passed in “/framework=net-4.0”, this is important and in line with current documentation.
This should now, when run, kick off the tests via the console, generate the necessary XML and publish the artifacts in TeamCity giving you a report, which is great for retrieving daily summaries of builds you run overnight which is a great health check on your continuous integration builds.
Please keep these installation notes at the forefront of your mind if you encounter difficulties:
If you wish to add different parameters to your tests you can find a full list below or by running “nunit-console-x86.exe /help”
NUNIT-CONSOLE [inputfiles] [options]
Runs a set of NUnit tests from the console.
You may specify one or more assemblies or a single project file of type .nunit.
/fixture=STR Test fixture or namespace to be loaded (Deprecated) (Short format: /load=STR)
/run=STR Name of the test case(s), fixture(s) or namespace(s) to run
/runlist=STR Name of a file containing a list of the tests to run, one per line
/config=STR Project configuration (e.g.: Debug) to load
/result=STR Name of XML result file (Default: TestResult.xml) (Short format: /xml=STR)
/xmlConsole Display XML to the console (Deprecated)
/noresult Suppress XML result output (Short format: /noxml)
/output=STR File to receive test output (Short format: /out=STR)
/err=STR File to receive test error output
/work=STR Work directory for output files
/labels Label each test in stdOut
/trace=X Set internal trace level: Off, Error, Warning, Info, Verbose
/include=STR List of categories to include
/exclude=STR List of categories to exclude
/framework=STR Framework version to be used for tests
/process=X Process model for tests: Single, Separate, Multiple
/domain=X AppDomain Usage for tests: None, Single, Multiple
/apartment=X Apartment for running tests: MTA (Default), STA
/noshadow Disable shadow copy when running in separate domain
/nothread Disable use of a separate thread for tests
/basepath=STR Base path to be used when loading the assemblies
/privatebinpath=STR Additional directories to be probed when loading assemblies, separated by semicolons
/timeout=X Set timeout for each test case in milliseconds
/wait Wait for input before closing console window
/nologo Do not display the logo
/nodots Do not display progress
/stoponerror Stop after the first test failure or error
/cleanup Erase any leftover cache files and exit
/help Display help (Short format: /?)