Configuring a TFS 2010 Team Build Server to Run Coded UI Tests

Development teams utilizing Coded UI Tests have several options where to run Coded UI Tests. When a developer creates an automated test either from an existing action recording or using the Coded UI Test Builder, the test can be run within Visual Studio 2010 just like a unit test. Once the Coded UI test is passing, it can be associated with a Test Case and/or run as part of the build along with the unit tests. I will cover associating a Coded UI Test to a Test Case and running the automated test within Microsoft Test Manager in a future post. In this post I will cover configuring the build server to run Coded UI Tests. While this post describes configuring the build server to run the Coded UI Tests, any machine configured as a test agent can be utilized by the build server and build to run the Coded UI Tests.

The primary difference between running the Coded UI Test on the build server than a unit test is that the test requires full access to the UI. Having the build server require full access to the UI introduces a couple challenges including a couple additional requirements. First, the build server requires the application to be installed on the build server the same way it was installed when the test was recorded. The build server also requires a Visual Studio 2010 Test Agent running on it. It also needs to be configured to record video if required. Finally the build has to be configured to run the Coded UI Tests. The following steps walk through configuring these items.

Configuring the Test Agent to Run Coded UI Tests

1. Run the Microsoft Visual Studio 2010 Test Agent Configuration Tool (Start > Program Files > Visual Studio 2010 > Visual Studio 2010 Test Agent Configuration Tool)

2. Configure the Test Agent to Run Interactive

3. Ensure that Log on automatically and Ensure screen saver is disabled are checked. This will enable the server to be ready to run the tests even if the server is rebooted. Also, the screen saver has to be disabled so the machine doesn’t get locked. The Coded UI Tests can not run if the screen is locked. If you use Remote Desktop (RDP) to connect to the server, when you log out the machine it will automatically lock it. To prevent this from happening, you must log into it from the machine or VM console. If this is not an option, an alternative is to log on to another server such as the TFS server, then RDP into the build server from the TFS server and close the RDP session to the TFS server to lock this server but the build server remains unlocked and the Coded UI Tests will be able to run.

4. Register the Test Agent to a Test Controller that is not configured for Test Manager. Unfortunately, the Coded UI tests run from the Build Server can not share the same test controller.

Enabling Video Recording for the Automated Tests

One of the options in the test settings is to enabling video recording as part of the data collection. By default the server doesn’t have the required components and configuration to enable this. Follow these steps to enable the video recording.

1. Install the RTM Update for Lab Management on the Build Server
http://www.microsoft.com/downloads/en/details.aspx?FamilyID=8406ef19-35a3-4c03-a145-08ba982f3cef&displaylang=en

2. Install Microsoft Expression Encoder
http://www.microsoft.com/downloads/en/details.aspx?displaylang=en&FamilyID=75402be0-c603-4998-a79c-becdd197aa79

3. If the Build Server is Windows Server then enable the Desktop Experience Feature.

Configuring the Build to run the Coded UI Tests

The final part to configure is to configure the build to run the Coded UI Tests.

1. Open the Test Settings file in Visual Studio by clicking Test > Edit Test Settings > Local.

2. Create a copy by clicking “Save As” and call it something like “BuildServer”.

3. Change the name to “BuildServer”.

4. Choose the Roles item. Change the local execution to “Remote Execution”.

5. Enter the name of the Test Controller that the Test Agent on the build server is using.

If you get the error “The following test controller is not available : YourServer. You must remove the association using the Lab Center within Microsoft Test Manager”, then you must remove this association.

a. To remove the association, remove the registration in the Visual Studio 2010 Test Controller Configuration Tool. Uncheck the “Register with Team Project Collection” option.

6. The Roles item in the BuildServer Test Settings should look like the following

7. Close the Test Settings and Check in the new build definition file.

8. Open the Build Definition. Click on the Process tab. Expand the Automated Tests > Test Assembly. Click on the TestSettings File ellipse. Chose the BuildServer test settings file.

9. Save the build definition and queue the new build. The Coded UI Test should run on the build and your test should pass. If the test fails, open the test results and view the details the same way you would do for a broken unit test.