Upload your UI Tests to the Xamarin Test Cloud

Whenever app development gets more serious and your code becomes more complex, testing your application automatically is a step you need to go to survive.Xamarin provides two neat ways to write UI Tests for your applications. Calabash is a framework you can write tests in Ruby with and Xamarin.UITests provides everything you need to build test cases in C# the NUnit way. Whichever framework youchoose, youcan write tests for your application very easilyand run them on an emulator or local device.

With both ways you can cover a managablerange of devices which might not be enough to test how your applications perform in the real world. Especiallywhen developing for Android,fragmentationis your daily companion and is not to be underestimated. Your application mayrun fine on all of your test devices and emulators but crashes on 80% of the rest.

Xamarin Test Cloud

This is, where Xamarin Test Cloud comes into play. It provides you a way to upload your app and tests to a huge farm of more than 1000 real devices and let them run there. Afterwards you will get a detailed test report of every single device including screenshots and Stack Traces for a comprehensive overview of which devices run your application nicely and which don’t.

I will not cover how to create these test cases in this blog post. If you are interested in the tests, take a look at thesample code at GitHub. What I want to show you today is how to upload your tests to the Xamarin Test Cloud and test your app on tons of devices in minutes.

Preparations

To use the Xamarin Test Cloud you need at least version 5.9 of Xamarin Studio and a Test Cloud subscription. If you already subscribe to one of the Xamarin Platform plans you can start immediately due all of theminclude 60 monthly device minutes which should be enough for our demo. If you have neither, you can at least take a look at the demo accout.

Uploading your test

To upload your project to the Xamarin Test Cloud, simply right click on the testsand select “Run in Test Cloud”. The following steps run almostautomatically and your application and tests are going to be uploaded into the cloud](http://pumpingco.de/content/images/2015/08/UploadUITests.png)

Important: Make sure, that you have changed your build configuration to Release and that it does not use the shared Mono runtime andsupports the x86 ABI.

As soon as this is finished, your browser pops up and asks you to define your test environment. Basically this is where you can select the devices that should run your test. You can surebetween several filters and pre-selections here. I personally liked the “Top 20 devices” option that selects the devices with most market share for you. Make sure that you also add some low end devices to test your application under extreme conditions](http://pumpingco.de/content/images/2015/08/DeviceSelection.png)

Warning: Keep an eye on your remaining test hours! By selecting 30 devices and letting the tests run for only two minutes, your 60 device minutes that come with your Xamarin Platform subscription are already burned. When creating serious test scenarios, you should consider buying adedicatedTest Cloud subscription.

When all tests have finished you can take a look at the results dashboard where you can see how the tests performed on each device. For every single device you can follow eachtest and the screenshots that have been taken. This provides you insights of how your design works on the device. If a test fails you can also take a look at the Stack Trace and find out, whyit crashed](http://pumpingco.de/content/images/2015/08/TestResults.png)

This is how I found out that my demo Pollen application seems to have a memory leak when loading the list of pollen with their images which causes crashes on weaker devices, which I never recognized when testing on my emulators and local devices.

Conclusion

Summarizing it up, Xamarin Test Cloud provides a really attractive way to test your apps on a wide variety of devices within a few minutes.Mobile developers know the pain of fragmentation and no one loves deploying theirtests to dozens of physical devices the manual way.And let’s be honest, buying 50 different physical devices is much more expensive than one of these Test Cloud subscriptions!

I hope I managed to make Xamarin Test Cloud tempting for you.If you already have a Xamarin subscription it is definitely worth a closer look. Try it yourself – it might be the child you always wanted! Please also feel free to take a look at the tests inthe sample code!