Testing native applications with #Sikuli and #Spock framework

Recently i've been asked to code assignment for Spotify.
Looking ahead I've failed the task.
Nevertheless i would like to describe here how i did it.
I've been asked to write tests for Spotify native client for following scenarios

- test weather wrong credentials will fail login
- test weather correct credentials will allow login
- test weather search is working properly
- test weather Spotify client plays music.

I was asked to write tests using Sikuli. I haven't heard about this tool, but i was impressed by it. That's why i'm writing this post.
So lets start.

Environment setup

I'm a Linux guy, thus i'll describe setup for this OS

Install Sikuli IDE

Download Linux distribution. I decided not to use deb packages to install.
After that you'll find that you cannot take screenshots, because libaux.so:2.1 is not found (You'll find a stacktrace in Sikuli IDE with JNI error).
Install OpenCV will help but on Ubuntu 12.04 you need install following packages,   check if all libs are installed with

ldd $SIKULI_INSTALL_DIR/libs/libVisionProxy.so

where libVisionProxy.so is JNI wrapper for OpenCV

You can skip the Sikuli IDE, but i strongly recommend to do this, because you can write scripts with it easily and learn how it works.

And finally, Sikuli jar should be uploaded to local maven repo:

upload sikuli-script to local maven repo mvn install:install-file -Dfile=sikuli-script.jar -DgroupId=org.sikuli -DartifactId=sikuli-script -Dversion=0.10.2 -Dpackaging=jar

So what is the Sikuli? You can find a lot of info about in Google Search. But in few words - it's a JNI wrapper above image processing libraries installed in you operating System. For Linux it's an OpenCV, as for other systems i don't know, maybe some Intel libraries for image processing. But it's not only wrapper, it provides an API for iteration with desktop. It allows you mouse moves and clicks, keyboard iterations symulations, etc. Why wouldn't we use it for test scenarios? Right we should. Actually for native clients Sikuli is essential.

Writing tests

I'm a Groovy and Spock fun, thus i've chosen them to write tests. I've decided that UI tests are not simple unit tests but more acceptance|functional and should be behavioral tests.

Setting up prerequisites for Spotify tests

All functional tests imply following scenario:

- run application under test
- run tests against this application
- shutdown this application

I used Spock annotation extension approach to run and stop application under tests. Using this approach you can do some stuff before and after Spock scenarios, features ind iterations (for data driven tests). Here is a Spock extension for running and stopping Spotify client before and after running test:

A Spock extension annotation

A Spock extension implementation

Further I'll show how to use this extension in tests.

This Spock extension is simple. Before scenario Spotify client is started and after stopped.

Tests

There a lot of papers and presentations about UI testing approach. Personally i like @vakaleo and his essays about page object pattern. The benefits of using this approach are
- hide implementation details in tests scripts
- DRY principle

Login tests

LoginScrean page object was created, let's look what is under the hood

Sikuli Screen object represents desktop. Using Screen object tests can interact with desktop like

s.type("./src/test/resources/sikuli-password.png", pwd)

or

wait(new Pattern("src/test/resources/sikuli-login-failed.png").similar(0.75))

in first case Sikuli will try to find a region on desktop similar with image

and type password there

For the second case Sikuli will wait until following image with 0.75 ratio of similarity will appear on screen

Playing with waiting of second image on desktop we can write tests for login window.

Pay attention to the Spock specification annotation. Annotating scenario with @RunSpotify will trigger listeners registered in SpotifySpockExtention, therefore Spotify client will be started and stopped for specification.

Lets look at more complex specification

Search and play tests

The page object SpotifyUIScrean


Here i would like to explain how music playing can be validated. Imagine simple player. When you start composition progress bar starts moving from left to right. That motion should be checked. You can find the code in progress closure in SpotifyUIScrean page object.
Slider should be found on desktop

then SikuliEventAdapter should be registered on this region and observe method should be triggered for this region. If something changed on this region targetChanged method will be triggered and we can treat it as progress bar motion

And finally grug-and-drop feature. In my test "verify that Marilyn Manson Cyclops song is playing" volume level increased to the maximum position. look at upVolume closure for details.
Player control should be found

On the right side

should be found

and then drug slider to

I had to do this trick because slider presents twice on Spotify UI, for volume control and song progress bar.

That's all. Here is a source code

PS.
i found wonderful post where web ui was tested with #Geb and #Sikuli in combination.

DISCLAIMER
I don't pretend this is a best solution and i would like to see you criticism of my code. I have a response from Spotify guys on my solution which is controversial IMHO :)