A new language, a reused test approach

The product that our team develops doesn't have much of a GUI, so testing it involves a lot of coding, for example against the API of our webservice. I have done such testing before, shortly, and had figured out an approach to writing the test code that I intended to reuse. 

 

First a little bit of the background:

 

My team develops in Java and Scala. I have been coding in Java a lot while I was still working as a programmer, so I first thought, well, let's start in Java then. The whole team will be able to read and write the code and it will lower the hurdle to involve them in the testing. After struggling about three weeks with the set up and the first checks (how I wanted to write them), I became a bit frustrated. Not because it took longer than expected but because I had done the same thing in Clojure and the same work took about a tenth of the time. The difference was significant. While it is surely possible to do the testing style I want with object oriented Java, it is not easy.

 

I was torn between trying to use a tool that is known to the whole team (OO Java) and a tool that is better suited to the task at hand (functional Clojure). But my team does use a functional programming language. It's just that I hadn't written any Scala ever, apart from a couple of rows about 4 years back on a Code Retreat (for like 30 minutes or so). Well, if the mountain won't come to the prophet... So last Friday, I started using a new programming language, Scala, and tried to pull together all the pieces I like to have at hand for writing some checks.

 

So what are those bits and pieces? 

 

Most of the time I am not writing unit tests but on a higher level, like acceptance tests. Well, I am not the world's best programmer. That applies, sad enough, also to my test code, so I am pretty eager to write my checks test-driven to keep them somewhat tidy and correct. And as I am much less eager to come up with a lot of test data, I fall back on an idea that I heard of at the XPdays Germany 2010 from  Nicole Rauch  and Marc Philipp: Property based testing with tools like Quickcheck and its derivatives for different languages. Therefore I want a testing framework that allows me to use it with a Quickcheck-like library. In Scala I got lucky: The standard testing framework ScalaTest has built in support for ScalaCheck and property based testing.

 

Closely coupled with property based testing as I use it is the need of generating random but custom-tailored test data for my test cases. Most Quickcheck based frameworks that I came across have support for generating test data built in. And so far it paid off to spend some time to get accustomed to how to generate all kinds of basic and elaborate objects. Usually I use this heavily in my unit tests to randomize my inputs and not only run two or three specific cases but for example a hundred cases in one go. More often than not I stumble over cases that I did not think of before. Also here I got lucky with Scala as ScalaCheck does indeed have very good support to generate all sorts of data for my tests.

 

Ok, what do we have? 

 

Property based testing, generated data, but something is missing: The validation. When I worked with Clojure, I simply used predicate-methods to feed the actual assertion. The framework I used would give me back well-formed and significant error messages. Instead of giving me a "true was false" it would use my predicate name in the error message and I could tailor the outcome of a test when something goes wrong to be able to find the issue faster.

 

Just like Uncle Bob Martin described in Clean Code, names matter. A lot. I feel that, if I want my tests to be useful, I need to consider the error messages when a test fails. If the error doesn't tell my team what's wrong, the test is not useful. Worst case: I'd be the only one understanding what is going on and therefore condemned to maintain everything myself. Not a team effort, right? So a very important puzzle piece in my test approach is a tool to give my team awesome error messages.

 

This took me a while in Scala. And I can tell you that this point was the effective reason to drop Java for my current task. I couldn't get meaningful error messages for a price I was willing to pay. Either the messages were not at all useful to find the underlying reason for a failing test or I would have to write a whole bunch of extra code to get something significant. Especially when using generated data and lots of checks in one run, you'll want to know with which data a test fails.

 

So far I am happy with my current solution to this problem: Matchers. Funnily enough I got the idea through getting frustrated with Java and its matchers. In Scala they come without much extra work and allow me to  tailor the failure messages  exactly with the information the developers need to identify the problem the check is meant for. They also allow me to organize my code in a suitable way. I can have the matchers for an object in one file without cluttering the original object or the file with the test code with pure validation code that isn't used for anything else.

 

Is that all?

 

Well, sort of. Surely there are some other things to take care of in my current situation. For example I need a good library for communicating with REST APIs and deal with json data. Such things were easy to find on day one. But those needs are different from project to project and not part of my general approach. While the way of how I puzzle together my tools grew over a bit of time. It seems fit for many cases with adjustments here and there and might even inspire yours a bit. Or you have suggestions what I could try out in addition. 

 

Feel free to comment.