Flexible Continuous Testing
Autotest <3
Like lots of rubyists, I love agile development, BDD/TDD and continuous
testing. And one of my favourite pieces of software for that task is Autotest. Whenever I start a new project, the first thing I do is open up a test file, write a placeholder test case, and fire up autotest. It’s the one gem I use on every single application I write; from gems to rails apps to bare bones rack apps, it’s always there to make my development much more enjoyable.
Well, almost always.
Some time ago I was working on a gem where I wanted to use the Expectations test framework. Expectations is not based on test/unit (as opposed to context or contest, for instance). But Autotest, by default, requires test/unit. The result wasn’t great.
Not too long ago, I was getting excited about rip, an alternative ruby package manager. So I tried using it. But autotest automatically requires rubygems. Not so compatible.
Not too long ago I started working on a webapp that has regular MRI based tests alongside JRuby based tests. But Autotest automatically ran the test suite with /usr/bin/ruby1.8. That just plain won’t work.
Watchr
“Hi. I’m Martin …. and I’m a continuous testing addict”
I was beginning to feel the withdrawal symptoms. So I started working on a solution that would satisfy my cravings in any dev environment. The result is Watchr, a continuous testing tool similar to autotest, but with a different philosophy.
Watchr aims to be much more flexible, at the cost of a few lines of setup. That is, it will read a small script file and run tests accordingly. The script file uses a very simple ruby DSL. Very simple as in a single method, that’s all it needs.
# pattern action
watch('test/test_.*\.rb') {|md| system "ruby #{md[0]}"}
This tells watchr to monitor all test files (in this case all ruby files in the test/ directory that start with “test_”), and when one of those is saved, runs it with ruby. A continuous testing script for a basic project could be
watch('test/test_.*\.rb') {|md| system "ruby #{md[0]}"}
watch('lib/(.*)\.rb') {|md| system "ruby test/test_#{md[1]}.rb"}
which, in addition to running any saved test file as above, will also run a lib file’s associated test. This mimics the equivalent autotest behaviour.
The command follows the structure:
watch('pattern') {|match_data_object| command_to_run }
Leaving the action user defined is the key. You can run JRuby-based tests, use rip, write your test suite with alternative testing frameworks, … or any combination of these.
For a good example of actual scripts, be sure to check out watchr’s own specs.watchr and docs.watchr scripts.
Install:
gem install watchr --source=http://gemcutter.org
Run:
$ cd to/your/project/root $ watchr path/to/script
Links
source: http://github.com/mynyml/watchr
wiki: http://wiki.github.com/mynyml/watchr
bug tracker: http://github.com/mynyml/watchr/issues
Tags: continuous testing, ruby, test
September 17th, 2009 at 18:20
“But Autotest automatically requires test/unit. The result wasn’t great.”
not true. you can specify whatever library you want via testlib.
September 18th, 2009 at 08:26
Well I stand corrected. I had missed that option. Thank you for the update - I modified the post.
September 19th, 2009 at 18:47
ERROR: Error installing watchr:
watchr requires rev (>= 0.3.0, runtime)
September 19th, 2009 at 19:25
@Ed rev is a gem dependency, it should have gotten installed automatically. Do you mind opening an issue at http://github.com/mynyml/watchr/issues with a bit more details? e.g. your OS and gem version. Thanks
September 20th, 2009 at 19:17
But does it integrate with Growl?
September 21st, 2009 at 11:17
I also got this error.
ERROR: Error installing watchr:
watchr requires rev (>= 0.3.0, runtime)
on cygwin and ubuntu
September 21st, 2009 at 13:12
@Ed, andy: I just pushed a workaround for this issue (v0.5.4). The gem should install fine now.
@Bob Aman: Watchr is more “abstract” than Autotest, in that it can be used for more than just running tests. For example, take a look at Watchr’s own scripts - it runs tests (specs.watchr), updates rdoc/yard documentation (docs.watchr) and builds the gem (gem.watchr). So it would be hard to decide what exactly to notify the user about. Though it would be very interesting if someone were to provide a watchr-growl-specs gem plugin that users could require in their script.
September 22nd, 2009 at 16:32
[...] watchr – Flexible Continuous Testing A replacement for autotest that advertises simplicity and more flexibility. [...]
September 23rd, 2009 at 09:51
As soon as I saw this, I put together an ‘autotest’ style script for my .NET project (if you’re interested: http://jonfuller.codingtomusic.com/2009/09/23/autotest-in-net/) … I’ve been waiting for watchr for a long time (just didn’t know it!)
Thanks mynyml!
September 23rd, 2009 at 13:01
@Jon that’s really interesting! Seeing Watchr used with a language other than ruby totally made my day.
September 24th, 2009 at 05:34
[...] source and some documentation (including example scripts) is available on Github, and you can read more on Martin’s blog [...]
September 24th, 2009 at 16:30
Oh, I love that it reacts to filesystem changes instead of polling. I’ve been trying to use RSpactor instead of autospec to save some CPU cycles but it’s never worked for me. I’m going to give Watchr a shot!
September 24th, 2009 at 17:04
@Elliot I’ve disabled the evented backend while I fix something - but it should be back in there this weekend
September 25th, 2009 at 18:13
The evented backend has been reenabled. It will kick in automatically on *nix systems if rev is installed (gem install rev)
September 28th, 2009 at 08:04
[...] O código-fonte e alguma documentação (incluindo exemplos de scripts) está disponível no Github, e também a um artigo escrito Martin. [...]
September 29th, 2009 at 03:24
I love watchr , built a simple git auto commit snippet it works fine.
only possible issue is if I add a new file to the folder, it does not seem to get recognised, until I restart the watchr,
any suggestions or fixes
September 29th, 2009 at 11:28
@Senthil You can pick up new files by saving/touching the script file instead of completely restarting watchr. That tells it to rescan for files matching the patterns. It’s not done automatically, to avoid wasting resources.