“Participants using continuous testing were three times as likely to complete the task before the deadline. Participants using continuous compilation were twice as likely to complete the task, providing empirical support to a common feature in modern development environments. Most participants found continuous testing to be useful and believed that it helped them write better code faster, and 90% would recommend the tool to others. The participants did not find the tool distracting, and intuitively developed ways of incorporating the feedback into their workflow.”

$ cat .git/hooks/post-commit
#!/bin/sh
git ls-files > Manifest

ruby 1.8.6
ruby 1.8.7 patchlevel  72 (my system's default)
ruby 1.8.7 patchlevel 174
ruby 1.9.1
ruby 1.9.2 preview1
jruby 1.3.1
rubinius 0.11.0

rvm makes it ridiculously easy to manage them.

Installing

Let’s start by installing rvm

$ gem install rvm
$ rvm-install

and follow the instructions rvm-install gives you. Next, let’s install ruby1.9 (v1.9.1 by default).

$ rvm install 1.9

That’s it! rvm fetches the source, compiles the interpreter, and installs it in its own sandbox in ~/.rvm

For moar awesomes, you can install some non-MRI/YARV interpreters.

$ rvm install jruby
$ rvm install rubinius

Switching between versions

$ rvm use 1.9
$ ruby -v
ruby 1.9.1p243 (2009-07-16 revision 24175) [i686-linux]
$ which ruby
/home/mynyml/.rvm/ruby-1.9.1-p243/bin/ruby
$ rvm use system  #revert to using your system's ruby
$ ruby -v
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]
$ which ruby
/usr/bin/ruby

This speaks for itself, I believe.

Gems

Because each new version lives in its own sandbox, it starts off almost gemless (rvm pre-installs RubyGems and rake). You can install gems just as you normally would.

$ rvm use 1.9
$ gem install mynyml-every  #don't use sudo
$ rvm gemdir
/home/mynyml/.rvm/gems/ruby/1.9.1
$ gem list

*** LOCAL GEMS ***

mynyml-every (1.0)
rake (0.8.7)

Cross Version Testing

Here’s a rake task that will allow you to run your app’s tests across multiple ruby versions (provided you defined your own test task separately)

namespace(:test) do
  desc "Run tests on multiple ruby versions"
  task(:portability) do
    versions = %w( 1.8.6  1.8.7  1.9  jruby  rubinius )
    versions.each do |version|
      system <<-BASH
        bash -c 'source ~/.rvm/scripts/rvm;
                 rvm use #{version};
                 echo "-------- `ruby -v` ---------\n";
                 rake -s test'
      BASH
    end
  end
end

And run with:

$ rake -s test:portability

The reason the task looks hackish is because rvm isn’t a script, but a bash function. This function is loaded automatically when you open up a new shell because of the source line rvm-install added to your .bashrc file, but .bashrc itself isn’t loaded when you use #system. So the rake task first sources that function, and then runs everything else within the same session so that it uses its environment.

[update] As Wayne pointed out in the comments, there is a new rvm idiom that can be used to achieve this:

rvm 1.8.6,1.8.7,1.9,jruby,rubinius rake -s test

The rvm docs have also been updated to include this info.

Links

rvm’s official website has pretty good docs. You can also get a fairly good summary of functionality with

$ rvm help

source: http://github.com/wayneeseguin/rvm/
site/docs: http://rvm.beginrescueend.com/
tracker: https://www.pivotaltracker.com/projects/26822
group: http://groups.google.com/group/rubyversionmanager/

Autotest <3

Like lots of rubyists, I love agile development, BDD/TDD and continuous
testing. And one of my favourite pieces of software for that task is Autotest. Whenever I start a new project, the first thing I do is open up a test file, write a placeholder test case, and fire up autotest. It’s the one gem I use on every single application I write; from gems to rails apps to bare bones rack apps, it’s always there to make my development much more enjoyable.

Well, almost always.

Some time ago I was working on a gem where I wanted to use the Expectations test framework. Expectations is not based on test/unit (as opposed to context or contest, for instance). But Autotest, by default, requires test/unit. The result wasn’t great.

Not too long ago, I was getting excited about rip, an alternative ruby package manager. So I tried using it. But autotest automatically requires rubygems. Not so compatible.

Not too long ago I started working on a webapp that has regular MRI based tests alongside JRuby based tests. But Autotest automatically ran the test suite with /usr/bin/ruby1.8. That just plain won’t work.

Watchr

“Hi. I’m Martin …. and I’m a continuous testing addict”

I was beginning to feel the withdrawal symptoms. So I started working on a solution that would satisfy my cravings in any dev environment. The result is Watchr, a continuous testing tool similar to autotest, but with a different philosophy.

Watchr aims to be much more flexible, at the cost of a few lines of setup. That is, it will read a small script file and run tests accordingly. The script file uses a very simple ruby DSL. Very simple as in a single method, that’s all it needs.

#           pattern                   action
watch('test/test_.*\.rb') {|md| system "ruby #{md[0]}"}

This tells watchr to monitor all test files (in this case all ruby files in the test/ directory that start with “test_”), and when one of those is saved, runs it with ruby. A continuous testing script for a basic project could be

watch('test/test_.*\.rb') {|md| system "ruby #{md[0]}"}
watch('lib/(.*)\.rb')     {|md| system "ruby test/test_#{md[1]}.rb"}

which, in addition to running any saved test file as above, will also run a lib file’s associated test. This mimics the equivalent autotest behaviour.

The command follows the structure:

watch('pattern') {|match_data_object| command_to_run }

Leaving the action user defined is the key. You can run JRuby-based tests, use rip, write your test suite with alternative testing frameworks, … or any combination of these.

For a good example of actual scripts, be sure to check out watchr’s own specs.watchr and docs.watchr scripts.

Install:

gem install watchr --source=http://gemcutter.org

Run:

$ cd to/your/project/root
$ watchr path/to/script

Links

source: http://github.com/mynyml/watchr
wiki: http://wiki.github.com/mynyml/watchr
bug tracker: http://github.com/mynyml/watchr/issues

I make extensive use of this functionality myself, and though there are
already a few ways to do it (editor, autotest, …), none of them worked well
enough for true focusing (the editor is thrown off by contexts, etc).

So I finally put together a gem to do focus the way I needed it to work. To
use, simply call the focus method above the test you want focused.

gem install phocus --source http://gemcutter.org

require 'test/unit'
require 'phocus'

class TestWidget < Test::Unit::TestCase

  def test_foo
    assert false
  end

  focus
  def test_bar
    assert true
  end

  def test_baz
    assert false
  end
end

Running this suite will only call test_bar (and the suite will pass).

focus can be used on multiple tests, including across test suites.

class TestWidget < Test::Unit::TestCase

  focus
  def test_foo
  end

  def test_bar
  end
end

class TestGatget < Test::Unit::TestCase

  def test_abc
  end

  focus
  def test_def
  end
end

Running these will only call test_foo and test_def.

And Phocus isn’t affraid of fancy tests, either:

class TestUser < Test::Unit::TestCase

  context "Authentication" do
    test "should deny on wrong username" do
      ...
    end

    focus
    test "should deny on wrong password" do
      ...
    end
  end
end

will work just as well. In fact, Phocus is known to work with at least the
following testing frameworks:

• test/unit
• minitest/unit
• shoulda
• context
• contest

Links

Source: http://github.com/mynyml/phocus
Issues: http://github.com/mynyml/phocus/issues

The first step is to capture the output so that we can compare it later. A simple way to do this is to swap $stdout for a StringIO object. ($stdout is an IO object. Kernel#puts and Kernel#print are wrappers around $stdout.puts and $stdout.write)

def keep_stdout(&block)
  begin
    orig_stream, $stdout = $stdout, StringIO.new
    block.call($stdout)
  ensure
    s, $stdout = $stdout.string, orig_stream
    s
  end
end

Clearly we want to restore the stdout stream after we are done with our call so that we can get out test restults printed.

Next, let’s create an assertion that wraps our stdout capture method.

def assert_output(expected, &block)
  keep_stdout do |stdout|
    block.call
    if expected.is_a?(Regexp)
      assert_match expected, stdout.string
    else
      stdout.string.should be(expected.to_s)
      assert_equal expected.to_s, stdout.string
    end
  end
end

The regexp matching is so we can assert that the output contains what we’re looking for, as opposed to being exactly equal to it.

Now we can do:

assert_output('abc') { puts "abc" }      #=> true
assert_output(/abc/) { puts "abc\ndef" } #=> true

If you have slightly more complex strings that you want to partially match, remember that you have to escape them first as they might contain special regexp chars. This can be achieved with a small string extention:

class String
  def to_regexp
    Regexp.new(Regexp.escape(self))
  end
end

#to_regexp is probably a bad name, but it’s only a test helper so it’ll work for now.

This allows:

assert_output("(1 + 1)".to_regexp) do
  puts "(2 + 2)\n(1 + 1)\nabc"
end
#=> true

To test for string formatting, though, I needed the expected string to be more complex than ‘abc’.

def test_formatting
  out = <<-STR.unindent
        (1 + 1)
    #=> 2
        (2 + 2)
    #=> 4
  STR
  # ...
end

This uses the small unindent gem to allow us to keep normal indentation; otherwise we’d have to write:

def test_formatting
  out = <<-STR
    (1 + 1)
#=> 2
    (2 + 2)
#=> 4
  STR
  # ...
end

Which works fine, of course. Just not as fun.

Putting it all together

require 'test/unit'
require 'rubygems'
require 'unindent'

class OutputTest < Test::Unit::TestCase
  def test_formatting
    out = <<-STR.unindent
          (1 + 1)
      #=> 2
          (2 + 2)
      #=> 4
      ----------
          (3 + 3)
      #=> 6
      ----------
    STR
    assert_output(out) do
      @obj.first_method
      @obj.other_method
    end
  end
end

You can see real life usage in simple_example’s test file.

  [1.4, 2.4, 3.4].map {|n| n.floor } #=> [1, 2, 3]
  [1.4, 2.4, 3.4].map(&:floor)       #=> [1, 2, 3]

It has become so popular, in fact, that it’s been included in ruby 1.8.7

Funny thing is, the block form is only an extra 3 characters (6 if you want to count spaces). Though I guess the additional “effort” needed to stop and think about what to name the variable is somewhat distracting, especially since the variable is redundant anyway. Regardless, it’s enough of an improvement on such a commonly used idiom that we happily adopted it.

But after having used it plenty (some rails ./script/console hacking sessions can make quite intense use of it), its syntax started feeling heavy.

My favorite way to access an object’s internals is by far the good ol’ dot. I sometimes use OpenStruct instead of a hash just so I can access the values as methods, and I use `andand.method` instead of `try(:method)` for the same reason.

I knew I would be happier if I could use the same trick with #map, so I wrote an Enumerable method. Compare,

  enum = [1.4, 2.4 ,3.4]
  enum.map {|n| n.floor } #=> [1, 2, 3]
  enum.map(&:floor)       #=> [1, 2, 3]
  enum.every.floor        #=> [1, 2, 3]

It has the added advantage that it can take arguments and blocks, so it’s not restricted to only a few methods.

  %w( axb dxf ).every.gsub(/x/,'y')
  %w( axb dxf ).every.gsub(/x/) { 'y' }

I turned it into a gem so I could easily use it in irb and the rails console, or add it to my rails projects.

To use it:

  # on the command line
  gem sources --add http://gems.github.com
  gem install mynyml-every

  # in your app
  require 'rubygems'
  require 'every'

If you want to use #every as part of a gem or plugin but don’t want to add a dependency for only a few uses, you can simply make it a git submodule, or even just copy the content into your app since it’s contained in a single file.

Project page: http://github.com/mynyml/every