Putting it together

by Luismi Cavallé

We already knew that there were two styles of testing: state-based and interaction-based. The former consists of specifying how the state of an object should change after exercising its methods. As for the latter, the interaction between the object and its collaborators is what is specified. Martin Fowler distinguishes between the users of each style as classicists (state-based) and mockists (interaction-based), because of the tool needed to perform this style of testing, known as mocks.

Similarly to mocking, which is a technique that enables a style of testing different to the classical, we could begin to talk of a testing style made possible by the use of macros and call it *structure-based testing*.

I’ve already blogged about Shoulda and its macros, but given the recent news about the Shoulda - RSpec love affair and the remarkable increasing popularity of Shoulda-like macros, I reckon it’s time to revisit the subject.

I find appropriate to call this style of testing structure-based because it doesn’t involve the interactions nor the changes of the state, but the actual structure of the object under test. In that sense, structure means the opposite of behaviour. These tests don’t say anything about the behaviour of the object, just what it should look like.

describe User do                             # class User < ActiveRecord::Base
  it { should belong_to(:account) }          #   belongs_to :account          
  it { should have_many(:posts) }            #   have_many :posts             
  it { should validate_presence_of(:email) } #   validate_presence_of :email  
end                                          # end

The thing is, I think there’s hardly any value in this kind of testing. Obviously these tests are easy to write and quick to run. The cost of writing them is low so the benefit could also be low and still be a good investment. However, there is usually an almost exact correspondence between the spec and the implementation. It would be possible to automatically generate one from the other. I can see some remote value in thinking twice about the structure of the object, but that won’t give me any clue about whether that structure will provide the behaviour I need to implement the current functionality. I’m just writing the same thing twice, but with different languages.

Satisfaction Guaranteed

Unit-testing Rails applications is not straightforward. Dan North justifies this by blaming the framework itself and the coupled nature of its components. Shoulda macros gives you a shortcut, they “make tests easy on the fingers and eyes”, that’s the framework’s motto. And they certainly provide what they promise: instant gratification, effortless testing with high code coverage and relatively fast execution time you can perform “all the fucking time”, like the cool kids do. The question is, does it provide any real value to the development process?

The answer depends, of course. It depends on each developer, team and context. My current approach of Rails testing is pretty different. But that might very well change in the future. The success probably lies in continuously challenging the why and the how of the way you do things, keeping in mind that there is no silver bullet and that, ultimately, it’s all about effectively building software that provides value to someone.

How we do BDD

10 Feb 09

Since I believe we also have something to say about all the fuss involved in testing, here is a 15-point summary on how we do it at BeBanjo, my current employer, as it was presented at Conferencia Rails 2008:

  1. We perform testing on two different levels: Acceptance and Unit.
  2. For Acceptance Testing we use Cucumber (formerly Story Runner). Cucumber automatically fills the gap between a user story written in plain English and an executable test.
  3. For Unit Testing we use RSpec. We could also use Shoulda, Context+Matchy or even Test:Unit+Mocka. Any of these would do. We believe it’s a matter of taste, essentially you can do the same with each of them.
  4. We find Acceptance Testing excellent as a verification tool, but not as good as a design/development tool.

  5. We therefore need Unit Testing as a development/design tool to enable us to be more productive and to make our code more maintainable.

  6. We follow an outside-in approach driven by the user story. We try to write the minimal code necessary to make the next step pass. From the view to the model, outside-in, adding the corresponding specs on the way.
  7. A comprehensive set of acceptance tests is essential to the way we do specs. If we didn’t have it, we would unit-test in a completely different way.
  8. We understand that “Mocks aren’t stubs” concludes that there’s no winner between classicists and mockists. It’s all about trade-offs, didn’t you know?
  9. We only spec the interesting behaviour of the object under test, the behaviour that other parts of the system rely on.
  10. We don’t usually spec views. We think it is unnecessary with acceptance testing.
  11. We spec controllers following an interaction-based approach using mocks. Controllers have no state, their interesting behaviour is determined by the way they interact with other objects (models and views). That’s why we think interaction-based testing is the most appropriate for controllers.
  12. We spec models following a state-based approach. And Rails fixtures are evil. Instead we use any implementation of the Factory pattern (fixture-replacement, FactoryGirl, machinist…)
  13. We don’t use macros in our specs. Macros encourage a style of testing I call structure-based (vs. state-based or interaction-based), which, in my opinion, gives almost no value. Using them feels like writing the same thing twice. Perhaps because that’s exactly what you’re doing.
  14. BDD works better when accompanied by other agile practices: pair-programming, refactoring, interface-first, short iterations, etc.

  15. We are opinionated, like our favourite web framework.

You can find the slides of the presentation here (in English) and the video (in Spanish) here.

I’m working on a series of posts to elaborate on each of these points. Stay tuned!

The Rails ways

11 Nov 08

Ya lo tenemos asumido pero es que Rails es revolucionario en muchos aspectos. Uno de los más importantes, pienso, es el hecho de que Rails no sea solo un framework web, una serie de librerias chulas puestas juntas. Rails es, además de eso, dos cosas fundamentales: una comunidad y una manera de desarrollar común. Es seguramente el caracter opinionado del framework el que marca la diferencia en este sentido. Rails toma partido en cuanto a como cree que se deben hacer las cosas. El concepto de la convención sobre la configuración resulta fundamental. Rails promueve algunas prácticas a base de convenciones y azucar sintáctico, pero además, casi más importante es que es el propio framework el que desalienta lo que considera menos adecuado a base de vinagre sintáctico.

Todo esto propicia que alrededor de la tecnología se genere una apasionada comunidad que abraza esta manera de desarrollar y que se hace consciente del valor que tiene que todos utilicen la tecnología de la misma manera. Nace así la Rails Way o el Acervo Cultural de Rails™, como le dice Sergio


Esta comunidad y esta manera de hacer las cosas crece y se extiende más allá de las fronteras del propio framework y ocurren cosas como que casi toda una comunidad utilice un editor determinado (TextMate), migre repentinamente a un sistema de control de versiones (Git) haciendo parecer legacy todo lo que no lo utilice, o tenga una serie de plugins “de cabecera” que remplazarán por otros (casi toda la comunidad al mismo tiempo) tan pronto como surjan alternativas mejores. ¿Existen una comunidad de desarrollo web en la que la proporción de Macs sea tan alta?

En definitiva, casi más importante que una tecnología es que Rails sea el centro de una manera integral de entender el desarrollo de aplicaciones web.

Evidentemente la comunidad ha crecido, evolucionado y madurado. Del stack básico que propone Rails (prototype.js + actionpack + activerecord + Test::Unit) van surgiendo cada vez más alternativas. Al mismo tiempo, cada vez más gente utiliza Rails para resolver problemas más distintos. Esto hace inevitable cierta disgregración que, seguramente, también sea buena.

Aún así la conciencia de movernos todos en manada siga muy viva y da lugar a cosas como que un simple post en un blog pueda llegar a generar el debate sobre si la comunidad debe abandonar un framework de testing.


En el programa de la Conferencia Rails que se celebra esta semana todo esto está reflejado. Además de las charlas más técnicas relacionadas con la infraestructura, se pueden encontrar las referidas a las alternativas al stack básico, como la de jQuery de Christos o la de Cucumber de Nando y Rai. También hay varias que se ocupan del “cómo” más que del “qué”, como la de Sergio sobre buenas maneras o la que presento yo mismo junto a Jorge Gómez Sancha acerca de como llevamos a la práctica esto del BDD (shameless self-promotion)

Nos vemos en la CR08!