Executable Requirements

In the last post I talked about BDD as a way to improve our testing experience and to write more maintainable and higher quality tests. We also talked about two approaches to write BDD style tests/behaviors. I think the outcome was very nice; but can we improve it?

Using BDD to create executable requirements

I am going to use the credit card and ATM example from the BDD Simply post. I am using SpecFlow for this article; but there are other choices, like NBehave and StorEvil, that allow you to do more or less the same.

Let's code the ATM cash withdrawal specification together:

  1. You should first download and install the framework from here. I have installed V1.5 which is the latest version at the time of writing this article.
  2. Open Visual Studio and create a Class Library project. I am going to call mine 'Atm.Specs'.
  3. Select the project file in the solution explorer, right click, Add New Item. From the list choose 'SpecFlow Feature File'. This is installed as part of the SpecFlow installation. I am naming my file 'CashWithdrawal.feature'.
  4. Replace the template text in the file with your own story. For this you can use the report generated by the StoryQ tests from the BDD Simply post with very little editing.
  5. Save the feature file. This will auto generate a test class for you in the file grouped with the feature file. The file will include one test per scenario - in this case one test called CardHasBeenDisabled.
  6. Oh, we missed the references. Add nUnit.dll and TechTalk.SpecFlow.dll references to your project (You do not have to use nUnit and can use MSTest if you want to. That needs a bit of configuration.). SpecFlow on my machine is installed on 'C:\Program Files (x86)\TechTalk\SpecFlow'.

This is the text from my feature file:

Feature: Account Holder withdraws cash
In order to have access to my money
As an Account Holder
I want to withdraw cash from an ATM

Scenario: Card has been disabled
Given card is disabled
When the account holder requests money
Then the ATM should retain the card
And the ATM should say the card has been retained

If you run the test now it will be ignored because step definitions have not been implemented yet.

alt text

This is very sweet because when an existing feature with implemented tests changes (AKA when requirements change), those tests break and go red; but when a new feature is added its tests are flagged as inconclusive which does not break your build (test). In other words, your tests turn red for broken logic and yellow for yet to be implemented features.

On the right pane you can see that the framework has output some stub that you can use to implement required test steps. Copy and paste the provided stubs into a new file. Do not put this into the auto generated file; because it gets overwritten.

I created a file called 'CardHasBeenDisabled.cs' and implemented the steps as below:

[Binding]
public class CardHasBeenDisabled
{
    private Card _card;
    private Core.Atm _atm;

    [Given(@"card is disabled")]
    public void GivenCardIsDisabled()
    {
        _card = new Card(false);
        _atm = new Core.Atm();
    }

    [When(@"the account holder requests money")]
    public void WhenTheAccountHolderRequestsMoney()
    {
        _atm.RequestMoney(_card);
    }

    [Then(@"the ATM should retain the card")]
    public void ThenTheATMShouldRetainTheCard()
    {
        Assert.That(_atm.CardIsRetained, Is.True);
    }

    [Then(@"the ATM should say the card has been retained")]
    public void ThenTheATMShouldSayTheCardHasBeenRetained()
    {
        Assert.That(_atm.Message, Is.EqualTo(DisplayMessage.CardIsRetained));
    }
}

I am using the same Atm and Card classes from the BDD Simply post.

It is nice but it costs

Using SpecFor or StoryQ was very easy and straightforward. It did take a bit of mindset adjustment to write tests the BDD way which IMO is very easy to adopt and quite worth it.

To use SpecFlow we had to install it, which is not always welcome. The tests I wrote using SpecFlow above are also very similar to those I wrote using SpecFor. In fact SpecFlow tests are a bit wordier due to the use of SpecFlow attributes. We also have to maintain the feature file.

So what did we gain from using SpecFlow framework instead of SpecFor class?

Benefits

Specification owned by Business Analysts

With StoryQ (and SpecFor) the specification was mixed with the test implementation; but with SpecFlow the specification ends up in a separate file - the feature file. One of the benefits of having separate specification is that it can be owned by BAs. In other words, because the feature file is not coupled with the code and in fact does not know anything about the code, BAs can create and maintain it.

Who knows? You may even get lucky and get feature files from your BA for software requirements instead of 100 pages of word document with executive summary, big flowcharts and wordy and ambiguous requirements. That is pretty nice actually - instead of reading a boring long word document you get several feature files that you can execute and code against. You will have completed the feature/functionality when your tests (which BTW just turned into a regression suite) turn green.

Instant feedback when Business Analysts change a specification

When the requirement changes your tests break. Also the executable specification can be parameterized which provides a very nice framework for BAs to verify some of their guesses about the system and document it in an executable way. To learn more about this you may have a look at Gherkin which is the language of SpecFlow and a lot of other BDD frameworks.

YAGNI

YAGNI is another benefit. You write code to pass an executable requirements which means you are only coding what you need. You can of course go crazy and make a mess no matter what; but having a strict BDD practice in place guides you very well and keeps you on track avoiding some of the common YAGNI traps.

Conclusion

By separating specification and test implementation you may gain a few substantial benefits. I am in no way saying that you should always do this, or that these benefits always outweigh the cost.

These benefits are very nice and attractive; but you should make sure that they apply to your project/team. For example, if all team members are developers then I think StoryQ or even SpecFor would do just fine and removes some unnecessary maintenance load.

If customer (or their representatives; e.g. BAs) is willing to get involved in the SDLC earlier, which I believe provides a lot of value, then having a separate feature file could help. If they are happy to provide you with more explicit requirements in the form of GWT (Given When Then) having a separate feature file will be very beneficial.

As usual consider this as an option, do a cost/value analysis of this vs. simpler approaches and choose the one that suits you better.

Hope this helps.

comments powered by Disqus
Google