REST API test cucumber steps best practice

Question

Trying to write up cucumber feature steps for REST API test.

I am not sure which approach is better:

Given I log in with username and password
When I add one "tv" into my cart
And I check my cart
Then I should see the item "tv" is in my cart

or

Given the client authenticate with username and password
When the client send POST to "/cart/add" with body "{item: body}"    
Then the response code should be "200"
And the response body should expect "{success: true}"
When the client send GET to "/cart"    
Then the response code should be "200"
And the response body should expect "{"items": ["tv"]}"

Is there any convention to follow when people trying to write cucumber steps for REST API?

Answer 1

I just stumbled on this helpful article: http://gregbee.ch/blog/effective-api-testing-with-cucumber

To summarize...

Scenario: List fruit
  Given the system knows about the following fruit:
    | name       | color  |
    | banana     | yellow |
    | strawberry | red    |
  When the client requests a list of fruit
  Then the response is a list containing 2 fruits
  And one fruit has the following attributes:
    | attribute | type   | value  |
    | name      | String | banana |
    | color     | String | yellow |
  And one fruit has the following attributes:
    | attribute | type   | value      |
    | name      | String | strawberry |
    | color     | String | red        |

Validating a result against JSON is tricky business because if the result is an array, the elements may not be the same order as how you are validating in the test.

Answer 2

Here's a (close enough) example to what the Pragmatic Programmer's "the Cucumber Book" says about testing REST APIs via Cuke,and it seems to more closely relate to your second example:

Feature: Addresses
  In order to complete the information on the place
  I need an address

Scenario: Addresses
  Given the system knows about the following addresses:
   [INSERT TABLE HERE or GRAB FROM DATABASE]
  When client requests GET /addresses
  Then the response should be JSON:
  """
    [
     {"venue": "foo", "address": "bar"},
     { more stuff }
    ]
  """
STEP DEFINITION:

Given(/^the system knows about the following addresses:$/) do |addresses| 
# table is a Cucumber::Ast::Table
  File.open('addresses.json', 'w') do |io|
    io.write(addresses.hashes.to_json)
  end
end    

When(/^client requests GET (.*)$/) do |path|
   @last_response = HTTParty.get('local host url goes here' + path)
end

Then /^the response should be JSON:$/ do |json|
   JSON.parse(@last_response.body).should == JSON.parse(json)
end
ENV File:

require File.join(File.dirname(__FILE__), '..', '..', 'address_app')
require 'rack/test'
require 'json'
require 'sinatra'
require 'cucumber'
require 'httparty'
require 'childprocess'
require 'timeout'

server = ChildProcess.build("rackup", "--port", "9000")
server.start
Timeout.timeout(3) do
  loop do
    begin
      HTTParty.get('local host here')
      break
    rescue Errno::ECONNREFUSED => try_again
      sleep 0.1
    end
  end
end

at_exit do
  server.stop
end

Answer 3

I've been using cucumber to test and more importantly to document the API that I created using rails-api in my current project. I looked around for some tools to use and I ended up using a combination of cucumber-api-steps and json_spec . It worked well for me.

There is no convention on how to write the cucumber steps. The way you write your steps depends on how you want to use your cucumber suite. I used the cucumber output as the reference for our Angular JS client devs to implement the API client. So my cucumber steps contained the actual JSON requests and responses along with the status code for each scenario. This made it really easy to communicate with a client side team whenever something changed ( especially when the client side team was not physically present at my workplace ).

Everytime I would create or update an API, the CI server would run cucumber as part of the build and move the HTML formatted output to a "build_artifacts" location that can be opened in the browser. The client side devs would always get the most recent reference that way.

I've written all of this down in a blog post about creating a tested, documented and versioned JSON API , hope it helps you in some way.

Answer 4

One of Cucumber's original intents, which contributes to its design, is to bridge the gap between technical implementation, and people who know the business needs, so that the test descriptions could be written and/or understood by non-developers. As such, it's not a great fit to detailed technical specs, or blow-by-blow unit testing.

So that would point me to your first test description, if that's also the reason you are using Cucumber.

There is no major problem with implementing tests like the second version, Cucumber can support it. There probably are not a large number of statement types you would need to parse either. But you could end up fighting the test framework a little, or going against your rationale for using Cucumber in the first place.

As for a convention, I am not aware of enough REST API tests in practice to comment, and none that I have seen tested have used Cucumber as the framework.

Update: Browsing around SO on the subject, I did find a link to this: https://github.com/jayzes/cucumber-api-steps which is more similar to your second format.

Answer 5

There are a few libraries now for server side REST testing with cucumber in Ruby . Here's a couple:

• Cucumber-API-Steps . (Recommended)
• Cucumber-API . A small tutorial for that is here .

The library I've been using for server side REST testing with cucumber is Cucumber-API-Steps .

Cucumber-API-Steps

Here's how I'd write your test using 'cucumber-api-steps' (Recommended):

@success
Scenario: Successfully add to cart
  Given I am logged in
  When I send a POST request to “/cart/add” with the following:
       | item | body |
  Then the response status should be “200”
  And the JSON response should have "success" with the text "true"
  When I send a GET request to “/cart”
  Then the response status should be “200”
  And the JSON response should be "{'items': ['tv']}"

And here's what my tests look like using 'cucumber-api-steps' :

@success
Scenario: Successfully log in
  Given I am logged out
  When I send a POST request to “/login” with:
       | username | katie@gmail.com |
       | password | mypassword |
  Then the response status should be “200”
  And the JSON response should have "firstName" with the text "Katie"

Cucumber-API

Here's how I'd write your test using 'cucumber-api' :

@success
Scenario: Successfully add to cart
  Given I am logged in
  When I send a POST request to “/cart/add”
       And I set JSON request body to '{item: body}'
  Then the response status should be “200”
  And the response should have key “success” with value “true”
  When I send a GET request to “/cart”
  Then the response status should be “200”
  And the response should follow "{'items': ['tv']}"

And here's what my tests look like using 'cucumber-api' :

@success
Scenario: Successfully log in
  Given I am logged out
  When I send a POST request to “/login” with:
       | username | katie@gmail.com |
       | password | mypassword |
  Then the response status should be “200”
  And the response should have key “firstName”

• Note about Cucumber-API: There is no way currently to do should have key “firstName” with value “Katie” . The "with value" part has not been done yet.
• Also "Follow" expects a JSON file

Another resource is here , but it is old (2011).

Answer 6

I would recommend your first scenario.

From my own experiences I personally feel that the biggest value you get from using BDD as a software delivery method, is when you place the emphasis on business value.

In other words the scenarios should be examples of what behaviour the business wants, rather than technical implementation. This ensures that the development is driven by the goals of the business and the deliverables match their expectations.

This is known as outside-in development.

Additional tests of the system behaviour can and should be used to cover the technical requirements but I think there's less value in spending effort writing these up in natural language, which is often time consuming and laborious across a large number of scenarios.

I recommend the following approach:

1) Work with the BAs and POs to develop examples of the behaviour they want using non-implementation specific language (like your first example).

2) Engineers use these to drive the development from a test first approach, automating them as integration tests - with the majority below the browser (against your REST API for example) and the most core scenarios also through the browser(if you are developing one).

3) Engineers TDD the feature code with unit tests until both the unit tests and BDD examples pass.

Answer 7

I think the first one is better. I would put the technical in the ruby classes and modules. Eg like module cart.add(items) in the when step and in the then step put expect(cart.item).to include('items' => a_string_matching(item))

By this, the ruby classes and modules can be reuse in another features steps. Eg like maybe you have another scenario which would add multiple items into the cart then validate the total amount.

However, the second 1 I think can make it like technical features. Eg like common/global header or body request is expected across all the api.

Answer 8

See here: https://github.com/ctco/cukes-rest . It provides a Cucumber DSL to test RESTful APIs.