Zero to RESTful in 4 easy steps. API Design.

Georgiana Gligor
Owner @Tekkie Consulting

This article is part 2 of a series showing how to prepare, design, and implement a RESTful API. After setting up the grounds in the first part, this time we are going to look at how an API is designed, what concerns we should have in mind when doing that. But first, we will have a look at what REST is, and what are the architectural constraints it imposes on a system.

What is REST?

Representational State Transfer is an application architectural style that, instead of imposing specific technology choices, defines a set of constraints for the software system to follow. This way, the actual implementation details can change, while still taking advantage from the overall benefits of the RESTful approach.

Key REST Concepts

A resource represents any information that can be named. Usually resources are concepts within the application domain, no matter if they refer to concrete concepts such as persons, or to virtual ones like files. A starting point of visualising it for those familiar with OOP is to use a 1-to-1 mapping with the domain model classes. Since we are going to build an application tracking vehicle data, we can say that one resource is vehicle, another one its insurance. The difficulty of explaining this term comes from the fact that aspects particular to the business can dictate deviations from the most intuitive way of representation, without this breaking the validity.

REST has been formally described by Roy J. Fielding in his PhD thesis. He started from a system with no distinguished boundaries between its components, and incrementally applying five mandatory constraints and an optional one to elements within the architecture:

1. client-server: separates UI concerns from data storage

2. stateless: keep session details entirely on the client, and free the server from dealing with it, in order to allow scalability, reliability, and visibility; the downside is that each request needs to contain enough context in order to get processed

3. cache: data within a response must be labeled as cacheable or non-cacheable

4. uniform interface between components, as defined by the following four interface sub-constraints: identification of resources; manipulation of resources through representations; self-descriptive messages; and hypermedia as the engine of application state (aka HATEOAS)

5. layered system: component can only "see" and interact with layers immediate to it; for example, clients cannot assume they interact directly with the data source, as they can be talking to a cache layer

6. [optional] code-on-demand: client functionality may be extended by downloading and executing code; this means that the client doesn't need to start with all the code, as they can grab it on demand; just imagine how functionality is added by injecting Javascript code to the browser

Representation is a part of the resource state that is transferred between the client and the server. It usually refers to the current state of the resource, but it can also indicate its desired state, you can think of this as the client performing an action dry-run when making the request.

While not a constraint itself, the communication mechanisms offered by HTTP are the choice of most people implementing REST. We are going to use HTTP as well, and use its powerful verbs to define operations on our resources. Let's use the common calendar paradigm to quickly illustrate the difference between resources and their representations, by showing an .ics representation first:

GET /calendar/123sample
Host example.dev
Accept: text/calendar

could return something similar to

BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//Tekkie Consulting//123sample//EN
CALSCALE:GREGORIAN
BEGIN:VTIMEZONE
TZID:Europe/Bucharest
END:VTIMEZONE
BEGIN:VEVENT
UID:123456789@example.dev
DTSTART;TZID=Z:20150311T123456
DTEND;TZID=Z:20150311T125959
END:VEVENT
END:VCALENDAR

If we ask for a JSON representation of the same resource

GET /calendar/123sample
Host example.dev
Accept: application/json

it might look like this:

{
  "version": "2.0",
  "creator": {"company": "Tekkie Consulting", "product": "123sample"},
  "type": "Gregorian",
  "language": "English",
  "timezone": {
    "id": "Europe/Bucharest"
  },
    "events": [{
      "id": "123456789@example.dev",
      "start": "2015-03-11T12:34:56.000Z",
      "end": "2015-03-11T12:59:59.000Z"
  }]
}

For the remaining of this article series we will choose JSON as the representation default of our resources.

BDD in PHP

In order to specify how our API will behave, we want to use Behaviour Driven Development. In this section we are going to introduce some concepts and tools to help with that. Setting this up will be very helpful when we will need to write down the requirements as user scenarios.

There are two main keywords that arise in discussions about doing BDD in PHP. The most popular is Behat, which uses Gherkin to specify the requirements in a business-like language. The other one is Codeception, which uses PHP code that is human readable to do the same, but in addition offers a group of tools to write all your tests in (acceptance, functional, as well as unit tests).

I have used Behat quite a lot in my Symfony projects, so I naturally tried to take the simple route and install it first, using $ composer install behat/behat. I noticed packaging conflicts because Behat wanted to use symfony/event-dispatcher ~2.1 but I was already using 3.0.1. So, instead of even attempting to fix that by imposing more strict requirements on my existing packages, I just decided I long wanted to give Codeception a try anyway, as others told me they absolutely love it in their day to day flow. No more nice text Gherkin spec files to share with the business people, back to 100% PHP!

So I went on and installed it using the Composer method, which worked perfectly, no digging required:

$ composer install codeception/codeception

and then bootstrap it

$ vendor/bin/codecept bootstrap

We notice a new folder called /tests/ in our root directory has been populated with a lot of goodness.

As we are in development mode, we start our application locally using the built-in PHP server:

$ php -S localhost:12345 -t web

so we will make sure this URL is properly configured under tests/acceptance.suite.yml.

Next we will generate a basic acceptance test by running:

$ vendor/bin/codecept generate:cept acceptance Welcome

Our tests/acceptance/WelcomeCept.php will contain a very simplistic test, that only checks the status route:

$I = new AcceptanceTester($scenario);
$I->wantTo('check the status route');
$I->amOnPage('/');
$I->see('Up and running.');

And the results are indeed what we expect:

$ vendor/bin/codecept run
Codeception PHP Testing Framework v2.1.5
Powered by PHPUnit 4.8.21 by Sebastian Bergmann and contributors.

Acceptance Tests (1) ---------------------------------------------
Check the status route (WelcomeCept)                        Ok
------------------------------------------------------------------
Functional Tests (0) ------------------------
---------------------------------------------

Unit Tests (0) ------------------------------
---------------------------------------------

Time: 214 ms, Memory: 11.25Mb

OK (1 test, 1 assertion)

The acceptance tests of Codeception are usually much slower than functional ones, as they require a web server to run tests. Fortunately, functional tests are going to be very good at describing our intended functionality, so we are going to use those instead. Let's enable the Silex module first, by adding it to tests/functional.suite.yml:

class_name: FunctionalTester
modules:
    enabled:
      - Silex:
          app: 'app/bootstrap.php'
      - \Helper\Functional

How do we design an API?

As we have seen in the REST introductory section, a key concept in this architecture style are resources, as everything else gravitates around them. Therefore I recommend going through the list of domain concepts first, and identify what our resources would be:

• vehicle - the main concept describing the central resource of our application

• road tax - has a start and end date of validity, and is associated to a vehicle

• insurance - similar properties to road tax, but is a distinct concept

• technical check - cannot exist without being attached to a vehicle; has a starti and end date of validity

Now that we know what our application resources are, what are the operations we define on each? We are certainly interested to be able to create, update, and retrieve vehicle information. Then we need to add and retrieve road tax details, add and retrieve technical checks, as well as add and retrieve insurance information.

All the above operations look strikingly similar to CRUD operations (part of them are missing in our list for domain reasons). So let's go ahead and define the full list of CRUD operations on a vehicle resource:

• Read information about all vehicles

  • GET /vehicles

  • If there are no vehicles, we will receive 200 OK status and a body of []

  • If there are items in the storage, we will receive 200 OK status and a body containing all of them, like [{ "name": "family car", "make": "Mazda", "model": "3", "registration": "AB10TKK", "VIN": "JMZBLA2F701213123", "engine": "1999", "emissions": "175", "registered": "2010-10-01T12:23:45Z" }]

  • If the server had issues fulfilling the request, a 500 Internal Server Error will be issued. It is the catch-all error for unrecoverable errors, when "it's the server's fault" and we can't offer recovery options to the client. This will be the case for all operations, so we will not repeat the information further.

• Create a vehicle

  • POST /vehicles/

  • Request body will contain all the data about our car, for example { "name": "family car", "make": "Mazda", "model": "3", "registration": "AB10TKK", "VIN": "JMZBLA2F701213123", "engine": "1999", "emissions": "175", "registered": "2010-10-01T12:23:45Z" }

  • On successful creation of the resource, we expect a 200 OK status code (some recommend using 201 Created instead) and the body to contain the same data we sent, plus the ID this resource has received in the backend.

  • If there was a problem with the request (let's say we will have a requirement for VIN to be mandatory and the request does not include it) that the client can fix, we signal that by issuing a 400 Bad Request status code and a body describing the problem: { "VIN": "mandatory" }

• Read information about a specific vehicle

  • GET /vehicles/{id}

  • If the vehicle with the given ID exists, we expect a 200 OK status code and the body to contain relevant data about it.

  • If there's no information about a vehicle with that ID, we expect a 404 Not Found and an empty response body.

• Update

  • PUT /vehicles/{id}

  • If the vehicle with the given ID exists, we expect a 200 OK status code and the body to contain relevant data about it.

  • If there was a problem with the request that the client can fix, we signal that by issuing a 400 Bad Request status code and a body describing the problem: { "VIN": "mandatory" }

• Delete

  • DELETE /vehicles/{id}

  • If there is a vehicle with that ID, it will be removed from the system and a 200 OK status code with an empty body will be returned.

  • If there's no information about a vehicle with that ID, we expect a 404 Not Found and an empty response body.

The dates are present here in ISO 8601 standard, to maintain dates in a timezone-aware format. Facebook had to switch in 2012 entire events API to account for timezone issues.

We will not go into further detail on operations about road tax, technical check and insurance, as they are very similar to those described above.

Keep the API consistent

It is important to offer a consistent API, first of all for our sanity, and then for the benefits of any 3rd party consuming it. This is very easy to explain with the public-published distinction that Martin Fowler makes. Once something is published, making changes to it requires a complex process of deprecating existing functionality, offering new one on a different URI, which takes more time and effort than a normal code update.

An important aspect to consider upfront is the API versioning. Some API designers prefer to define the version in the base URL, like GitHub which uses the version number, or like Twilio who uses https://api.twilio.com/2010-04-01 as their starting point. WePay also uses release timestamps in the URL to differentiate between versions. Almost everyone does versioning one way or another, and if you want to play safe you just start with /v1/ yourself and go from there. The biggest advantage of choosing this approach is to be able to have a drastically different API in the next version. However, the important question to ask yourself is: "Will the API itself change drastically over time? Or will the resources and their representation change?" Strategies to change a non-versioned resource structure or operations on it are a bit more complex, and they usually involve offering a new URI to handle the new data, and deprecating the current one. As Stefan Tilkov points out, REST is so much more than just some URI patterns and GET, PUT, POST, DELETE. Choosing the best strategy is highly dependant on the problem at hand, and we all know stories of under- or over-engineered APIs.

When constructing our URIs, we should take care to not mix singular and plurals. So we will not expose /vehicles/{id} and /tax/{id} in the same application, we will prefer taxes in the second case. It is generally recommended to use plural forms instead of singulars.

As the URIs identify resources, it is considered good practice to keep only nouns in defining them, and avoid verbs. To describe the actions that are performed on the resource, the HTTP verbs should be used. For example, usage of /vehicles/create should in fact be a POST on /vehicles.

For more hands-on examples, I recommend the well-written White House API standards, which contain also handy examples to show what is considered "bad" and what is "good". There are also some good hints on REST API Tutorial, but I recommend caution when navigating the site because not all information there is adhering strictly to the RESTful we met at the beginning of this article.

Even more granular advices concern the naming conventions and case usage, both for URIs and for JSON structures. Just make sure you pick one type and adhere to it everywhere.

Define application behaviour

Let's setup the vehicle behaviour we want by defining functional tests for them. We are going to use Cest classes as they will later help us set up the tests better. They are simple classes that group the cept functionality in a more OOP manner, help us use methods to group certain things together.

So we are going to generate our cest first, using

$ vendor/bin/codecept generate:cest functional Vehicles
Test was created in /Users/g/Sites/learn/silex-tutorial/tests/functional/VehiclesCept.php

Let's go ahead and define how creation of a new vehicle record should look like. We are basically saying "if a POST request is executed against the /vehicles endpoint, with a name for the new item, we are going to create it and retrieve the full record, including its ID". This will enable us to perform later operations on the item by accessing /vehicles/ID.

wantTo('create a new vehicle');
      $I->sendPOST('/vehicles', [
        'name' => 'Pansy'
      ]);
      // we mark scenario as not implemented
      $scenario->incomplete('work in progress');

      $I->seeResponseCodeIs(200);
      $I->seeResponseIsJson();
      $I->seeResponseJsonMatchesXpath('//id');
      $I->seeResponseJsonMatchesXpath('//name');
      $I->seeResponseMatchesJsonType([
        'id' => 'integer',
        'name' => 'string'
      ]);
      $I->seeResponseContainsJson([
        'id' => 123,
        'name' => 'Pansy'
      ]);
    }
}

Please note that, for now, we mark the scenario as incomplete, to prevent execution of our assertions. We will remove this in the next part of our tutorial, where we will get to implement the actual functionality.

A commit containing more information about the other operations defined for vehicles and the assertions we make can be consulted on GitHub.

Running the functional tests indicates that we have indeed defined several tests which are skipped for now, here are the relevant bits for you to check:

$ vendor/bin/codecept run

Functional Tests (4) ----------------------------------------------------
Check the status route (StatusRouteCest::checkTheStatusRoute)      Ok
Create a new vehicle (VehiclesCest::createItem)                    Incomplete
Retrieve current vehicles (VehiclesCest::retrieveItems)            Incomplete
Modify an existing item (VehiclesCest::updateItem)                 Incomplete
-------------------------------------------------------------------------

Time: 372 ms, Memory: 12.00Mb

OK, but incomplete, skipped, or risky tests!
Tests: 4, Assertions: 1, Incomplete: 3.

The remaining definition of use cases remains for the reader as an exercise.

Conclusions

We have learned what the REST architectural style is, learned the difference between resources and their representations, setup Codeception to help us define the user stories, learned how to design an API from a practical point of view, and saw what we need to be aware of to keep it consistent. In the next article we will refine the user stories and work on the implementation.