Exam >
On this page

Getting started

Overview

Given a simple CRUD web service for widgets management.

#.table: fill=#fff visual=table dashed #.sut: fill=#0e0 bold #.db: fill=#0e0 visual=database #.api: fill=#0e0 visual=lollipop #gravity: 2 #padding: 10 #spacing: 30 #direction: right [<sut> Widgets Service]<->[<db> DB] [<db> DB]--[<table> Widgets| id | integer || name | varchar || quantity | integer || updated | datetime] [<actor> client]-->[<api> POST /widgets] [<actor> client]-->[<api> GET /widgets] [<actor> client]-->[<api> PUT /widgets] [<actor> client]-->[<api> DELETE /widgets/{id}] [POST /widgets]-[Widgets Service] [GET /widgets]-[Widgets Service] [PUT /widgets]-[Widgets Service] [DELETE /widgets/{id}]-[Widgets Service]
Widgets service

Widgets are stored in DB like this:

WIDGETS
IDNAMEQUANTITYUPDATED
1widget one272022-03-12 15:26:16.211
2widget two142022-03-12 15:26:16.222

Testing API

Creation API

Let's illustrate the logic of the POST-endpoint with the basic happy-path example.

The following markup:
<e:example name="Successful widget creation">
  <e:given>
    <e:db-set caption="There are no widgets" table="WIDGETS"/>
  </e:given>
  <e:then>
    <e:post url="/widgets">
      <e:case desc="Successful creation">
        <e:body> {"name" : "widget1", "quantity": "10"} </e:body>
        <e:expected statusCode="201" reasonPhrase="Created"> { "id": "{{number}}", "name": "widget1", "quantity": 10, "updatedAt": "{{isoLocalDateTimeAndWithinNow "5s"}}" } </e:expected>
        <e:check>
          <e:db-check caption="Widget was created:" table="widgets" cols="id, name, quantity, updated" orderBy="name">
            <e:row>!{number}, widget1, 10, !{within 5s}</e:row>
          </e:db-check>
          <p> or the same but with 
            <var>regex</var> and 
            <var>not null</var> matchers: 
          </p>
          <e:db-check caption="Widget was created:" table="widgets" cols="id, name, quantity, updated" orderBy="name">
            <e:row>!{regex}\d, widget1, 10, !{notNull}</e:row>
          </e:db-check>
        </e:check>
      </e:case>
    </e:post>
  </e:then>
</e:example>
will be rendered as:
Successful widget creation

Given


There are no widgets
EMPTY

Then


POST /widgets
Content-Type application/json
Use cases:
1) Successful creation
POST /widgets HTTP/1.1
HTTP/1.1 201 Created92ms
{ "name": "widget1", "quantity": "10" }
Widget was created:
idnamequantityupdated
!{number}widget110!{within 5s}

or the same but with regex and not null matchers:

Widget was created:
idnamequantityupdated
!{regex}\dwidget110!{notNull}

Request has required fields. Invalid request should return error description.

The following markup:
<e:example name="Validation">
  <e:given>
    <e:db-set caption="There are no widgets" table="widgets"/>
  </e:given>
  <e:then>
    <e:post url="/widgets">
      <e:case desc="quantity is required">
        <e:body> {"name": "widget1"} </e:body>
        <e:expected statusCode="400" reasonPhrase="Bad Request"> { "error": "quantity is required" } </e:expected>
      </e:case>
      <e:case desc="name is required">
        <e:body> {"quantity": "10"} </e:body>
        <e:expected statusCode="400" reasonPhrase="Bad Request"> { "error": "name is required" } </e:expected>
      </e:case>
      <e:case desc="name should't be blank or more than 10 symbols">
        <e:body> {"name": "{{invalid}}", "quantity": "10"} </e:body>
        <e:expected statusCode="400" reasonPhrase="Bad Request"> { "error": "{{error}}" } </e:expected>
        <e:where vars="invalid, error">
          <e:vals>'' , blank value not allowed</e:vals>
          <e:vals>more_than_10, Value 'more_than_10' can't be stored to database column because exceeds length (10)</e:vals>
        </e:where>
      </e:case>
      <e:check>
        <e:db-check caption="No widgets were created:" table="widgets"/>
      </e:check>
    </e:post>
  </e:then>
</e:example>
will be rendered as:
Validation

Given


There are no widgets
EMPTY

Then


POST /widgets
Content-Type application/json
Use cases:
2) quantity is required
POST /widgets HTTP/1.1
HTTP/1.1 400 Bad Request10ms
{ "name": "widget1" }
3) name is required
POST /widgets HTTP/1.1
HTTP/1.1 400 Bad Request9ms
{ "quantity": "10" }
4) name should't be blank or more than 10 symbols
No widgets were created:
EMPTY

Or more compact equivalent of previous example (request/response body templates are stored in files):

The following markup:
<e:example name="Validate - parametrized">
  <e:given>
    <e:db-set caption="There are no widgets" table="widgets"/>
  </e:given>
  <e:then>
    <e:post url="/widgets">
      <e:case desc="Name and quantity validation">
        <e:body from="/data/getting-started/{{req}}"/>
        <e:expected from="/data/getting-started/error.json" statusCode="400" reasonPhrase="Bad Request"/>
        <e:where vars="name, req, error">
          <e:vals>ignored , invalid-no-name.json , name is required</e:vals>
          <e:vals>ignored , invalid-no-quantity.json, quantity is required</e:vals>
          <e:vals>'' , create-req.json , blank value not allowed</e:vals>
          <e:vals>more_than_10, create-req.json , Value 'more_than_10' can't be stored to database column because exceeds length (10)</e:vals>
        </e:where>
      </e:case>
      <e:check>
        <e:db-check caption="No widgets were created:" table="widgets"/>
      </e:check>
    </e:post>
  </e:then>
</e:example>
will be rendered as:
Validate - parametrized

Given


There are no widgets
EMPTY

Then


POST /widgets
Content-Type application/json
Use cases:
5) Name and quantity validation
No widgets were created:
EMPTY

Deletion API

Widget can be deleted by id.

The following markup:
<e:example name="Successful widget deletion">
  <e:given>
    <e:db-set caption="Given widget:" table="widgets" cols="name, quantity, id=1, updated={{now}}">
      <e:row>widget1, 10</e:row>
    </e:db-set>
  </e:given>
  <e:then>
    <e:delete url="/widgets/1">
      <e:case desc="Successful deletion">
        <e:expected/>
        <e:check>
          <e:db-check caption="Widget was deleted:" table="widgets"/>
        </e:check>
      </e:case>
      <e:case desc="Absent widget deletion">
        <e:expected statusCode="404" reasonPhrase="Not Found"/>
      </e:case>
    </e:delete>
  </e:then>
</e:example>
will be rendered as:
Successful widget deletion

Given


Given widget:
namequantityidupdated
widget11012022-03-12 15:26:35.331

Then


DELETE /widgets/1
Content-Type application/json
Use cases:
6) Successful deletion
DELETE /widgets/1 HTTP/1.1
20012ms
Widget was deleted:
EMPTY
7) Absent widget deletion
DELETE /widgets/1 HTTP/1.1
HTTP/1.1 404 Not Found5ms

Retrieving API

There is endpoint for retrieving all widgets.

The following markup:
<e:example name="Successful widget retrieving">
  <e:given>
    <e:set var="upd1" value="{{now tz='GMT+1'}}" hidden=""/>
    <e:set var="upd2" value="{{now plus='1 day'}}" hidden=""/>
    <e:set var="format" value="yyyy-MM-dd'T'HH:mm:ss.SSS" hidden=""/>
    <e:db-set caption="Given widgets:" table="widgets" cols="*name, *quantity, updated, id=1..10">
      <e:row>widget1, 10, {{upd1}}</e:row>
      <e:row>widget2, 20, {{upd2}}</e:row>
      <e:row>widget3, 30, {{date '01.02.2000 10:20+03:00' format="dd.MM.yyyy HH:mmz"}}</e:row>
      <e:row>widget4, 40, {{date upd2 plus='12 h'}}</e:row>
    </e:db-set>
  </e:given>
  <e:then>
    <e:get url="/widgets">
      <e:case desc="Can retrieve stored widgets">
        <e:expected> [{ "id": 1, "name": "widget1", "quantity": 10, "updatedAt": "{{dateFormat upd1 format}}" }, { "id": 2, "name": "widget2", "quantity": 20, "updatedAt": "{{dateFormat upd2 format}}" }, { "id": 3, "name": "widget3", "quantity": 30, "updatedAt": "{{dateFormat (date '01.02.2000 10:20+03:00' format="dd.MM.yyyy HH:mmz") format}}" }, { "id": 4, "name": "widget4", "quantity": 40, "updatedAt": "{{dateFormat (date upd2 plus='12 h') format}}" }] </e:expected>
      </e:case>
    </e:get>
  </e:then>
</e:example>
will be rendered as:
Successful widget retrieving

Given


Given widgets:
namequantityupdatedid
widget1102022-03-12 17:26:35.3861
widget2202022-03-13 15:26:35.3872
widget3302000-02-01 10:20:00.0003
widget4402022-03-14 03:26:35.3874

Then


GET /widgets
Content-Type application/json
Use cases:
8) Can retrieve stored widgets
GET /widgets HTTP/1.1
20036ms

CRUD-style testing

If gray-box testing (with direct Database checking) is not viable, here is the example of black-box approach:

The following markup:
<e:example name="black-box CRUD">
  <e:given>
    <e:db-set caption="Given no widgets:" table="widgets"/>
  </e:given>
  <e:when> Posting a new widget: 
    <e:post url="/widgets">
      <e:case desc="Create">
        <e:body>{"name" : "widget1", "quantity": "10"}</e:body>
        <e:expected statusCode="201" reasonPhrase="Created"> { "id": "{{number}}", "name": "widget1", "quantity": 10, "updatedAt": "{{string}}" } </e:expected>
      </e:case>
    </e:post>
  </e:when>
  <e:then>
    <e:set var="id" value="{{responseBody 'id'}}" hidden=""/>
    <e:set var="updatedAt" value="{{responseBody 'updatedAt'}}" hidden=""/>
    <p> The widget has been created with 
      <var>id</var> = 
      <code cc:echo="#id"/> and 
      <var>updatedAt</var> = 
      <code cc:echo="#updatedAt"/> and is available in widget list: 
    </p>
    <e:get url="/widgets">
      <e:case desc="Read">
        <e:expected> [{ "id": {{id}}, "name": "widget1", "quantity": 10, "updatedAt": "{{updatedAt}}" }] </e:expected>
      </e:case>
    </e:get>
  </e:then>
  <e:when> Updating the widget 
    <var>name</var> and 
    <var>quantity</var>: 
    <e:put url="/widgets">
      <e:case desc="Update">
        <e:body>{"id": {{id}}, "name": "new name", "quantity": "0"}</e:body>
        <e:expected> { "id": {{id}}, "name": "new name", "quantity": 0, "updatedAt": "{{formattedAndWithinNow "yyyy-MM-dd'T'HH:mm:ss.SSS" "5s"}}" } </e:expected>
      </e:case>
    </e:put>
  </e:when>
  <e:then>
    <p> The widget data has been changed: </p>
    <e:get url="/widgets">
      <e:case desc="Read">
        <e:expected> [{ "id": {{id}}, "name": "new name", "quantity": 0, "updatedAt": "{{formattedAndWithinNow "yyyy-MM-dd'T'HH:mm:ss.SSS" "5s"}}" }] </e:expected>
      </e:case>
    </e:get>
  </e:then>
  <e:when> Deleting the widget: 
    <e:delete url="/widgets/{{id}}">
      <e:case desc="Delete">
        <e:expected/>
      </e:case>
    </e:delete>
  </e:when>
  <e:then>
    <p> The widget disappeared from the list: </p>
    <e:get url="/widgets">
      <e:case desc="Read">
        <e:expected> [] </e:expected>
      </e:case>
    </e:get>
  </e:then>
</e:example>
will be rendered as:
black-box CRUD

Given


Given no widgets:
EMPTY

When


Posting a new widget:
POST /widgets
Content-Type application/json
Use cases:
9) Create
POST /widgets HTTP/1.1
HTTP/1.1 201 Created8ms
{ "name": "widget1", "quantity": "10" }

Then


The widget has been created with id = 4 and updatedAt = 2022-03-12T15:26:35.455 and is available in widget list:

GET /widgets
Content-Type application/json
Use cases:
10) Read
GET /widgets HTTP/1.1
2009ms

When


Updating the widget name and quantity:
PUT /widgets
Content-Type application/json
Use cases:
11) Update
PUT /widgets HTTP/1.1
20014ms
{ "id": 4, "name": "new name", "quantity": "0" }

Then


The widget data has been changed:

GET /widgets
Content-Type application/json
Use cases:
12) Read
GET /widgets HTTP/1.1
2007ms

When


Deleting the widget:
DELETE /widgets/4
Content-Type application/json
Use cases:
13) Delete
DELETE /widgets/4 HTTP/1.1
2009ms

Then


The widget disappeared from the list:

GET /widgets
Content-Type application/json
Use cases:
14) Read
GET /widgets HTTP/1.1
20010ms

Testing async behavior

Assume we need to trigger a job and do checks only after it's finished:

Await with custom method

Trigger the job and check that it's finished by polling custom method isDone:

The following markup:
<e:example name="custom polling">
  <e:given>
    <e:post url="/jobs">
      <e:case desc="Trigger job with some optional body">
        <e:body>{"name" : "value"}</e:body>
        <e:expected>{"id" : "{{number}}" }</e:expected>
      </e:case>
    </e:post>
  </e:given>
  <e:when>
    <e:set var="id" value="{{responseBody 'id'}}" hidden=""/>
    <e:await untilTrue="isDone(#id)" atMostSec="3" pollDelayMillis="500" pollIntervalMillis="1000"/> Job 
    <code cc:echo="#id"/> is finished. 
  </e:when>
  <e:then> Now we can check result: 
    <e:db-check table="jobResult" cols="result" where="id={{id}}">
      <e:row>done</e:row>
    </e:db-check>
  </e:then>
</e:example>
will be rendered as:
custom polling

Given


POST /jobs
Content-Type application/json
Use cases:
15) Trigger job with some optional body
POST /jobs HTTP/1.1
20016ms
{ "name": "value" }

When


Job 1 is finished.

Then


Now we can check result:
jobResult
result
done

Await with API polling

Same but with http polling of some job-execution API:

The following markup:
<e:example name="http polling">
  <e:when> Trigger job on 
    <code cc:set="#url">/jobs</code> with some optional body 
    <code cc:set="#json">{"name" : "value"}</code>
    <e:await untilHttpPost="{{url}}" hasStatusCode="200">{{json}}</e:await>
    <e:set var="id" value="{{responseBody 'id'}}" hidden=""/> and wait until it's finished. 
    <e:await untilHttpGet="/jobs/{{id}}" hasBodyFrom="/data/getting-started/job-finished.json"/> Job id = 
    <code cc:echo="#id"/>
  </e:when>
  <e:then> Now we can check result: 
    <e:db-check table="jobResult" cols="result" where="id={{id}}">
      <e:row>done</e:row>
    </e:db-check>
  </e:then>
</e:example>
will be rendered as:
http polling

When


Trigger job on /jobs with some optional body {"name" : "value"} and wait until it's finished. Job id = 2

Then


Now we can check result:
jobResult
result
done

Await on check

Same but with awaiting by db-check command:

The following markup:
<e:example name="db-check polling">
  <e:when> Trigger job on 
    <code cc:set="#url">/jobs</code> with some optional body 
    <code cc:set="#json">{"name" : "value"}</code>
    <e:await untilHttpPost="{{url}}" hasStatusCode="200">{{json}}</e:await>
    <e:set var="id" value="{{responseBody 'id'}}" hidden=""/> Job id = 
    <code cc:echo="#id"/>
  </e:when>
  <e:then> Await for result: 
    <e:db-check table="jobResult" cols="id, result" where="id={{id}}" awaitAtMostSec="4">
      <e:row>{{id}}, done</e:row>
    </e:db-check>
  </e:then>
</e:example>
will be rendered as:
db-check polling

When


Trigger job on /jobs with some optional body {"name" : "value"} Job id = 3

Then


Await for result:
jobResult
idresult
3done