Getting started
Overview
Given a simple CRUD web service for widgets management.
Widgets are stored in DB like this:
ID | NAME | QUANTITY | UPDATED |
---|---|---|---|
1 | widget one | 27 | 2022-03-12 15:26:16.211 |
2 | widget two | 14 | 2022-03-12 15:26:16.222 |
Testing API
Creation API
Let's illustrate the logic of the POST-endpoint with the basic happy-path example.
<e:example name="Successful widget creation">
<e:given>
<e:db-set caption="There are no widgets" table="WIDGETS"/>
</e:given>
<e:then>
<e:post url="/widgets">
<e:case desc="Successful creation">
<e:body> {"name" : "widget1", "quantity": "10"} </e:body>
<e:expected statusCode="201" reasonPhrase="Created"> { "id": "{{number}}", "name": "widget1", "quantity": 10, "updatedAt": "{{isoLocalDateTimeAndWithinNow "5s"}}" } </e:expected>
<e:check>
<e:db-check caption="Widget was created:" table="widgets" cols="id, name, quantity, updated" orderBy="name">
<e:row>!{number}, widget1, 10, !{within 5s}</e:row>
</e:db-check>
<p> or the same but with
<var>regex</var> and
<var>not null</var> matchers:
</p>
<e:db-check caption="Widget was created:" table="widgets" cols="id, name, quantity, updated" orderBy="name">
<e:row>!{regex}\d, widget1, 10, !{notNull}</e:row>
</e:db-check>
</e:check>
</e:case>
</e:post>
</e:then>
</e:example>
will be rendered as:Given
EMPTY |
Then
Use cases: | |||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1) Successful creation | |||||||||||||||||
| HTTP/1.1 201 Created92ms | ||||||||||||||||
{ "name": "widget1", "quantity": "10" } | |||||||||||||||||
or the same but with regex and not null matchers:
|
Request has required fields. Invalid request should return error description.
<e:example name="Validation">
<e:given>
<e:db-set caption="There are no widgets" table="widgets"/>
</e:given>
<e:then>
<e:post url="/widgets">
<e:case desc="quantity is required">
<e:body> {"name": "widget1"} </e:body>
<e:expected statusCode="400" reasonPhrase="Bad Request"> { "error": "quantity is required" } </e:expected>
</e:case>
<e:case desc="name is required">
<e:body> {"quantity": "10"} </e:body>
<e:expected statusCode="400" reasonPhrase="Bad Request"> { "error": "name is required" } </e:expected>
</e:case>
<e:case desc="name should't be blank or more than 10 symbols">
<e:body> {"name": "{{invalid}}", "quantity": "10"} </e:body>
<e:expected statusCode="400" reasonPhrase="Bad Request"> { "error": "{{error}}" } </e:expected>
<e:where vars="invalid, error">
<e:vals>'' , blank value not allowed</e:vals>
<e:vals>more_than_10, Value 'more_than_10' can't be stored to database column because exceeds length (10)</e:vals>
</e:where>
</e:case>
<e:check>
<e:db-check caption="No widgets were created:" table="widgets"/>
</e:check>
</e:post>
</e:then>
</e:example>
will be rendered as:Given
EMPTY |
Then
Use cases: | |
---|---|
2) quantity is required | |
| HTTP/1.1 400 Bad Request10ms |
{ "name": "widget1" } | |
3) name is required | |
| HTTP/1.1 400 Bad Request9ms |
{ "quantity": "10" } | |
4) name should't be blank or more than 10 symbols | |
EMPTY |
Or more compact equivalent of previous example (request/response body templates are stored in files):
<e:example name="Validate - parametrized">
<e:given>
<e:db-set caption="There are no widgets" table="widgets"/>
</e:given>
<e:then>
<e:post url="/widgets">
<e:case desc="Name and quantity validation">
<e:body from="/data/getting-started/{{req}}"/>
<e:expected from="/data/getting-started/error.json" statusCode="400" reasonPhrase="Bad Request"/>
<e:where vars="name, req, error">
<e:vals>ignored , invalid-no-name.json , name is required</e:vals>
<e:vals>ignored , invalid-no-quantity.json, quantity is required</e:vals>
<e:vals>'' , create-req.json , blank value not allowed</e:vals>
<e:vals>more_than_10, create-req.json , Value 'more_than_10' can't be stored to database column because exceeds length (10)</e:vals>
</e:where>
</e:case>
<e:check>
<e:db-check caption="No widgets were created:" table="widgets"/>
</e:check>
</e:post>
</e:then>
</e:example>
will be rendered as:Given
EMPTY |
Then
Use cases: | |
---|---|
5) Name and quantity validation | |
EMPTY |
Deletion API
Widget can be deleted by id
.
<e:example name="Successful widget deletion">
<e:given>
<e:db-set caption="Given widget:" table="widgets" cols="name, quantity, id=1, updated={{now}}">
<e:row>widget1, 10</e:row>
</e:db-set>
</e:given>
<e:then>
<e:delete url="/widgets/1">
<e:case desc="Successful deletion">
<e:expected/>
<e:check>
<e:db-check caption="Widget was deleted:" table="widgets"/>
</e:check>
</e:case>
<e:case desc="Absent widget deletion">
<e:expected statusCode="404" reasonPhrase="Not Found"/>
</e:case>
</e:delete>
</e:then>
</e:example>
will be rendered as:Given
name | quantity | id | updated |
---|---|---|---|
widget1 | 10 | 1 | 2022-03-12 15:26:35.331 |
Then
Use cases: | ||
---|---|---|
6) Successful deletion | ||
| 20012ms | |
| ||
7) Absent widget deletion | ||
| HTTP/1.1 404 Not Found5ms | |
Retrieving API
There is endpoint for retrieving all widgets.
<e:example name="Successful widget retrieving">
<e:given>
<e:set var="upd1" value="{{now tz='GMT+1'}}" hidden=""/>
<e:set var="upd2" value="{{now plus='1 day'}}" hidden=""/>
<e:set var="format" value="yyyy-MM-dd'T'HH:mm:ss.SSS" hidden=""/>
<e:db-set caption="Given widgets:" table="widgets" cols="*name, *quantity, updated, id=1..10">
<e:row>widget1, 10, {{upd1}}</e:row>
<e:row>widget2, 20, {{upd2}}</e:row>
<e:row>widget3, 30, {{date '01.02.2000 10:20+03:00' format="dd.MM.yyyy HH:mmz"}}</e:row>
<e:row>widget4, 40, {{date upd2 plus='12 h'}}</e:row>
</e:db-set>
</e:given>
<e:then>
<e:get url="/widgets">
<e:case desc="Can retrieve stored widgets">
<e:expected> [{ "id": 1, "name": "widget1", "quantity": 10, "updatedAt": "{{dateFormat upd1 format}}" }, { "id": 2, "name": "widget2", "quantity": 20, "updatedAt": "{{dateFormat upd2 format}}" }, { "id": 3, "name": "widget3", "quantity": 30, "updatedAt": "{{dateFormat (date '01.02.2000 10:20+03:00' format="dd.MM.yyyy HH:mmz") format}}" }, { "id": 4, "name": "widget4", "quantity": 40, "updatedAt": "{{dateFormat (date upd2 plus='12 h') format}}" }] </e:expected>
</e:case>
</e:get>
</e:then>
</e:example>
will be rendered as:Given
Sat Mar 12 17:26:35 MSK 2022
Sun Mar 13 15:26:35 MSK 2022
yyyy-MM-dd'T'HH:mm:ss.SSS
name | quantity | updated | id |
---|---|---|---|
widget1 | 10 | 2022-03-12 17:26:35.386 | 1 |
widget2 | 20 | 2022-03-13 15:26:35.387 | 2 |
widget3 | 30 | 2000-02-01 10:20:00.000 | 3 |
widget4 | 40 | 2022-03-14 03:26:35.387 | 4 |
Then
Use cases: | |
---|---|
8) Can retrieve stored widgets | |
| 20036ms |
CRUD-style testing
If gray-box testing (with direct Database checking) is not viable, here is the example of black-box approach:
<e:example name="black-box CRUD">
<e:given>
<e:db-set caption="Given no widgets:" table="widgets"/>
</e:given>
<e:when> Posting a new widget:
<e:post url="/widgets">
<e:case desc="Create">
<e:body>{"name" : "widget1", "quantity": "10"}</e:body>
<e:expected statusCode="201" reasonPhrase="Created"> { "id": "{{number}}", "name": "widget1", "quantity": 10, "updatedAt": "{{string}}" } </e:expected>
</e:case>
</e:post>
</e:when>
<e:then>
<e:set var="id" value="{{responseBody 'id'}}" hidden=""/>
<e:set var="updatedAt" value="{{responseBody 'updatedAt'}}" hidden=""/>
<p> The widget has been created with
<var>id</var> =
<code cc:echo="#id"/> and
<var>updatedAt</var> =
<code cc:echo="#updatedAt"/> and is available in widget list:
</p>
<e:get url="/widgets">
<e:case desc="Read">
<e:expected> [{ "id": {{id}}, "name": "widget1", "quantity": 10, "updatedAt": "{{updatedAt}}" }] </e:expected>
</e:case>
</e:get>
</e:then>
<e:when> Updating the widget
<var>name</var> and
<var>quantity</var>:
<e:put url="/widgets">
<e:case desc="Update">
<e:body>{"id": {{id}}, "name": "new name", "quantity": "0"}</e:body>
<e:expected> { "id": {{id}}, "name": "new name", "quantity": 0, "updatedAt": "{{formattedAndWithinNow "yyyy-MM-dd'T'HH:mm:ss.SSS" "5s"}}" } </e:expected>
</e:case>
</e:put>
</e:when>
<e:then>
<p> The widget data has been changed: </p>
<e:get url="/widgets">
<e:case desc="Read">
<e:expected> [{ "id": {{id}}, "name": "new name", "quantity": 0, "updatedAt": "{{formattedAndWithinNow "yyyy-MM-dd'T'HH:mm:ss.SSS" "5s"}}" }] </e:expected>
</e:case>
</e:get>
</e:then>
<e:when> Deleting the widget:
<e:delete url="/widgets/{{id}}">
<e:case desc="Delete">
<e:expected/>
</e:case>
</e:delete>
</e:when>
<e:then>
<p> The widget disappeared from the list: </p>
<e:get url="/widgets">
<e:case desc="Read">
<e:expected> [] </e:expected>
</e:case>
</e:get>
</e:then>
</e:example>
will be rendered as:Given
EMPTY |
When
Posting a new widget:
Use cases: | |
---|---|
9) Create | |
| HTTP/1.1 201 Created8ms |
{ "name": "widget1", "quantity": "10" } |
Then
4
2022-03-12T15:26:35.455
The widget has been created with id = 4
and updatedAt = 2022-03-12T15:26:35.455
and is available in widget list:
Use cases: | |
---|---|
10) Read | |
| 2009ms |
When
Updating the widget name and quantity:
Use cases: | |
---|---|
11) Update | |
| 20014ms |
{ "id": 4, "name": "new name", "quantity": "0" } |
Then
The widget data has been changed:
Use cases: | |
---|---|
12) Read | |
| 2007ms |
When
Deleting the widget:
Use cases: | |
---|---|
13) Delete | |
| 2009ms |
Then
The widget disappeared from the list:
Use cases: | |
---|---|
14) Read | |
| 20010ms |
Testing async behavior
Assume we need to trigger a job and do checks only after it's finished:
Await with custom method
Trigger the job and check that it's finished by polling custom method isDone
:
<e:example name="custom polling">
<e:given>
<e:post url="/jobs">
<e:case desc="Trigger job with some optional body">
<e:body>{"name" : "value"}</e:body>
<e:expected>{"id" : "{{number}}" }</e:expected>
</e:case>
</e:post>
</e:given>
<e:when>
<e:set var="id" value="{{responseBody 'id'}}" hidden=""/>
<e:await untilTrue="isDone(#id)" atMostSec="3" pollDelayMillis="500" pollIntervalMillis="1000"/> Job
<code cc:echo="#id"/> is finished.
</e:when>
<e:then> Now we can check result:
<e:db-check table="jobResult" cols="result" where="id={{id}}">
<e:row>done</e:row>
</e:db-check>
</e:then>
</e:example>
will be rendered as:Given
Use cases: | |
---|---|
15) Trigger job with some optional body | |
| 20016ms |
{ "name": "value" } |
When
1Job
1
is finished.
Then
Now we can check result:
result |
---|
done |
Await with API polling
Same but with http polling of some job-execution API:
<e:example name="http polling">
<e:when> Trigger job on
<code cc:set="#url">/jobs</code> with some optional body
<code cc:set="#json">{"name" : "value"}</code>
<e:await untilHttpPost="{{url}}" hasStatusCode="200">{{json}}</e:await>
<e:set var="id" value="{{responseBody 'id'}}" hidden=""/> and wait until it's finished.
<e:await untilHttpGet="/jobs/{{id}}" hasBodyFrom="/data/getting-started/job-finished.json"/> Job id =
<code cc:echo="#id"/>
</e:when>
<e:then> Now we can check result:
<e:db-check table="jobResult" cols="result" where="id={{id}}">
<e:row>done</e:row>
</e:db-check>
</e:then>
</e:example>
will be rendered as:When
Trigger job on
/jobs
with some optional body {"name" : "value"}
2and wait until it's finished. Job id =
2
Then
Now we can check result:
result |
---|
done |
Await on check
Same but with awaiting by db-check
command:
<e:example name="db-check polling">
<e:when> Trigger job on
<code cc:set="#url">/jobs</code> with some optional body
<code cc:set="#json">{"name" : "value"}</code>
<e:await untilHttpPost="{{url}}" hasStatusCode="200">{{json}}</e:await>
<e:set var="id" value="{{responseBody 'id'}}" hidden=""/> Job id =
<code cc:echo="#id"/>
</e:when>
<e:then> Await for result:
<e:db-check table="jobResult" cols="id, result" where="id={{id}}" awaitAtMostSec="4">
<e:row>{{id}}, done</e:row>
</e:db-check>
</e:then>
</e:example>
will be rendered as:When
Trigger job on
/jobs
with some optional body {"name" : "value"}
3Job id =
3
Then
Await for result:
id | result |
---|---|
3 | done |