On this page

Exam

Extension for Concordion BDD framework

Overview

Exam is oriented on declarative end-to-end gray-box application testing in a way a manual tester would do it: send request, verify response/database/message queue etc.

Manual testing

Hence the Exam functionality falls apart in different libraries (plugins) that are tailored for specific kinds of checks: database interactions, message queue interactions, http interactions, file system interactions etc. and may be used separately.

Exam testing

Each library consist of a plugin class that should be configured and attached to ExamExtension and set of commands that can be used in .xhtml specification files.

Exam leverages the Concordion Extension API and may be enabled by standard Concordion annotation based way:

@RunWith(ConcordionRunner.class)
@ConcordionOptions(declareNamespaces = {"c", "http://www.concordion.org/2007/concordion", "e", ExamExtension.NS})
public class Specs {
    @Extension
    private final ExamExtension exam = new ExamExtension(
        new WsPlugin("/app", 8080),
        new DbPlugin("org.postgresql.Driver", "jdbc:postgresql://localhost:5432/postgres", "postgres", "postgres")
    );
}

But the recommended way is to implement io.github.adven27.concordion.extensions.exam.core.AbstractSpecs class that will do the same internally but make configuration points more visible and abstract the enabling-related boilerplate code away:

public class Specs extends AbstractSpecs {
    @Override
    protected ExamExtension init() {
        return new ExamExtension(
            new WsPlugin("/app", 8080),
            new DbPlugin("org.postgresql.Driver", "jdbc:postgresql://localhost:5432/postgres", "postgres", "postgres")
        );
    }
...
    /*  Methods are listed in order of execution: */

    @Override
    protected void beforeSetUp() {
        // Optional: Run some actions BEFORE Exam set up
    }

    @Override
    protected void beforeSutStart() {
        // Optional: Run some actions AFTER Exam set up and BEFORE start of a System Under Test
    }

    @Override
    protected void startSut() {
        // Start System Under Test before specs suite if needed
    }

    @Override
    protected void stopSut() {
        // Stop System Under Test after specs suite if needed
    }

    @Override
    protected void afterSutStop() {
        // Optional: Run some actions AFTER stop of a System Under Test and BEFORE Exam tear down
    }

    @Override
    protected void afterTearDown() {
        // Optional: Run some actions AFTER Exam tear down
    }
}

Core

Core commands may be used to extend default Concordion possibilities of setting up, verifying and decorating of examples.

Commands

The following are core-commands available

Specification by example

example

Decorated version of Concordion example command:

<e:example name="Happy path">
...
</e:example>

Context variables setting

set

Handlebars-aware version of Concordion set command:

<pre e:set="dateVar">{{now}}</pre>

Verifying

equals, equalsFile

Handlebars-aware version of Concordion assert command:

<span e:equals="actual">{{now}}</span>
<span e:equalsFile="actual">/data/expected.txt</span>

jsonEquals, jsonEqualsFile

Json-aware version of Concordion assert command:

<span e:jsonEquals="actual">{"a" : 1}</span>
<span e:jsonEqualsFile="actual">/data/expected.json</span>

xmlEquals, xmlEqualsFile

Xml-aware version of Concordion assert command:

<span e:xmlEquals="actual">
<! [CDATA[
    <message>
        <val>{{placeholder}}</val>
    </message>
] ]>
</span>

<span e:xmlEqualsFile="actual">/data/expected.xml</span>

Decorating

given, when, then

Support of the BDD-style example description:

<e:given>
...
</e:given>
<e:when>
...
</e:when>
<e:then>
...
</e:then>

Handlebar support

Support of the Handlebars templates

Plugins

Database plugin

DbPlugin is a wrapper around DbUnit library and enables to set up and verify state of the database.

Exam integration with DB

Usage

1. Add dependency

testImplementation "io.github.adven27:exam-db:{{version}}"

2. Configure and attach the DbPlugin

Commands

There are commands for setting up and verifying database state. Also there is a command to just show current database state and generate dataset files based on it.

Set up database

db-set

Creates and applies a DbUnit dataset for specified table:


<e:db-set table="person" cols="id=1..10, name, birthday, created={{now}}">
    <e:row>Bob, {{date '01.01.2001' format='dd.MM.yyyy'}}</e:row>
    <e:row> Ed, {{date '02.02.2002' format='dd.MM.yyyy'}}</e:row>
</e:db-set>

db-execute

Applies specified DbUnit datasets:

<e:db-execute datasets="/data/db/adam.xml, /data/db/bob.json, /data/db/carl/person.csv"/>

db-clean

Cleans specified tables with DELETE_ALL DbUnit operation:

<e:db-clean tables="person, person_fields"/>

Verify database

db-check

Creates a DbUnit dataset and verifies that it matches specified database table:


<e:db-check table="person" cols="id=1..10, name, birthday, created={{now}}">
    <e:row>Bob, {{date '01.01.2001' format='dd.MM.yyyy'}}</e:row>
    <e:row> Ed, {{date '02.02.2002' format='dd.MM.yyyy'}}</e:row>
</e:db-check>

db-verify

Verifies that database matches a state described in a DbUnit datasets:

<e:db-verify datasets="/data/db/adam.xml, /data/db/bob.json, /data/db/carl/person.csv"/>

Debug database

db-show

Creates a DbUnit dataset files and prints content of specified database table:

<e:db-show table="person" saveToResources="/data/db/person.xml"/>

Web Service plugin

WsPlugin enables to document and verify REST/SOAP API.

Usage

1. Add dependency

testImplementation "io.github.adven27:exam-ws:{{version}}"

2. Configure and attach the WsPlugin

Commands

There are commands for specifying use cases of REST and SOAP endpoints.

Verify REST API

get

Specifying use cases of a GET-endpoint:

<e:get url="/some/url" contentType="application/json">
    <e:case desc="use-case 1" urlParams="param1=1">
        <e:expected statusCode="200">
            { "response": "body" }
        </e:expected>
    </e:case>
    <e:case desc="use-case 2" urlParams="param1=1&amp;param2=2">
        <e:expected from="/ws/response.json" statusCode="200"/>
    </e:case>
</e:get>

post

Specifying use cases of a POST-endpoint:

<e:post url="/some/url" contentType="application/json">
    <e:case desc="use-case 1">
        <e:body>
            {"request": "body"}
        </e:body>
        <e:expected statusCode="201" reasonPhrase="Created">
            { "response": "body" }
        </e:expected>
    </e:case>
    <e:case desc="use-case 2">
        <e:body from="/ws/request.json"/>
        <e:expected from="/ws/response.json" statusCode="201" reasonPhrase="Created"/>
    </e:case>
</e:post>

put

Specifying use cases of a PUT-endpoint:


<e:put url="/some/url" contentType="application/json">
    <e:case desc="use-case 1">
        <e:body>
            {"request": "body"}
        </e:body>
        <e:expected>
            { "response": "body" }
        </e:expected>
    </e:case>
    <e:case desc="use-case 2">
        <e:body from="/ws/request.json"/>
        <e:expected from="/ws/response.json"/>
    </e:case>
</e:put>

delete

Specifying use cases of a DELETE-endpoint:


<e:delete url="/some/{{id}}" contentType="application/json">
    <e:case desc="use-case 1" cookies="cook={{var}}, anotherCook=asd">
        <e:expected statusCode="200">
            { "response": "body" }
        </e:expected>
    </e:case>
    <e:case desc="use-case 2">
        <e:expected from="/ws/response.json" statusCode="200"/>
    </e:case>
</e:delete>

Verify SOAP API

soap

Specifying use cases of a SOAP-endpoint:


<e:soap url="/some/url">
    <e:case desc="use-case 1">
        <e:body>
        <! [CDATA[
            <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
                <soap:Body>
                    <ns2:getItemRequest xmlns:ns2="http://ws.io">
                        <date>{{now 'yyyy-MM-dd'}}</date>
                    </ns2:getItemRequest>
                </soap:Body>
            </soap:Envelope>
        ] ]>
        </e:body>
        <e:expected>
        <! [CDATA[
            <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
                <soap:Body>
                    <ns2:getItemResponse xmlns:ns2="http://ws.io">
                        <val>{{placeholder}}</val>
                    </ns2:getItemResponse>
                </soap:Body>
            </soap:Envelope>
        ] ]>
        </e:expected>
    </e:case>
    <e:case desc="use-case 2">
        <e:body from="/ws/request.xml"/>
        <e:expected from="/ws/response.xml"/>
    </e:case>
</e:soap>

Message Queue plugin

MqPlugin enables to set up and verify the state of some message queue. A message queue is represented as an implementation of the io.github.adven27.concordion.extensions.exam.mq.MqTester interface and that implementation is completely responsible for interacting with that specific queue.

Exam integration with MQ

There are several out-of-the-box implementations that can be used directly or as an example for custom ones: exam-mq-kafka, exam-mq-rabbit, exam-mq-ibmmq, exam-mq-redis.

Usage

1. Add dependency

testImplementation "io.github.adven27:exam-mq:{{version}}"

2. Configure and attach the MqPlugin

Commands

There are commands for setting up and verifying queue state.

Set up queue

mq-send

Sends a message to specified queue:

<e:mq-send name="someQueue" from="/data/message.json"/>

mq-purge

Purges specified queue:

<e:mq-purge name="someQueue"/>

Verify queue

mq-check

Verifies that specified messages exist in a queue:


<e:mq-check name="someQueue">
    <e:message>{"msg" : "1"}</e:message>
    <e:message from="/data/mq/msg2.json"/>
</e:mq-check>

UI plugin

UiPlugin is a wrapper around Selenide and WebDriverManager libraries and enables to verify Web UI.

Exam interactions with Web UI

Usage

1. Add dependency

testImplementation "io.github.adven27:exam-ui:{{version}}"

2. Configure and attach the UiPlugin

Commands

There is a command for starting browser, open an URL and executing step methods.

Open browser

browser

Starts browser and executes step methods:


<e:browser url=":8888/sut/url" failFast="false">
    <e:step name="someTextCheck" set="checkResult" desc="Checking text and setting result to variable">text to check</e:step>
    <e:step name="usePrevResultAndText(#checkResult, #TEXT)" desc="Do something with previous check result and some text">some text</e:step>
    <e:step name="doSomething" desc="Just execute doSomething method"/>
</e:db-check>

Appendices

Commands

List of all commands provided by Exam.

Examples

Example projects.

Getting Started

Markdown support

Markdown support.

Markdown support