Assertions

Article Summary

Assertions allow you to compare the actual state of your application to your expectations. Assertions are necessary to conclusively determine test success.

Assertions begin with the invocation of the t.expect method. The following simple assertion compares variable x to variable y and succeeds if the two are equal:

await t.expect(x).eql(y);

The left side of the assertion can contain asynchronous functions that extract information from the page. TestCafe offers a set of assertion methods that compare and evaluate assertion operands. TestCafe employs the Smart Assertion Query Mechanism to eliminate false negatives.

Table of Contents

Why use assertions

Tests without an explicit success condition are inconclusive.

If one of the following errors occurs, the test automatically fails:

  1. TestCafe cannot reach the test page URL.
  2. TestCafe cannot perform a test action — for example, if the target element does not exist.
  3. Your website throws a JavaScript error.

But you still don’t know if test actions have had the desired effect. That’s why you should add assertions (custom success conditions) to your test.

Examples

If your test includes a log-in routine, you may use assertions to perform the following checks:

  1. Log-in status check

    Confirm that the log-in routine has been successful. Check the page for items that are invisible to unauthenticated users:

    await t.expect(Selector('#account-preferences').filterVisible().exists).ok();

  2. User check

    Check that the page displays the correct username.

    await t.expect(Selector('#user-name').innerText).contains('Jane Doe');

  3. Cookies check

    Confirm that the user received the cookies necessary to continue the session:

    let cookies = await t.getCookies({ domain: 'yourwebsite.com' });
await t.expect(cookies.length).eql(2);

Assertion structure

The following simple assertion compares variable x to variable y and succeeds if the two are equal:

await t.expect(x).eql(y);

This assertion consists of the following parts:

The assertion declaration

Assertions start with the invocation of the t.expect method. Assertions are asynchronous methods, and require the use of the await keyword.

You can declare an assertion in any function that has access to the Test Controller, such as the test body, test hooks, and Role definitions. Manually import the TestController object to declare assertions elsewhere.

The first operand

The first operand indicates the actual state of the application. It can contain an asynchronous function that extracts page information:

await t.expect(Selector('#article-header').innerText).contains('10 Best Vacation Spots');

When you pass a compatible function to the assertion, TestCafe engages the Smart Assertion Query Mechanism. If the function fails the first time, TestCafe executes it again to account for possible client-side changes.

The assertion method

The assertion method determines the nature of the comparison. For example. the eql assertion method succeeds when the two operands are equal. See the assertion methods section for an overview of available assertion methods.

The second operand

The second operand indicates the expected value of the assertion’s first operand. The second operand of the assertion cannot contain a function.

How to chain assertions

You can chain assertions with other TestController methods:

// TestCafe executes the 'click' action after the assertion:
t.expect(x).eql(y).click('#button');

You can only chain complete assertions:

// The test fails to start because the assertion is incomplete:
t.expect(x).click('#button');

Use assertions to extract page information

To extract information from the page, pass an asynchronous function to the first operand.

If the function fails, TestCafe retries it multiple times within the assertion timeout. See Smart Assertion Query Mechanism for more information.

TestCafe automatically awaits the following functions:

  • Client functions

    const getLocationPart = ClientFunction(locationPart => {
    return window.location[locationPart];
});

await t.expect(getLocationPart('host')).eql('devexpress.github.io');

  • Selector property invocations

    await t.expect(Selector('#article-header').innerText).contains('10 Best Vacation Spots');

  • Two RequestLogger methods — count and contains

    import { RequestLogger } from 'testcafe';

const logger = RequestLogger();

fixture('RequestLogger')
    .page('https://devexpress.github.io/testcafe/example/')
    .requestHooks(logger);

test('Check request', async t => {
    await t.expect(logger.count()).ok();
    await t.expect(logger.contains(record => record.response.statusCode === 200)).ok();
});

  • HTTP requests and responses

    const responseBody = t.request('http://localhost:3000/helloworld').body;
await t.expect(responseBody).contains('Hello World');

Common Errors and Best Practices

  • Do not place Selector queries without properties into the left-hand side of the assertion:

    test('test', async t => {
    await t
    .typeText('input', '123')
    .expect(Selector('input')).contains('123'); // fails
});

  • If you place theawait keyword into the left-hand side of the assertion, TestCafe does not enage the Smart Assertion Query Mechanism.

    test('test', async t => {
    await t
    .typeText('input', '123')
    .expect(await Selector('input').value).eql('123');
});

  • If you pass the result of Selector evaluation to the assertion, TestCafe does not enage the Smart Assertion Query Mechanism.

    test('test', async t => {

const buttonValue = await Selector('#btn').textContent; // The constant stores the return value of the Selector query. 

await t.expect(buttonValue).contains('Loading...'); // This assertion compares two static values. if the assertion fails, TestCafe does not retry it, because the result would not change.

await t.expect(Selector('#btn').textContent).contains('Loading...'); // This assertion contians a proper Selector query. TestCafe retries the assertion in case of failure.
});

  • Do not place a function that returns a promise in the left hand-side of the assertion.

Assertion methods

TestCafe offers a comprehensive set of assertion methods.

Different assertion methods require different arguments. For example, the within method requires two numeric arguments. The ok and notOk methods are Boolean, and do not require an argument. Refer to the documentation of a specific assertion method for more information.

We can divide assertion methods into groups based on their mathematical function:

Strict equality

The following assertion methods perform a strict equality check:

eql (X = Y)

Method reference: eql

Examples:

// successful assertions:
await t.expect(20).eql(20);
await t.expect('20').eql('20');
await t.expect({ username: 'steve@example.com' }).eq({ username: 'steve@example.com' });

// failed assertions:
await t.expect(15).eql(20);
await t.expect('20').eql('15');
await t.expect({ username: 'steve@example.com' }).eq({ username: 'dave@example.com' });
// invalid assertions:
await t.expect('20').eql(20); // operand data types don't match

notEql (X ≠ Y)

Method reference: notEql

Examples:

// successful assertions:
await t.expect(15).notEql(20);
await t.expect('20').notEql('15');
await t.expect({ username: 'steve@example.com' })notEql({ username: 'dave@example.com' });

// failed assertions:
await t.expect(20).notEql(20);
await t.expect('20').notEql('20');
await t.expect({ username: 'steve@example.com' })notEql({ username: 'steve@example.com' });

// invalid assertions:
await t.expect('20').notEql(20);  // operand data types don't match

Value comparison

The following assertion methods compare two numeric values:

gt (X > Y)

Method reference: gt

Examples:

const today = new Date(); // current date
const lovelessReleaseDate = new Date(1991, 10, 4); // November 4th, 1991

// successful assertions:
await t.expect(20).gt(15)
await t.expect(today).gt(lovelessReleaseDate);

// failed assertions:
await t.expect(15).gt(15)
await t.expect(lovelessReleaseDate).gt(today);

// invalid assertions:
await t.expect('16').gt(15); // invalid data type (string)
await t.expect(today).gt(15); // operand data types don't match

gte (X ≥ Y)

Method reference: gte

Examples:

const today = new Date(); // current date
const lovelessReleaseDate = new Date(1991, 10, 4); // November 4th, 1991

// successful assertions:
await t.expect(20).gte(15);
await t.expect(15).gte(15);
await t.expect(today).gte(lovelessReleaseDate);
await t.expect(today).gte(today);

// failed assertions:
await t.expect(14).gte(15);
await t.expect(lovelessReleaseDate).gte(today);

// invalid assertions:
await t.expect('16').gte(15); // invalid data type (string)
await t.expect(today).gte(15); // operand data types don't match

lte (X ≤ Y)

Method reference: lte

Examples:

const today = new Date(); // current date
const lovelessReleaseDate = new Date(1991, 10, 4); // November 4th, 1991

// successful assertions:
await t.expect(14).lte(15);
await t.expect(15).lte(15);
await t.expect(lovelessReleaseDate).lte(today);
await t.expect(today).lte(today);

// failed assertions:
await t.expect(20).lte(15);
await t.expect(today).lte(lovelessReleaseDate);

// invalid assertions:
await t.expect('14').lte(15) // invalid data type (string)
await t.expect(today).lte(15) // operand data types don't match

lt (X < Y)

Method reference: lt

const today = new Date(); // current date
const lovelessReleaseDate = new Date(1991, 10, 4); // November 4th, 1991

// successful assertions:
await t.expect(14).lt(15)
await t.expect(lovelessReleaseDate).lt(today);

// failed assertions:
await t.expect(15).lt(15)
await t.expect(today).lt(lovelessReleaseDate);

// invalid assertions:
await t.expect('14').lt(15); // invalid data type (string)
await t.expect(today).lt(15); // operand data types don't match

Superset

The following assertion methods check whether X includes Y:

contains (Y ∈ X)

Method reference: contains

Examples:

// successful assertions:
await t.expect(['x','y']).contains('y');
await t.expect('Username: steve@example.com').contains('Username');
await t.expect({ username: 'steve@example.com', subscriptionPlan: 'basic' }).contains({ username: 'steve@example.com' });

// failed assertions:
await t.expect(['x','y']).contains('z');
await t.expect('Username: steve@example.com').contains('Password');
await t.expect({ username: 'steve@example.com', subscriptionPlan: 'basic' }).contains({ username: 'dave@example.com' });

// invalid assertions:
await t.expect({ username: 'steve@example.com', subscriptionPlan: 'basic' }).contains('steve@example.com'); // operand data types don't match

notContains (Y ∉ X)

Method reference: notContains

Examples: 

// successful assertions:
await t.expect(['x','y']).notContains('z');
await t.expect('Username: steve@example.com').notContains('Password');
await t.expect({ username: 'steve@example.com', subscriptionPlan: 'basic' }).notContains({ username: 'dave@example.com' });

// failed assertions:
await t.expect(['x','y']).notContains('y');
await t.expect('Username: steve@example.com').notContains('Username');
await t.expect({ username: 'steve@example.com', subscriptionPlan: 'basic' }).notContains({ username: 'steve@example.com' });

// invalid assertions:
await t.expect({ username: 'steve@example.com', subscriptionPlan: 'basic' }).notContains('dave@example.com'); // operand data types don't match

Numeric range check

The following assertion methods check whether the Y range of numbers includes X.

within (X ∈ [Yᵃ, Yᵇ])

Method reference: within

Examples:

await t.expect(10).within(1, 20); // success
await t.expect(22).within(1, 20); // failure

// invalid assertions:
await t.expect(22).within([20, 100]); // the second operand is an array
await t.expect(22).within(1); // insufficient number of range definition arguments
await t.expect(10).within(20, 1); // invalid range definition: 20 is greater than 1

notWithin (X ∉ [Yᵃ, Yᵇ])

Method reference:notWithin

Examples:

await t.expect(22).notWithin(1, 20); // success
await t.expect(10).notWithin(1, 20); // failure

// invalid assertions:
await t.expect(12).notWithin([20, 100]); // the second operand is an array    
await t.expect(22).notWithin(1); // insufficient number of range definition arguments
await t.expect(22).notWithin(20, 1); // invalid range definition: 20 is greater than 1

Truthiness check

The following assertion methods check the Boolean value of X.

ok (⊤(X))

Method reference: ok

const inputField = Selector('#developer-name');
const fakeElement = Selector('#this-element-does-not-exist');

// successful assertions:
await t.expect(inputField.exists).ok();
await t.expect('Hello!').ok();

// failed assertions:
await t.expect(fakeElement.exists).ok(); // the element doesn't exist  
await t.expect('').ok(); // the string is empty
await t.expect(null).ok(); // null isn't truthy
await t.expect(undefined).ok(); // undefined isn't truthy

notOK (⊥(X))

Method reference: notOk

Examples: 

const inputField = Selector('#developer-name');
const fakeElement = Selector('#this-element-does-not-exist');

// successful assertions:
await t.expect(fakeElement.exists).notOk(); // the element doesn't exist  
await t.expect('').notOk(); // the string is empty
await t.expect(null).notOk(); // null is falsy
await t.expect(undefined).notOk(); // undefined is falsy

// failed assertions:
await t.expect(inputField.exists).notOk();
await t.expect('Hello!').notOk();

Type check

The following assertion methods check whether X belongs to data type Y.

typeOf (X ∈ type Y)

Method reference: typeOf

Examples:

await t.expect(12).typeOf('Number'); // success
await t.expect(12).typeOf('Object'); // failure

notTypeOf (X ∉ type Y)

Method reference: notTypeOf

Examples:

await t.expect(12).notTypeOf('Object'); // success
await t.expect(12).notTypeOf('Number'); // failure

Regular expression check

The following assertion methods check whether X matches the regular expression Y:

match

Method reference: match

X matches the regular expression Y.

Examples:

const emailRegex = new RegExp(/^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/);

await t.expect('email@email.com').match(emailRegex); // success
await t.expect('email@email.com').match(/^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/); // success
await t.expect('email.com').match(emailRegex); // failure

notMatch

Method reference: notMatch

X does not match the regular expression Y.

Examples:

const emailRegex = new RegExp(/^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/);

await t.expect('email.com').notMatch(emailRegex); // success
await t.expect('email.com').notMatch(/^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/); // success
await t.expect('email@email.com').notMatch(emailRegex); // failure

Assertion options

Custom error message

Type: String

You can define a custom error message for the assertion. Pass the string with the message to the assertion method:

await t.expect({ a: 'bar' }).eql({ a: 'foo' }, 'this assertion will fail');

Assertion Failure Custom Message

Assertion timeout

Type: Number

If an assertion’s first operand contains a compatible function, the assertion is subject to the Smart Assertion Query Mechanism.

If such an assertion fails, TestCafe executes it again until it meets either of the following criteria:

  • The assertion succeeds.
  • The assertion timeout elapses.
await t.expect(Selector('h1').innerText).eql('text', 'check element text', { timeout: 20000 });

To set the timeout for the entire test run, define the assertion timeout in one of the following ways:

Note

The timeout option applies to built-in TestCafe assertion methods. Use the t.wait() method to specify timeouts for third-party assertion methods (assert or chai).

Allow Unawaited Promise

TestCafe awaits Promises from compatible asynchronous functions. If your assertion includes a custom function that returns a Promise, the assertion fails.

When you create assertions, avoid the use of custom functions that return a Promise. If you can’t work around this limitation, use the allowUnawaitedPromise option:

await t
        .expect(new Promise(resolve => setTimeout(resolve, 100)))
        .ok('received a promise', { allowUnawaitedPromise: true });

How Assertions Work

Functional web tests are fundamentally asynchronous. It takes time for the application to respond to user actions.

The application’s reponse speed depends on factors such as the speed of the network, the speed of the database, and front-end animations. Since the response is not immediate, it is often impossible to execute assertions right after the action ends.

Traditional end-to-end frameworks solve this issue with an extra timeout:

Asynchronous Functional Testing with Extra Waiting

TestCafe improves on this approach with its Smart Assertion Query Mechanism.

Smart Assertion Query Mechanism

If the first operand of an assertion is a compatible asynchronous function — for example, a Selector property — TestCafe retries the assertion multiple times within the assertion timeout.

The timeout option sets the timeout value on a per-assertion basis. To set the timeout for the entire test run, pass the timeout value to the assertion-timeout CLI option or the runner.run option (Test Runner API).

The test fails if the assertion doesn’t succeed by the end of the timeout period.

TestCafe Smart Assertion Query Mechanism

Example:

Consider the following HTML page:

<html>
<body>
    <div id="btn" onclick="window.setTimeout(() => this.textContent = 'Loading...', 100)">Click me!</div>
</body>
</html>

When the user clicks the #btn element, the browser initiates a 100ms wait period. When the wait period elapses, the browser changes the button’s content.

A regular assertion would fail because the application does not respond instantly. TestCafe accounts for the possibility of a slow reponse, and the following assertion succeeds:

test('Button click', async t => {
    const btn = Selector('#btn');

    await t
        .click(btn)
        .expect(btn.textContent).contains('Loading...');
});

Debug Assertions

Assertions may fail thanks to a badly written Element Selector query or a short assertion timeout. Read the Debug Tests guide for more information.