Run Tests

Article Summary

There are two ways to launch TestCafe tests.

1. The simplest option is to use the testcafe shell command:

   testcafe chrome test.js
   

2. The Test Runner API allows you to launch and configure TestCafe tasks with JavaScript code.

To launch TestCafe, specify the target browser and test files.

To configure your test run, use the configuration file, or specify CLI options and flags.

Live mode allows you to reload tests as you write them. Quarantine mode retries failed tests to eliminate false negatives.

You can run tests concurrently to speed up your test run.

Table of Contents

• Command Line Interface Overview
• Test Runner API Overview
• Select Target Browsers
• Run tests in multiple browsers
• Select Test Files
• Options and flags
• Alternative test modes
• Use a Proxy
• View Test Results
• Further Reading

Command Line Interface

The command line interface of TestCafe is simple and straightforward. For most users, it’s the preferred way to launch TestCafe tests.

testcafe <browsers> <test files> <options>

Test Runner API

The Test Runner API is a Node.js API that can launch, configure and terminate TestCafe tests. It works with JavaScript and TypeScript.

Advantages

The Test Runner has the following advantages over the CLI:

• You can introduce additional logic to gain fine control over the execution of your test suite.
• You can repeat the same test in a number of different run configurations without restarting it by hand.
• You can integrate TestCafe with your favorite Node.js tools, such as gulp, create-react-app, or Angular Builders.
• You can commit the test launch script to version control, with all the maintenance benefits that it entails.

Quick Guide

1. Use the testcafe.createRunner method to create a Runner instance.
2. Add configuration methods.
   • Use the runner.src to specify the test file path or directory.
   • Use the runner.browsers method to specify the browser(s) you would like to run tests in.
3. Call the runner.run method to run tests.
4. If necessary, stop the test run with the runner.stop method.

The following example runs two test files in chrome and safari:

const createTestCafe = require('testcafe');
const testcafe = await createTestCafe('localhost', 1337, 1338);

try {
    const runner = testcafe.createRunner();

    const failed = await runner
        .src(["tests/fixture1.js", "tests/func/fixture3.js"])
        .browsers(["chrome", "edge"])
        .run();

    console.log('Tests failed: ' + failed);
}
finally {
    await testcafe.close();
}

Select Target Browsers

TestCafe supports a wide array of modern browsers. Additionally, you can run tests on Mobile devices and mobile device emulators. Read the advanced Browsers guide for more information about custom browsers, headless browsers, and more.

CLI

Specify the browser alias as the first command line argument.

testcafe chrome my-fixture.js

Runner

Use the runner.browsers method to select the target browser:

await runner
    .browsers('chrome')
    .src('./tests/')
    .run();

Run tests in multiple browsers

TestCafe uses Native Automation to run tests in local Chromium-based browsers. Native automation increases test speed and stability. To run tests in multiple Chromium-based browsers, separate browser aliases with a comma:

# CLI
testcafe chrome,edge my-fixture.js

/* TestCafe Test Runner */
await runner
    .browsers(['chrome', 'edge'])
    .src('./tests/')
    .run();

If your browser string includes non-Chromium-based browsers, TestCafe disables Native Automation:

testcafe chrome,firefox my-fixture.js # Runs without native automation.

If you want to run tests in non-Chromium-based browsers and take advantage of Native Automation, create two separate test runs:

testcafe chrome,edge my-fixture.js # Chromium-based browsers
testcafe firefox,safari my-fixture.js --disable-native-automation # Other browsers

Use the all alias to launch the test in all your local browsers:

testcafe all my-fixture.js

Configuration file

Use the browsers configuration file option to select the target browser in your configuration file:

{
    "browsers": "chrome"
}

Pass an array of browser aliases to select multiple browsers:

{
    "browsers": ["ie", "firefox"]
}

If you specify target browsers in a configuration file, you can omit the corresponding CLI option.

Select Test Files

CLI

Include the path to the test file or directory as the second command line argument:

testcafe chrome my-fixture.js

You can specify multiple test files and directories:

testcafe safari my-fixture.js ./studio/tests

Runner

Use the runner.src method to specify test files and test file directories:

await runner
    .src('my-fixture.js')
    .browsers('chrome')
    .run();

If you want to specify multiple test files and directories, store them in an array:

await runner
    .browsers('safari')
    .src(['my-fixture.js', './studio/tests'])
    .run();

Configuration File

Use the src configuration file option to specify test files and test file directories in your configuration file:

{
    "src": "/home/user/tests/fixture.js"
}

If you want to specify multiple test files and directories, store them in an array:

{
    "src": ["/home/user/auth-tests/fixture.testcafe", "/home/user/mobile-tests/"]
}

If you specify test files in a configuration file, you can omit the corresponding CLI option.

Options and flags

There are several ways to configure your test run. Both CLI users and Runner users can save their settings to a configuration file.

CLI

CLI users can specify command line options. The example below executes tests from the my-tests folder at the 0.1 speed.

testcafe chrome my-tests --speed 0.1

Runner

Test Runner users can pass Test Runner Options to the runner.run method. The example below executes tests from the my-tests folder at the 0.1 speed:

const createTestCafe = require('testcafe');
const testcafe = await createTestCafe('localhost', 1337, 1338);

try {
    const runner = testcafe.createRunner();

    const failed = await runner
        .src(["my-tests"])
        .browsers(["chrome", "edge"])
        .run({
        speed: 0.1
    });

    console.log('Tests failed: ' + failed);
}
finally {
    await testcafe.close();
}

Configuration File

Use the configuration file to store test settings:

{
    "browsers": ["chrome", "safari"],
    "src": "/home/user/tests/fixture.js",
    "speed": 0.1
}

Alternative test modes

Live Mode

Main article: Live Mode

When you run TestCafe in Live Mode, TestCafe restarts your test suite every time you change the test code.

When the test is over, TestCafe keeps the page open so you can inspect it.

To start TestCafe in live mode, use the -L (--live) CLI flag or the testcafe.createLiveModeRunner Test Runner API function.

Quarantine Mode

Main article: Quarantine Mode

Enable quarantine mode to eliminate false negatives and detect unstable tests. When a test fails, TestCafe quarantines it, and repeats it until the test yields conclusive results.

To start TestCafe in quarantine mode, use the -q (--quarantine-mode) CLI flag, or the quarantineMode Test Runner API option.

Use a Proxy

If your network uses a proxy to access the test page, you need to specify the proxy URL. Use the --proxy command line argument or the runner.useProxy API method:

testcafe chrome ./my-tests/ --proxy proxy.mycompany.com

await runner
    .browsers('chrome')
    .src('./my-tests/')
    .useProxy('proxy.mycompany.com')
    .run();

You can also specify URLs that should bypass the proxy. Pass the list of URLs in the --proxy-bypass command line argument or a runner.useProxy parameter:

testcafe chrome ./my-tests/ --proxy proxy.corp.mycompany.com --proxy-bypass localhost:8080

await runner
    .browsers('chrome')
    .src('./my-tests/')
    .useProxy('proxy.corp.mycompany.com', 'localhost:8080')
    .run();

You can also specify the proxy URL in the configuration file.

View Test Results

When the test run is over, TestCafe displays the test report in the command shell. The reporter determines the format of the output. TestCafe includes five built-in reporters.

The following screenshot shows the result of a successful test, as reported by spec, the default reporter.

If a test fails, TestCafe outputs the error and highlights the failed action:

Feel free to create a custom reporter plugin, or use one of the community reporters.

Further Reading

• Learn how to run tests concurrently.
• Learn how to use live mode to reload tests as you write them.
• Learn how to quarantine tests to eliminate false negatives.
• Learn how to execute shell commands on startup to deploy your app before the test run.