Run Tests

You can run TestCafe tests from the command line or JavaScript/TypeScript API.

testcafe safari ./tests/my-fixture.js

const createTestCafe = require('testcafe');

const testCafe = await createTestCafe('localhost', 1337, 1338);

try {
    const runner = testCafe.createRunner();

    await runner
        .src('./tests/my-fixture.js')
        .browsers('safari')
        .run();
}
finally {
    await testCafe.close();
}

TestCafe also allows you to create a configuration file where you can define test run settings. You can then omit these settings in the command line or API to use values from the configuration file.

Important

Front-end development tools (such as React DevTools or Vue DevTools) can interfere with TestCafe and cause errors. Do not open them while you run or debug TestCafe tests.

Specify Tests to Run

You should specify a path to a file or directory with tests you want to run in the second command line argument:

testcafe chrome ./tests/

In the API, use the runner.src method:

await runner
    .browsers('chrome')
    .src('./tests/')
    .run();

Related configuration file property: src

Run Tests from Multiple Sources

You can specify multiple test files or directories:

testcafe safari ./js-tests/fixture.js ./studio-tests/fixture.testcafe

await runner
    .browsers('safari')
    .src(['./js-tests/fixture.js', './studio-tests/fixture.testcafe'])
    .run();

Use Glob Patterns

TestCafe also supports glob patterns to run a set of files that match a specified pattern:

testcafe firefox ./tests/*mobile*

await runner
    .browsers('firefox')
    .src('./tests/*mobile*')
    .run();

Filter Tests and Fixtures by Name

Use the -t (--test) command line argument or runner.filter method to run a test by name:

testcafe safari ./tests/sample-fixture.js -t "Click a label"

await runner
    .browsers('safari')
    .src('./tests/sample-fixture.js')
    .filter(testName => testName === 'Click a label')
    .run();

Related configuration file property: filter.test

You can also use the -T (--test-grep) argument to specify a grep pattern for the test name:

testcafe chrome ./my-tests/ -T "Click.*"

Related configuration file property: filter.testGrep

To run a fixture by name, use the -f (--fixture) argument:

testcafe firefox ./my-tests/ -f "Sample fixture"

The runner.filter method’s predicate accepts the fixtureName parameter:

await runner
    .browsers('firefox')
    .src('./my-tests/')
    .filter((testName, fixtureName) => fixtureName === 'Sample fixture')
    .run();

Related configuration file property: filter.fixture

To use grep patterns for a fixture name, specify the -F (--fixture-grep) option:

testcafe safari ./my-tests/ -F "Page.*"

Related configuration file property: filter.fixtureGrep

Filter Tests and Fixtures by Metadata

You can also run tests whose metadata contains specific values. Use the --test-meta argument to do this:

testcafe chrome ./my-tests/ --test-meta device=mobile,env=production

The runner.filter method’s predicate accepts the testMeta parameter:

await runner
    .browsers('chrome')
    .src('./my-tests/')
    .filter((testName, fixtureName, fixturePath, testMeta) => {
        return testMeta.device === 'mobile' && testMeta.env === 'production';
    })
    .run();

Related configuration file property: filter.testMeta

To filter fixtures with specific metadata, use the --fixture-meta argument:

testcafe firefox ./my-tests/ --fixture-meta device=mobile,env=production

In runner.filter, the fixture metadata is available in the fixtureMeta parameter:

await runner
    .browsers('firefox')
    .src('./my-tests/')
    .filter((testName, fixtureName, fixturePath, testMeta, fixtureMeta) => {
        return fixtureMeta.device === 'mobile' && fixtureMeta.env === 'production';
    })
    .run();

Related configuration file property: filter.fixtureMeta

Specify Target Browsers

You should specify the browsers where you want to run tests in the first command line parameter.

TestCafe automatically detects supported browsers installed on the local machine. Use browser aliases to identify the target browser:

testcafe chrome ./tests/

In the API, use the runner.browsers method:

await runner
    .browsers('chrome')
    .src('./tests/')
    .run();

Related configuration file property: browsers

Use Multiple Browsers

You can run tests in several browsers. In the command line, specify a comma-separated browser list:

testcafe safari,chrome ./tests/

In the API, pass an array of browser identifiers to runner.browsers:

await runner
    .browsers(['safari', 'chrome'])
    .src('./tests/')
    .run();

Run Tests in All Installed Browsers

Use the all alias to run tests in all locally installed browsers TestCafe can detect:

testcafe all ./tests/

await runner
    .browsers('all')
    .src('./tests/')
    .run();

Test in Portable Browsers

You can also specify the path to a browser executable to launch portable browsers. Use the path: prefix followed by the full path:

testcafe path:d:\firefoxportable\firefoxportable.exe ./tests/

await runner
    .browsers('path:d:\firefoxportable\firefoxportable.exe')
    .src('./tests/')
    .run();

Note

If the path contains spaces, you should escape them in the command line as described in Command Line Interface.

Use Headless Mode

TestCafe can run tests in headless mode in browsers that support it. To run tests in headless mode, put the :headless suffix after the browser name:

testcafe firefox:headless ./tests/

await runner
    .browsers('firefox:headless')
    .src('./tests/')
    .run();

Enable Mobile Device Emulation

You can use Google Chrome mobile device emulation to test mobile layout and features on desktops. Specify the :emulation postfix followed by emulation options:

testcafe "chrome:emulation:device=iphone X" ./tests/sample-fixture.js

await runner
    .browsers('chrome:emulation:device=iphone X')
    .src('./tests/sample-fixture.js')
    .run();

Test in Cloud Testing Services

TestCafe can also run tests in cloud testing services such as BrowserStack or SauceLabs. Install the browser provider for your service and specify the browser alias as described in the browser provider documentation.

For instance, to use SauceLabs, install the testcafe-browser-provider-saucelabs module from npm and run tests as follows:

testcafe "saucelabs:Chrome@52.0:Windows 8.1" ./tests/sample-fixture.js

await runner
    .browsers('saucelabs:Chrome@52.0:Windows 8.1')
    .src('./tests/sample-fixture.js')
    .run();

Test on Remote and Mobile Devices

To run tests remotely on a mobile device or a computer with no TestCafe installation, specify remote instead of the browser alias in the command line:

testcafe remote ./tests/sample-fixture.js

TestCafe generates a URL and displays it in the console. When you visit this URL from the remote device, TestCafe runs tests in this browser. To run tests in several remote browsers, specify their number after the remote keyword: remote:2 or remote:4.

In the API, create a remote browser connection with the testcafe.createBrowserConnection method, visit the generated URL and run tests once the connection is initialized:

const createTestCafe   = require('testcafe');
const testCafe         = await createTestCafe('localhost', 1337, 1338);
const runner           = testCafe.createRunner();
const remoteConnection = testcafe.createBrowserConnection();

// Visit this URL from the remote device.
console.log(remoteConnection.url);

// Wait until the remote device's browser connects.
await new Promise(resolve => remoteConnection.once('ready', resolve));

await runner
    .src('./tests/sample-fixture.js')
    .browsers(remoteConnection)
    .run();

Specify the Report Format

A reporter is a module that formats and outputs test run results. TestCafe ships with five basic reporters, including reporters for spec, JSON, and xUnit formats. You can install other reporters as plugins or create a custom reporter.

Use the -r (--reporter) flag in the command line and the runner.reporter method in the API to specify which reporter to use.

testcafe all ./tests/sample-fixture.js -r xunit

await runner
    .browsers('all')
    .src('./tests/sample-fixture.js')
    .reporter('xunit')
    .run();

testcafe all ./tests/sample-fixture.js -r my-reporter

await runner
    .browsers('all')
    .src('./tests/sample-fixture.js')
    .reporter('my-reporter')
    .run();

Related configuration file property: reporter

To define the output target, specify it after a semicolon in the command line or as the second parameter in runner.reporter.

testcafe all ./tests/sample-fixture.js -r json:report.json

await runner
    .browsers('all')
    .src('./tests/sample-fixture.js')
    .reporter('json', 'report.json')
    .run();

You can use multiple reporters, but only one reporter can write to stdout:

testcafe all ./tests/sample-fixture.js -r spec,xunit:report.xml

await runner
    .browsers('all')
    .src('./tests/sample-fixture.js')
    .reporter(['spec', {
        name: 'xunit',
        output: 'report.xml'
    })
    .run();

Customize Screenshot and Video Settings

TestCafe can take screenshots of the tested page automatically when a test fails. You can also capture screenshots at arbitrary moments with the t.takeScreenshot and t.takeElementScreenshot actions.

Use the -s (--screenshots) command line flag or the runner.screenshots API method.

testcafe all ./tests/sample-fixture.js -s path=artifacts/screenshots,takeOnFails=true

await runner
    .browsers('all')
    .src('./tests/sample-fixture.js')
    .screenshots({
        path: 'artifacts/screenshots',
        takeOnFails: true
    })
    .run();

Related configuration file property: screenshots

To record videos of test runs, pass the --video command line flag or use the runner.video API method.

You can also specify video recording options in the --video-options command line argument or a runner.video parameter:

testcafe chrome ./test.js --video ./videos/ --video-options singleFile=true,failedOnly=true

await runner
    .browsers('chrome')
    .src('./test.js')
    .video('./videos/', {
        singleFile: true,
        failedOnly: true
    })
    .run();

Related configuration file properties:

Run Tests Concurrently

To speed up testing, TestCafe allows you to execute tests concurrently. In concurrent mode, TestCafe invokes multiple instances of each browser. For example, the testcafe chrome, firefox test.js --concurrency 3 command invokes 3 Chrome instances and 3 Firefox instances (6 browser instances in total). These instances constitute the pool of browsers against which tests run concurrently, that is each test runs in the first available instance.

TestCafe invokes the specified number of browser instances despite of the number of tests in the tested fixture. Empty browser instances that run no tests remain on the screen. For example, if you specify the --concurrency 3 CLI flag for a fixture with 2 tests, TestCafe runs two tests in two browser instances and keeps the third instance in standby mode.

TestCafe executes fixture hooks before all tests run (the before hook) and when all tests in the fixture are complete (the after hook).

Hooks in a fixture are synchronization points between fixture executions. TestCafe executes the next fixture only when all tests and hooks in the previous fixture are complete.

To enable concurrency, use the -c (--concurrency) command line option or the runner.concurrency API method.

Important

Microsoft Edge Legacy does not support concurrent test execution, because it cannot be started in a new window, not can it open a specified URL.

The following CLI command invokes three Chrome instances and runs tests concurrently:

testcafe -c 3 chrome tests/test.js

To run tests with the same settings in code, use the following code sample: 

var testRunPromise = runner
    .src('tests/test.js')
    .browsers('chrome')
    .concurrency(3)
    .run();

Related configuration file property: concurrency

Note that you can also use concurrency when testing against multiple browsers.

testcafe -c 4 safari,firefox tests/test.js

In this case, TestCafe distributes tests across four Safari instances and runs the same tests in four Firefox instances.

Note

If an uncaught error or unhandled promise rejection occurs on the server during test execution, all concurrently running tests fail.

Use Concurrency on Remote Devices

When you run tests on remote devices, create browser connections for each instance of each browser you test against. The total number of instances is c*n, where c is the concurrency factor and n is the number of browsers.

In the command line interface, specify the number of browser instances after the remote: keyword. For example, use -c 2 remote:6 to run tests in Chrome, Firefox, and Edge, with two instances of each browser. If you use the API, create a browser connection for each instance in code.

Launch all the required browser instances manually on a remote device. The total number of instances should be divisible by the concurrency factor c. Otherwise, an exception is thrown.

testcafe -c 2 remote:4 tests/test.js

If you test against multiple remote browsers, open and connect all instances of one browser before you connect the next browser.

Stage the Tested App

TestCafe can execute a specified shell command before it starts tests. For instance, you can run a command that starts a local web server and deploys the tested app. TestCafe automatically terminates the process when tests are finished.

Use the -a (--app) CLI flag or the runner.startApp API method to provide a command:

testcafe chrome ./my-tests/ --app "node server.js"

await runner
    .browsers('chrome')
    .src('./my-tests/')
    .startApp('node server.js')
    .run();

Related configuration file property: appCommand

TestCafe delays tests to allow the shell command to execute. The default timeout is 1000 milliseconds. Use the --app-init-delay CLI flag or a runner.startApp parameter to specify the timeout value.

testcafe chrome ./my-tests/ --app "node server.js" --app-init-delay 4000

await runner
    .browsers('chrome')
    .src('./my-tests/')
    .startApp('node server.js', 4000)
    .run();

Related configuration file property: appInitDelay

Specify a Proxy URL

If your network uses a proxy to access the internet, specify the proxy URL to TestCafe. Use the --proxy command line argument or the runner.useProxy API method:

testcafe chrome ./my-tests/ --proxy proxy.mycompany.com

await runner
    .browsers('chrome')
    .src('./my-tests/')
    .useProxy('proxy.mycompany.com')
    .run();

Related configuration file property: proxy

You can also specify URLs that should be accessed directly. Pass the list of URLs in the --proxy-bypass command line argument or a runner.useProxy parameter:

testcafe chrome ./my-tests/ --proxy proxy.corp.mycompany.com --proxy-bypass localhost:8080

await runner
    .browsers('chrome')
    .src('./my-tests/')
    .useProxy('proxy.corp.mycompany.com', 'localhost:8080')
    .run();

Related configuration file property: proxyBypass

Live Mode

Live mode ensures TestCafe and the browsers remain active while you work on tests. You can see test results instantly because the tests are restarted when you make changes.

Live mode demonstrated in action

Note

Live Mode replaces the deprecated testcafe-live module.

How to Enable Live Mode

Use the -L (--live) flag to enable live mode from the command line interface.

testcafe chrome tests/test.js -L

In the API, create a live mode runner with the testcafe.createLiveModeRunner function and use it instead of a regular test runner.

const createTestCafe = require('testcafe');

const testcafe = await createTestCafe('localhost', 1337, 1338);

try {
    const liveRunner = testcafe.createLiveModeRunner();

    await liveRunner
        .src('tests/test.js')
        .browsers('chrome')
        .run();
}
finally {
    await testcafe.close();
}

How Live Mode Works

When you run tests with live mode enabled, TestCafe opens the browsers, runs the tests, shows the reports, and waits for further actions.

Then TestCafe starts watching for changes in the test files and all files referenced in them (like page objects or helper modules). Once you make changes in any of those files and save them, TestCafe immediately reruns the tests.

When the tests are done, the browsers stay on the last opened page so you can work with it and explore it with the browser’s developer tools.

You can use live mode with any browsers: local, remote, mobile or headless.

Important

Live mode is designed to work with tests locally. Do not use it in CI systems.

Tip

Use the only function to work with a single test.

Console Shortcuts in Live Mode

  • Ctrl+S - stops the current test run;
  • Ctrl+R - restarts the current test run;
  • Ctrl+W - turns off/on the file watcher;
  • Ctrl+C - closes opened browsers and terminates the process.

Quarantine mode

Enable quarantine mode to eliminate false negatives and detect unstable tests. When a test fails, TestCafe quarantines it, and repeats it until the test yields conclusive results.

Note

Quarantine mode may increase the total run-time of your test suite.

Quarantine mode options

The successThreshold option (default value: 3) — the number of successful attempts necessary to confirm the test’s success. The attemptLimit option (default value: 5) — the maximum number of test attempts.

Note

The attemptLimit has to be greater than the successThreshold.

Success conditions

Quarantined tests succeed when the number of successful attempts reaches the successThreshold value.

Failure conditions

Quarantined tests fail as soon as it becomes impossible for them to succeed within the attemptLimit.

For example, the test fails immediately if: a) The number of successes is 0. b) You need 3 successful attempts to reach the successThreshold. c) You have fewer than 3 attempts left (before you reach the attemptLimit).

Stability status

If consecutive runs of a quarantined test yield varying outcomes, TestCafe marks the test as unstable. This means that the test’s outcome is, in all likelihood, inconclusive.

  • If test results do not change over time (the test succeeds or fails several times in a row), the test is stable, regardless of its success. You can trust the results.
  • If a test alternates between successes and failures, TestCafe marks the test as unstable. There is probably something wrong with the test.

Example

Enable quarantine mode with the -q (--quarantine-mode) command line flag, or the quarantineMode runner option:

testcafe chrome ./tests/ -q
testcafe chrome ./tests/ -q attemptLimit=5,successThreshold=2

await runner
    .browsers('chrome')
    .src('./tests/')
    .run({ quarantineMode: true });

await runner
    .browsers('chrome')
    .src('./tests/')
    .run({ quarantineMode: { successThreshold: 1, attemptLimit: 3 } });

See Martin Fowler’s Eradicating Non-Determinism in Tests article for more information about non-deterministic tests.

Troubleshooting

Disable the ‘Close Multiple Tabs’ Popup in IE

Internet Explorer displays a dialog when you try to close a browser window with more than one opened tab. This dialog blocks the page and pauses tests. Disable this dialog before you run tests.

To disable this dialog, open multiple tabs in one Internet Explorer window and close the browser. Check Always close all tabs and click Close all tabs. IE saves this setting and the dialog does not appear again.

Disable the `Close Multiple Tabs` Popup in IE

Repeat this action on any new machine where you run tests.