Runner.run Method

Runs tests according to the current configuration. Returns the number of failed tests.

async run(options) → Promise<Number>

Configuration conflicts

Command line options have precedence over TestCafe Runner options. Runner options have precedence over configuration file settings. When TestCafe overrides a configuration file setting, it outputs a description of the conflict to the console.

Important

Make sure the browser tab that runs tests stays active. Do not minimize the browser window. Inactive tabs and minimized browser windows switch to a lower resource consumption mode where tests are not guaranteed to execute correctly.

Options

You can pass the following options to the runner.run function:

baseUrl

Type: String

Description: Defines the starting URL for all the tests and fixtures in your test suite.

Default value: about:blank

Related Configuration File Option: baseUrl

skipJsErrors

Main article: Skip JavaScript Errors

Type: Boolean | Object {message, pageUrl, stack} | Object {fn ({message, pageUrl, stack}) => { }, dependencies} | fn ({message, pageUrl, stack}) => { }

Description: Use the skipJsErrors option to ignore JavaScript errors during the test run. Specify the message, page, and stack properties to ignore specific errors. Pass an object with a callback function and its dependencies to set custom error handling logic.

Default value: false

Related Configuration File Option: skipJsErrors

skipUncaughtErrors

Type: Boolean

Description: Defines whether to continue a test after an uncaught error or unhandled promise rejection occurs on the server (true), or consider such a test failed (false).

Default value: false

Related Configuration File Option: skipUncaughtErrors

quarantineMode

Type: Boolean | Object

Description: Defines whether to enable the quarantine mode and specify its options.

Default value: false

Related Configuration File Option: quarantineMode

debugMode

Type: Boolean

Description: Specifies if tests run in the debug mode. If this option is enabled, test execution is paused before the first action or assertion so that you can invoke the developer tools and debug. In the debug mode, you can execute the test step-by-step to reproduce its incorrect behavior. You can also use the Unlock page switch in the footer to unlock the tested page and interact with its elements.

Default value: false

Related Configuration File Option: debugMode

debugOnFail

Type: Boolean

Description: Specifies whether to enter the debug mode when a test fails. If enabled, the test is paused at the moment it fails, so that you can explore the tested page to determine what caused the failure.

Default value: false

Related Configuration File Option: debugOnFail

experimentalDebug

Type: Boolean

Description: Specifies whether to evaluate selectors and client functions in the Watch panel of a Node.js debugger.

Default value: false

Related Configuration File Option: experimentalDebug

selectorTimeout

Type: Number

Description: Specifies the time (in milliseconds) within which selectors make attempts to obtain a node to be returned. See Selector Timeout.

Default value: false

Related Configuration File Option: selectorTimeout

assertionTimeout

Type: Number

Description: Specifies the time (in milliseconds) within which TestCafe makes attempts to successfully execute an assertion if a selector property or a client function was passed as an actual value. See Smart Assertion Query Mechanism.

Default value: 3000

Related Configuration File Option: assertionTimeout

pageLoadTimeout

Type: Number

Description: Specifies the time (in milliseconds) TestCafe waits for the window.load event to fire after the DOMContentLoaded event. After the timeout passes or the window.load event is raised (whichever happens first), TestCafe starts the test. You can set this timeout to 0 to skip waiting for window.load.

Default value: 3000

Related Configuration File Option: pageLoadTimeout

ajaxRequestTimeout

Type: Number

Description: Wait time (in milliseconds) for fetch/XHR requests. If no response is received within the specified period, an error is thrown.

Default value: 120000

Related Configuration File Option: ajaxRequestTimeout

pageRequestTimeout

Type: Number

Description: Time (in milliseconds) to wait for HTML pages. If the page isn’t received within the specified period, an error is thrown.

Default value: 25000

Related Configuration File Option: pageRequestTimeout

browserInitTimeout

Type: Number

Description: Time (in milliseconds) for browsers to connect to TestCafe and report that they are ready to test. If one or more browsers fail to connect within this period, TestCafe throws an error.

Default value: 120000 for local browsers, 360000 for remote browsers.

Related Configuration File Option: browserInitTimeout

speed

Type: Number

Description: Specifies test execution speed. A number between 1 (fastest) and 0.01 (slowest). If an individual action’s speed is also specified, the action speed setting overrides the test speed.

Default value: 1

Related Configuration File Option: speed

stopOnFirstFail

Type: Boolean

Description: Stops the test run if a test fails. You can focus on the first error before all the tests finish.

Default value: false

Related Configuration File Option: stopOnFirstFail

disablePageCaching

Type: Boolean

Description: Prevents the browser from caching page content. When navigation to a cached page occurs in role code, local and session storage content is not preserved. Set disablePageCaching to true to retain the storage items after navigation. For more information, see Troubleshooting: Test Actions Fail After Authentication. You can also disable page caching for an individual fixture or test.

Default value: false

Related Configuration File Option: disablePageCaching

disableScreenshots

Type: Boolean

Description: Prevents TestCafe from taking screenshots. When this option is specified, screenshots are not taken whenever a test fails or when t.takeScreenshot or t.takeElementScreenshot is executed.

Default value: false

Related Configuration File Option: disableScreenshots

disableMultipleWindows

Type: Boolean

Description: Disables support for multi-window testing in Chrome and Firefox. Use this option if you encounter compatibility issues with your existing tests.

Related Configuration File Option: disableMultipleWindows

Example

const createTestCafe = require('testcafe');

const testcafe = await createTestCafe('localhost', 1337, 1338);

try {
    const runner = testcafe.createRunner();

    const failed = await runner.run({
        skipJsErrors: true,
        quarantineMode: true,
        selectorTimeout: 50000,
        assertionTimeout: 7000,
        pageLoadTimeout: 8000,
        speed: 0.1,
        stopOnFirstFail: true
    });

    console.log('Tests failed: ' + failed);
}
finally {
    await testcafe.close();
}

If a browser stops responding while it executes tests, TestCafe restarts the browser and reruns the current test in a new browser instance. If the same problem occurs with this test two more times, the test run finishes and an error is thrown.

Note

When you use a LiveModeRunner, call the runner.run method once. To run multiple live mode sessions simultaneously, create multiple TestCafe server instances.

Cancel Test Tasks

To stop an individual test task, cancel a promise returned by runner.run:

const taskPromise = runner
    .src('tests/fixture1.js')
    .browsers([remoteConnection, 'chrome'])
    .reporter('json')
    .run();

taskPromise.cancel();

You can cancel all pending tasks with the runner.stop method.