FAQ

General Questions

I have heard that TestCafe does not use Selenium. How does it operate?

Unlike most testing solutions, TestCafe is not built on Selenium. This allows us to implement features you cannot find in Selenium-based tools (for example, testing on mobile devices, user roles, automatic waiting, etc.).

TestCafe uses a URL-rewriting proxy which allows it to work without the WebDriver. This proxy injects the driver script that emulates user actions into the tested page.

You can read about this in How it Works.

What is the difference between TestCafe Studio and open-source TestCafe?

TestCafe Studio is a testing IDE built on top of the open-source TestCafe engine. It simplifies the way you record tests and helps you delegate testing to a QA team that does not want to write JavaScript.

Read the following article to learn how TestCafe Studio could fit into your workflow: What’s Better than TestCafe? TestCafe Studio.

The table below compares TestCafe and TestCafe Studio:

  TestCafe TestCafe Studio
No need for WebDriver, browser plugins or other tools
Cross-platform and cross-browser out of the box
Write tests in the latest JavaScript or TypeScript
Clear and flexible API supports ES6 and PageModel pattern
Stable tests due to the Smart Assertion Query Mechanism
Tests run fast due to intelligent Automatic Waiting Mechanism and Concurrent Test Execution
Custom reporter plugins
Use third-party Node.js modules in test scripts
Integration with popular CI systems  ✓*
Free and open-source  
Visual Test Recorder  
Interactive Test Editor  
Automatic Selector Generation  
Run Configuration Manager  
IDE-like GUI  

* You can use open-source TestCafe to run TestCafe Studio tests in CI systems.

Has TestCafe v2015.1 been deprecated?

TestCafe v2015.1 is no longer available for purchase or subscription renewal. We recommend that TestCafe v2015.1 users switch to TestCafe Studio to access the latest features. See this blog post for details.

Which browsers does TestCafe support? What are the exact supported versions?

You can find a list of supported browsers in our documentation. TestCafe is tested against the two latest versions of each browser except for the browsers whose versions are specified explicitly in this list.

We do not use the most recent JavaScript features in TestCafe code, which means it should work with any browser with HTML5 support released in the last three years.

Can I use third-party modules in tests?

You can import third-party modules to the test code and inject scripts into tested webpages.

To import a module to a test file, use the import statement.

import fs from 'fs';

fixture `fixture`
   .page('http://localhost/testcafe/');

test('test', async t => {
   const filePath = 'filepath.js';

   await t.expect(fs.existsSync(filePath)).ok();
});

If a Node.js module or a JavaScript file can be executed in the browser, you can inject it into the tested page.

fixture `My fixture`
    .page `https://example.com`
    // The fixture.clientScripts method injects jquery.js
    // into all pages visited in this fixture.
    .clientScripts('scripts/jquery.js');

test('test', async t => {
    const clientFunction = ClientFunction(() => {
        // You can use $ here
        return $('div').text();
    });

    const text = await clientFunction();
});

For more information about how to inject scripts, see Inject Scripts Into Tested Pages.

How do I work with configuration files and environment variables?

TestCafe allows you to specify settings in a configuration file.

If you need to use custom properties in the configuration, create a separate configuration file and import it to the tests.

Note

Vote for the GitHub issue #3593 if you want us to support custom properties in the TestCafe configuration file.

For example, you can create the following config.json file to pass a website’s base URL to test code:

{
    "baseUrl": "http://localhost/testcafe"
}

In the test code, import it like a regular JavaScript module:

import config from './config';

fixture `Fixture`
    .page `${config.baseUrl}/test1/index.html`;

Alternatively, you can use environment variables.

How do I identify elements with dynamic IDs?

TestCafe selectors should use element identifiers that persist between test runs. However, many JavaScript frameworks generate dynamic IDs for page elements. To identify elements whose id attribute changes, use selectors based on the element’s class, content, tag name, or position.

See the Select Elements With Dynamic IDs example for details.

Can I use TestCafe to test React Native apps?

You can only use TestCafe to test something that runs in a browser, including Progressive Web Applications and Electron apps. TestCafe cannot automate native mobile applications because they do not run in a browser.

Troubleshooting

I have installed TestCafe but I cannot run it. What should I do?

Check your firewall. First, make sure that your firewall does not block the ports TestCafe uses. TestCafe chooses free ports automatically by default. Use the --ports command line option or the createTestCafe API factory function to specify custom ports. After that, check that the firewall does not block these specific ports.

Check your proxy server. Another reason for this problem can be the proxy server you use to access the Internet. If your network is connected to the Web via a proxy, use the --proxy command line option or the useProxy API method to specify the proxy address.

For Linux check X11. Also note that if you run TestCafe on Linux, you need to make sure the system is running the X11 server. Without X11, you can only run tests in cloud services and headless Google Chrome. However, if you use the Xvbf server, you can run any other browser in the headless mode.

When I run a TestCafe test, I get an unexpected error. What can cause that?

JavaScript errors. The most common reason for this is a JavaScript error on the tested page. Load this page in the browser, open the console and see if the page has any errors. In case there are errors, you can either fix them or use the --skip-js-errors flag to tell TestCafe to skip them.

Browser extensions. If this does not help, try running the problematic test in incognito mode. You can do this by adding an appropriate flag after the browser name.

testcafe "chrome -incognito" tests/test.js
testcafe "firefox -private-window" tests/test.js

If the test runs successfully, it might be browser extensions causing the issue. Try disabling them one by one and restart the test in the regular mode at each iteration. This way you can find out which extension prevents the test from running.

Third-party modules. In rare cases, third-party modules can be the cause. If you use a locally installed TestCafe, try installing it globally and running the test outside of the project to eliminate the influence of third-party modules.

I have installed TestCafe plugins but they do not work. What have I done wrong?

Plugins should also be installed locally if you are using a locally installed TestCafe.

npm install --save-dev {pluginName}

If you are going to use a global TestCafe installation, or you wish to use the plugin in other projects as well, install it globally.

npm install -g {pluginName}

My test fails because TestCafe could not find the required webpage element. Why does this happen?

This happens because either:

  • one of the selectors you used in test code does not match any DOM element, or
  • you used an incorrect CSS selector or a client-side function that returns no element to specify an action’s target element.

To determine the cause of this issue, do the following:

  1. Look at the error message in the test run report to learn which selector has failed.
  2. Add the t.debug() method before this selector to stop test execution before it reaches this point.
  3. Run the test and wait until the browser stops at the breakpoint.

Next, use the browser’s development tools to ensure that elements satisfy the interaction requirements.

Also, try running the test in full screen. Use the t.maximizeWindow and t.resizeWindow actions to control the browser window size. If the test passes, it means your webpage hides the target element when the window is resized to smaller dimensions.

Finally, try updating TestCafe to the latest version to see if the problem persists.

TestCafe reports that a request has failed. What are the possible reasons?

When TestCafe does not receive a successful response from a server, it outputs the following error:

Failed to load the page at https://www.example.com.
Increase the value of the "pageRequestTimeout" variable, 
enable the "retryTestPages" option, or use quarantine mode 
to perform additional attempts to execute this test.

If you believe that this is a false negative, enable quarantine mode. The quarantine mode option forces TestCafe to repeat tests that fail until they yield conclusive results.

However, we recommend that you determine the cause of this issue and address it.

This error can occur in the following situations:

The Web server is not responding

Confirm the successful resolution of the website’s domain name. Check if the web application is online and configured to receive incoming requests.

If the TestCafe proxy does not receive the webpage within two minutes, TestCafe throws an exception.

The following options enable TestCafe to retry failed network requests for webpages:

Unstable or improperly configured network connection

  • Check the network connection’s settings.
  • Check your network equipment and the connection to the Internet/Web server.
  • Check the proxy server’s settings or try a different proxy server.
  • Use VPN.
  • Connect to a different network.

Not enough resources in the container or CI system

If you run TestCafe in a container or CI system, use the following steps to diagnose resource shortage:

  • Increase the container’s resource limits.
  • Set the concurrency factor to 1.
  • Deploy the application’s Web server on a separate machine.
  • Run tests on a local device outside a container.

If this fixes the tests, it indicates that they require additional resources. You can address this in the following ways:

  • Adjust the container’s or environment’s settings to allocate more resources.
  • If you use a cloud-based CI system, ask the service provider for an upgrade or consider a different CI service with better hardware or smaller loads.

According to users’ feedback, the following CI systems work best with TestCafe:

‘The browser can’t open the page: can’t establish a secure connection to the server’

If your computer is connected to multiple networks (for instance, if it uses a VPN connection), TestCafe may incorrectly detect the host IP and not open the tested pages.

<browsername> cannot open the page because <browsername> is unable to establish a secure connection to the server.

To fix the issue, launch TestCafe with the --hostname localhost CLI option:

testcafe chrome test.js --hostname localhost

How To: Create a Minimal Working Example When You Submit an Issue

Modern software breaks in increasingly complex ways. In many cases, software issues are hard to reproduce — let alone investigate. TestCafe is no different.

That’s why it’s important to include a Minimal Working Example (MWE) with your TestCafe bug report. A good MWE ensures the issue is easy to reproduce and troubleshoot.

A Minimal Working Example should:

  • Be simple and easy to follow. Convoluted scenarios are hard to reproduce.
  • Not contain code that does not help reproduce the issue. Remove actions that do not affect the outcome.
  • Include a complete set of relevant data: the URL of the test page, the list of launch options, and the steps you follow to launch the tests.

Good Practices

  • The test page URL should point to a public web page.

    Note

    If the resources in your example require a password, or another form of authentication, you need to explicitly permit the DevExpress staff to remotely access these resources for the purposes of research, testing, and debugging. Send a written permission to support@devexpress.com.

    import { Selector } from 'testcafe';
    
    fixture `Fixture`
      .page('http://example.com');
    
    test('test', async t => {
      await t
        .click('body')
        .expect(Selector('h1').text).eql('Example Domain');
    });
    
  • The repository should contain all the information and files necessary to reproduce the issue. Any user should be able to successfully follow your instructions.

    I uploaded the demo to a [github repository](link). Follow the steps below to reproduce the issue:
    - Clone/download the repository.
    - Run the following commands:
        yarn install
        yarn webpack --mode development 
    - Start the web server: 
        yarn serve dist 
    - Run the test:
        yarn testcafe chrome tests/**/*.test.js
    

Practices to Avoid

Examples that contain incomplete information slow down debugging. If your bug report doesn’t include all the necessary details, the project’s maintainers spend more time asking questions, and less time dealing with the issue at hand.

Common mistakes people make when they compile bug reports:

  • The bug report does not include the test page URL. The example below references the url variable, but does not define it in code.

    fixture `Add to card`
      .page(url);
    
    test('test', async t => {
        await t.click('body');
    });
    
  • The example repository does not include the files referenced in the test code. The example below imports the getCookie helper and ExamplePage page model. If the repository lacks these files, the example cannot be run.

    import { getCookie } from '/helpers';
    import { ExamplePage } './page-model';
    
    fixture `Fixture`
        .page(http://example.com);
    
    test('test', async t => {
      const cookies = await getCookie();
    
        await t.click(ExamplePage.login);
    });
    
  • The bug report only contains English language instructions, and no test code.

    1. Click the first menu item
    2. Open a modal popup to specify product count
    3. Click the 'Buy' button
    
  • The example requires that you install additional software. Mock third-party services and applications to simplify the example, and make it easier to reproduce.

  • I cannot legally share the example source code / The application that I test displays sensitive information.

    Attempt to reproduce the issue without proprietary code. Mock or remove all private data.

  • My application is password-protected, and I cannot share the credentials on GitHub for security reasons.

    Share confidential information with TestCafe support staff at support@devexpress.com.

  • The issue is not page-dependent. Which page should I use for the test example?

    Create a test for the TestCafe example page.

  • I notified a member of the TestCafe team of an issue. Why haven’t I received a response?

    We do not recommend you contact members of the TestCafe team directly. If the recipient is absent or no longer part of the TestCafe team, the information you share may never come to our attention. Create a GitHub issue with your example instead.

  • My tests only fail when I run them on Docker.

    Containers and virtual machines have limited resources. TestCafe tests may fail if TestCafe, the browser, or the application does not have enough resources to run smoothly. Try to increase the amount of resources allocated to the Docker image or increase the timeout values for Selectors and assertions. If you are sure that issue is with Docker, send us the complete Docker image and its sources.

  • My tests run well on the local machine, but fail as unstable on CI.

    It is difficult to determine why your tests fail on CI because your tests’ performance depends on your CI settings (environment settings, memory configuration, and so on). To begin troubleshooting, increase timeout values and make sure your web server correctly processes requests. If you’re stuck, send the copy of the complete TestCafe debug log to support@devexpress.com.

    set DEBUG=testcafe:*,hammerhead:*
    testcafe chrome index.js 2> testcafe.log