Native Automation FAQ

Q: What is native automation and why do I need it?

Native browser automation protocols, such as CDP, offer a superior browser automation experience.

Early versions of TestCafe used a reverse proxy to automate browsers. This design choice allowed the framework to execute complex tests across browsers and platforms when few other tools could do the same. Since then, native automation protocols matured and grew in popularity.

Third-party automation solutions, such as the TestCafe proxy, can only emulate browser events. Native protocols control the browser directly, because the protocol is an intrinsic part of the browser. In addition, native automation protocols are usually faster and more stable.

TestCafe v2.5.0 introduced an experimental mode that allowed users to automate Chromium-based browsers, such as Google Chrome and Microsoft Edge, with the native CDP protocol. In TestCafe v3.0.0 and higher, native automation is the default setting.

We believe that this change will positively impact user experience. Read the TestCafe Goes Native announcement for more information on the benefits of native automation. You can always disable native automation from the configuration file or the command line interface.

Q: Do I need to change my tests to run TestCafe 3.X?

Maybe.

The TestCafe proxy uses custom client-side automation scripts to emulate browser events. Native automation, as the name suggests, uses the CDP protocol to natively fire browser events. The browser behaves just like it does in a real-world scenario.

If you designed a test for an earlier version of TestCafe, you may have inadvertently adjusted its code to accommodate the framework’s flawed event emulation. As a result, you may notice slight differences in test behavior when you launch TestCafe 3.X. Tests that interact with custom elements, or perform calculations related to element state, may even fail.

For example, the TestCafe proxy ignored invalid SSL certificates. But in real world scenarios, browsers always check SSL certificates for errors. With native automation, TestCafe behaves the same way. If your SSL certificate is invalid, the browser displays a warning message, and does not let the test proceed. Launch the browser with the --ignore-certificate-errors flag to disable this behavior:

testcafe 'chrome --ignore-certificate-errors' test.js

If a test runs well on an earlier version of the framework, but fails on TestCafe 3.X, this behavior may not be a mistake. Check your code and make edits when appropriate.

Q: How do I disable native automation?

If native automation causes your tests to fail, or otherwise interrupts your TestCafe workflow, use one of the following options to disable the capability:

Configuration File

Set the disableNativeAutomation option to true in the configuration file.

{
   "disableNativeAutomation": "true"
}

CLI

Include the --disable-native-automation CLI flag when you launch TestCafe.

testcafe chrome tests --disable-native-automation

Test Runner API

Set the disableNativeAutomation option to true when you launch the TestCafe Test Runner.

const createTestCafe = require('testcafe');

const runner = createTestCafe();

runner.run({
    disableNativeAutomation: true;
});

Q: How do I run tests in Safari, Firefox, and other non-Chromium-based browsers?

When you launch TestCafe v3.0.0 and up, the framework engages native automation mode to automate Chromium-based browsers with the native CDP protocol. If your browser string contains incompatible browsers, TestCafe disables native automation:

testcafe chrome,friefox test.js # This test runs without native automation.

If you want to run tests in non-Chromium-based browsers and take advantage of native automation, create two separate test runs:

testcafe chrome,edge my-fixture.js # Chromium-based browsers
testcafe firefox,safari my-fixture.js #Other browsers

Q: What are the limitations of native automation?

The TestCafe automation proxy was independent of browsers’ internal logic. This independence allowed TestCafe to grow its capabilities beyond what browsers can do out of the box.

Native automation limits the range of available browser interactions. As a result, the TestCafe team needs to come up with elaborate workarounds to keep all framework capabilities intact.

Below is the list of capabilities that do not currently work with native automation. Please be patient as the team works to resolve these issues.

In addition, the TestCafe team is working to fix the following native automation bugs:

  • TestCafe hangs when it accesses text files (#7786).
  • TestCafe hangs when it accesses iframes in local HTML files.
  • TestCafe cannot interact with iframes that the document.write method injects into the page.
  • Contrary to established practice, TestCafe can generate request hook headers with uppercase letters.

Q: How do I check if TestCafe uses native automation?

The nativeAutomation property of the t.browser object indicates whether TestCafe uses native automation to control the browser. The property’s value is true when TestCafe uses native automation and false when TestCafe uses the Hammerhead proxy.

To make sure that you always use your preferred automation method, check the browser’s native automation status before you start the test:

import { Selector } from 'testcafe';

fixture`TestController.browser`
    .page`https://example.com`;

test('Native automation check', async t => {
    await t.expect(t.browser.nativeAutomation).ok();
    //the test continues only if you use native automation
});

Q: How do I debug the CDP connection between TestCafe and the browser?

Use the t.getcurrentcdpsession method to access the Chrome DevTools Protocol object for the current session.

The CDP object exposes properties and methods that pertain to the CDP connection between TestCafe and the active browser window. Note that every browser window maintains a separate CDP connection.

fixture `Get current CDP session`
    .page('https://devexpress.github.io/testcafe/example');

test(`Get current CDP session`, async t => {
    const mainWindowId = await t.testRun.activeWindowId;

    let clientCDP = await t.getCurrentCDPSession();

    await t.expect(clientCDP.webSocketUrl).contains(mainWindowId);
}