# Layout Tests with Manual Fallback

Some Blink features cannot be automatically tested using the Web Platform. Prime
examples are the APIs that require
[user activation](https://html.spec.whatwg.org/multipage/interaction.html#triggered-by-user-activation)
(also known as _a user gesture_), such as [Full Screen](https://developer.mozilla.org/en-US/docs/Web/API/Fullscreen_API).
Automated tests for these Blink features must rely on special APIs, which are
only exposed in testing environments, and are therefore not available in a
normal browser session.

A popular pattern used in these tests is to rely on the user to perform some
manual steps in order to run the test case in a normal browser session. These
tests are effectively
[manual tests](http://web-platform-tests.org/writing-tests/manual.html), with
additional JavaScript code that automatically performs the desired manual steps,
when loaded in an environment that exposes the needed testing APIs.

## Motivation

Layout tests that degrade to manual tests in the absence of testing APIs have
the following benefits.

* The manual test component can be debugged in a normal browser session, using
  the rich [developer tools](https://developer.chrome.com/devtools). Tests
  without a manual fallback can only be debugged in the test runner.
* The manual tests can run in other browsers, making it easy to check whether
  our behavior matches other browsers.
* The layout tests can form the basis for manual tests that are contributed to
  [web-platform-tests](./web_platform_tests.md).

Therefore, the desirability of adding a manual fallback to a test heavily
depends on whether the feature under test is a Web Platform feature or a
Blink-only feature, and on the developer's working style. The benefits above
should be weighed against the added design effort needed to build a manual test,
and the size and complexity introduced by the manual fallback.

## Development Tips

A natural workflow for writing a layout test that gracefully degrades to a
manual test is to first develop the manual test in a browser, and then add code
that feature-checks for testing APIs, and uses them to automate the test's
manual steps.

Manual tests should minimize the chance of user error. This implies keeping the
manual steps to a minimum, and having simple and clear instructions that
describe all the configuration changes and user gestures that match the effect
of the Blink-specific APIs used by the test.

## Example

Below is an example of a fairly minimal test that uses a Blink-Specific API
(`window.eventSender`), and gracefully degrades to a manual test.

```html
<!doctype html>
<meta charset="utf-8">
<title>DOM: Event.isTrusted for UI events</title>
<link rel="help" href="https://dom.spec.whatwg.org/#dom-event-istrusted">
<link rel="help" href="https://dom.spec.whatwg.org/#constructing-events">
<meta name="assert"
    content="Event.isTrusted is true for events generated by user interaction">
<script src="../../resources/testharness.js"></script>
<script src="../../resources/testharnessreport.js"></script>

<p>Please click on the button below.</p>
<button>Click Me!</button>

<script>
'use strict';

setup({ explicit_timeout: true });

promise_test(() => {
  const button = document.querySelector('button');
  return new Promise((resolve, reject) => {
    const button = document.querySelector('button');
    button.addEventListener('click', (event) => {
      resolve(event);
    });

    if (window.eventSender) {
      eventSender.mouseMoveTo(button.offsetLeft, button.offsetTop);
      eventSender.mouseDown();
      eventSender.mouseUp();
    }
  }).then((clickEvent) => {
    assert_true(clickEvent.isTrusted);
  });

}, 'Click generated by user interaction');

</script>
```

The test exhibits the following desirable features:

* It has a second specification URL (`<link rel="help">`), because the paragraph
  that documents the tested feature (referenced by the primary URL) is not very
  informative on its own.
* It links to the
  [WHATWG Living Standard](https://wiki.whatwg.org/wiki/FAQ#What_does_.22Living_Standard.22_mean.3F),
  rather than to a frozen version of the specification.
* It contains clear instructions for manually triggering the test conditions.
  The test starts with a paragraph (`<p>`) that tells the tester exactly what to
  do, and the `<button>` that needs to be clicked is clearly labeled.
* It disables the timeout mechanism built into `testharness.js` by calling
  `setup({ explicit_timeout: true });`
* It checks for the presence of the Blink-specific testing APIs
  (`window.eventSender`) before invoking them. The test does not automatically
  fail when the APIs are not present.
* It uses [Promises](https://developer.mozilla.org/docs/Web/JavaScript/Reference/Global_Objects/Promise)
  to separate the test setup from the assertions. This is particularly helpful
  for manual tests that depend on a sequence of events to occur, as Promises
  offer a composable way to express waiting for asynchronous events that avoids
  [callback hell](http://stackabuse.com/avoiding-callback-hell-in-node-js/).

Notice that the test is pretty heavy compared to a minimal JavaScript test that
does not rely on testing APIs. Only use testing APIs when the desired testing
conditions cannot be set up using Web Platform APIs.
