Skip navigation, jump to main content

End-to-end testing

We have successfully written unit and integration tests using Karma, Jasmine and Angular’s own testing tools. These precise tests give confidence that a single application part – like a Component or Service - or a group of connected parts work as intended.

Karma and Jasmine tests take a technical perspective. They focus on the front-end JavaScript code alone and run it in a controlled and isolated test environment. What is really important though is whether the whole application works for the user.

The most effective and reliable way to ensure a working application is manual testing: A dedicated software tester walks through the application feature by feature, case by case according to a test plan.

Manual tests are slow, labor-intensive and cannot be repeated often. They are unspecific from a developer perspective: If the test fails, we cannot easily pin down which part of the application is responsible or which code change causes the regression.

We need automated tests that take the user’s perspective. This is what end-to-end (E2E) tests do.

Strengths of end-to-end tests

As discussed in distribution of testing efforts, all types of automated tests have pros and cons. Unit and integration tests are fast and reliable, but do not guarantee a working application. End-to-end test are slow and often fail incorrectly, but they assess the fitness of the application as a whole.

When all parts of the application come together, a new type of bugs appears. Often these bugs have to do with timing and order of events, like network latency and race conditions.

The unit and integration tests we wrote worked with a fake back-end. We send fake HTTP requests and respond with fake data. We made an effort to keep the originals and fakes on par.

It is much harder to keep the front-end code in sync with the actual API endpoints and responses from the back-end. Even if the front-end and the back-end share type information about the transferred data, there will be mismatches.

It is the goal of end-to-end tests to catch these bugs that cannot be caught by other automated tests.

Deployment for end-to-end tests

End-to-end tests require a testing environment that closely resembles the production environment. You need to deploy the full application, including the front-end and the relevant back-end parts. For that purpose, back-end frameworks typically support configurations for different environments, like development, testing and production.

The database needs to be filled with pre-fabricated fake data. With each run of the end-to-end tests, you need to reset the database to a defined initial state.

The back-end services need to answer requests with deterministic responses. Third-party dependencies need to be set up so they return realistic data but do not compromise production data.

Since this guide is not about DevOps, we will not go into details here and focus on writing end-to-end tests.

How end-to-end tests work

An end-to-end test tries to mimic how a user interacts with the application. Typically, the test engine launches an ordinary browser and controls it remotely.

Once the browser is started, the end-to-end test navigates to the application’s URL, reads the page content and makes keyboard and pointer input. For example, the test fills out a form and clicks on the submit button.

Just like unit and integration tests, the end-to-end test then makes expectations: Does the page include the right content? Did the URL change? This way, whole features and user interfaces are examined.

End-to-end testing frameworks

Frameworks for end-to-end tests allow navigating to URLs, simulating user input and inspecting the page content. Apart from that, they have little in common. The test syntax and the way the tests are run differ widely.

There are two categories of end-to-end testing frameworks: Those that use WebDriver and those that do not.

The WebDriver protocol defines a way to control a browser remotely with a set of commands. It originates from the Selenium browser automation project and is now developed at the World Wide Web Consortium (W3C).

All common browsers support the WebDriver protocol and can be controlled remotely. The most important WebDriver commands are:

WebDriver is a key technology most end-to-end testing frameworks depend on. The main benefit of WebDriver is that tests can be run in different browsers, even simultaneously.

WebDriver is a high-level, generic, HTTP-based protocol. It connects the test running on one machine with a browser possibly running on another machine. The level of control over the browser is limited.

Not all frameworks build on WebDriver. Some frameworks integrate more directly into the browser. This makes them more reliable, less complex, but also less flexible since they only support few browsers.

In this guide, we will learn about two frameworks, one of each category:

Introducing Protractor

Protractor is an end-to-end testing framework based on WebDriver, made for Angular applications. Like the Angular framework itself, Protractor originates from Google.

Up until Angular version 12, Protractor was the default end-to-end testing framework in projects created with Angular CLI. Since Angular 12, Protractor is deprecated. In new CLI projects, there is no default end-to-end testing solution configured.

The main reason for Protractor’s deprecation is that it was not maintained for years. During that time, new browser automation standards and better end-to-end testing frameworks emerged.

Protractor has two outstanding features designed for testing Angular applications. But as we will learn, you cannot benefit from these optimizations any longer.

First, Protractor hooks into the Angular app under tests. After sending a WebDriver command, Protractor waits for Angular to update the page. Then it continues with the next command. By synchronizing the test and the application, the test gets faster and more reliable.

Second, Protractor has a feature called control flow. While WebDriver commands are asynchronous by nature, the control flow allows you to write tests as if they ran synchronously.

Protractor’s control flow implementation led to inconsistencies and bugs. The underlying WebDriverJS library removed the feature, so Protractor had to deprecate it as well. This means you need to use async / await to explicitly wait for a command to finish.

As a result, Protractor lost a useful feature. Protractor’s contenders, namely Cypress and WebDriver.io, still allow to write asynchronous test code in a synchronous manner.

Without the control flow, you practically need to disable the “wait for Angular” feature as well. This means both key Protractor features have lapsed.

In view of these events, this guide recommends against using Protractor for new projects. Protractor is a solid project, but today there is no compelling reason to choose Protractor over its competitors.

If you are looking for Protractor examples, have a look at the Protractor end-to-end tests for the Counter and Flickr search. They are not explained in this guide though.

Introducing Cypress

Cypress is an end-to-end testing framework that is not based on WebDriver. There are no Angular-specific features. Any web site can be tested with Cypress.

WebDriver-based testing solutions are flexible and powerful but turned out to be slow and unreliable. Cypress aims to improve both the developing experience and the reliability of end-to-end tests.

Cypress employs a fundamentally different architecture. A Node.js application starts the browser. The browser is not controlled remotely, but the tests run directly in the browser, supported by a browser plugin.

The test runner provides a powerful user interface for inspecting and debugging tests right in the browser.

Cypress is the product of one company, Cypress.io, Inc. The test runner we are going to use is open source and free of charge.

The company generates revenue with an additional paid service: The Cypress dashboard manages test runs recorded in a continuous integration environment. You do not have to subscribe to this service to write and run Cypress tests.

From our perspective, Cypress has a few drawbacks.

Cypress is not simply better than WebDriver-based frameworks. It tries to solve their problems by narrowing the scope and by making trade-offs.

That being said, this guide recommends to use Cypress for testing Angular applications. Cypress is well-maintained and well-documented. With Cypress, you can write valuable end-to-end tests with little effort.

In case you do need a WebDriver-based framework, have a look at Webdriver.io instead.

Installing Cypress

An easy way to add Cypress to an existing Angular CLI project is the Cypress Angular Schematic.

In your Angular project directory, run this shell command:

ng add @cypress/schematic

This command does four important things:

  1. Add Cypress and auxiliary npm packages to package.json.
  2. Add the Cypress configuration file cypress.json.
  3. Change the angular.json configuration file to add ng run commands.
  4. Create a sub-directory named cypress with a scaffold for your tests.

The output looks like this:

ℹ Using package manager: npm
✔ Found compatible package version: @cypress/schematic@1.4.2.
✔ Package information loaded.

The package @cypress/schematic@1.4.2 will be installed and executed.
Would you like to proceed? Yes
✔ Package successfully installed.
? Would you like the default `ng e2e` command to use Cypress? [ Protractor to Cypress Migration Guide: https://on.cypress.io/protractor-to-cypress?cli=true ] Yes
CREATE cypress.json (298 bytes)
CREATE cypress/tsconfig.json (139 bytes)
CREATE cypress/integration/spec.ts (178 bytes)
CREATE cypress/plugins/index.ts (180 bytes)
CREATE cypress/support/commands.ts (1377 bytes)
CREATE cypress/support/index.ts (651 bytes)
UPDATE package.json (1225 bytes)
UPDATE angular.json (3952 bytes)
✔ Packages installed successfully.

The installer asks if you would like the ng e2e command to start Cypress. The choice only makes a difference if the project was created with Angular CLI prior to version 12, where ng e2e used to start Protractor.

If there are any Protractor tests in the project, you probably want to revive them later and use them as a reference. In this case, answer “No”. Otherwise, it is safe to answer “Yes”.

Writing an end-to-end test with Cypress

In the project directory, you will find a sub-directory called cypress. It contains:

The test files in integration are TypeScript files with the extension .ts.

The tests itself are structured with the test framework Mocha. The assertions (also called expectations) are written using Chai.

Mocha and Chai is a popular combination. They roughly do the same as Jasmine, but are much more flexible and rich in features.

If you have written unit tests with Jasmine before, the Mocha structure will be familiar to you. A test file contains one or more suites declared with describe('…', () => { /* … */}). Typically, one file contains one describe block, possible with nested describe blocks.

Inside describe, the blocks beforeEach, afterEach, beforeAll, afterAll and it can be used similar to Jasmine tests.

This brings us to the following end-to-end test structure:

describe('… Feature description …', () => {
  beforeEach(() => {
    // Navigate to the page
  });

  it('… User interaction description …', () => {
    // Interact with the page
    // Assert something about the page content
  });
});

Testing the counter Component

Step by step, we are going to write end-to-end tests for the counter example application.

As a start, let us write a minimal test that checks the document title. In the project directory, we create a file called cypress/integration/counter.ts. It looks like this:

describe('Counter', () => {
  beforeEach(() => {
    cy.visit('/');
  });

  it('has the correct title', () => {
    cy.title().should('equal', 'Angular Workshop: Counters');
  });
});

Cypress commands are methods of the cy namespace object. Here, we are using two commands, visit and title.

cy.visit orders the browser to visit the given URL. Above, we use the path /. Cypress appends the path to the baseUrl. Per default, the baseUrl is set to http://localhost:4200 in Cypress’ configuration file, cypress.json.

cy.title returns the page title. To be specific, it returns a Cypress Chainer. This is an asynchronous wrapper around an arbitrary value. Most of the time, a Chainer wraps DOM elements. In the case of cy.title, it wraps a string.

The Chainer has a should method for creating an assertion. Cypress relays the call to the Chai library to verify the assertion.

cy.title().should('equal', 'Angular Workshop: Counters');

We pass two parameters, 'equal' and the expected title string. equal creates an assertion that the subject value (the page title) equals to the given value ('Angular Workshop: Counters'). equal uses the familiar === comparison.

This should style of assertions differs from Jasmine expectations, which use the expect(…).toBe(…) style. In fact, Chai supports three different assertion styles: should, assert, but also expect. In Cypress you will typically use should on Chainers and expect on unwrapped values.

Running the Cypress tests

Save the minimal test from the last chapter as cypress/integration/counter.ts.

Cypress has two shell commands to run the end-to-end tests:

The Cypress schematic we have installed wraps these commands so they integrate with Angular.

$project-name$ is a placeholder. Insert the name of the respective Angular project. This is typically the same as the directory name. If not, it can be found in angular.json in the projects object.

For example, the Counter example has the project name angular-workshop. Therefore, the commands are:

The cypress open command will open the test runner:

Interactive Cypress test runner

In the main window pane, all tests are listed. To run a single test, just click on it. To run all, click the “Run all specs” button.

In a dropdown menu on the top-right, you can select the browser. Chrome, Firefox and Edge will appear in the list given you have installed them on your machine.

This graphical user interface is an Electron application, a framework based on Chromium, the open source foundation of the Chrome browser. You can always run your tests in Electron since it ships with Cypress.

Suppose you run the tests in Chrome, the in-browser test runner looks like this:

Cypress test runner in the browser

On the left side, the specs are listed. On the right side, the web page under test is seen.

By clicking on a spec name, you can see all commands and assertions in the spec.

Opened Cypress spec with commands

You can watch Cypress running the specs command by command. This is especially useful when a spec fails. Let us break the spec on purpose to see Cypress’ output.

cy.title().should('equal', 'Fluffy Golden Retrievers');

Failed spec in Cypress: Time out retrying: expected 'Angular Workshop: Counters' to equal 'Fluffy Golden Retrievers'. Error in counter.ts

Cypress provides a helpful error message, pointing to the assertion that failed. You can click on “Open in IDE” to jump to the spec in your code editor.

A unique feature of the in-browser test runner is the ability to see the state of the page at a certain point in time. Cypress creates DOM snapshot whenever a command is run or an assertion verified.

By hovering over a command or assertion, you can travel back in time. The page on the right side then reflects the page when the command or assertion was processed.

The time travel feature is invaluable when writing and debugging end-to-end tests. Use it to understand how your test interacts with the application and how the application reacts. When a test fails, use it to reconstruct what circumstances lead to the failure.

Asynchronous tests

Every Cypress command takes some time to execute. But from the spec point of view, the execution happens instantly.

In fact, Cypress commands are merely declarative. The execution happens asynchronously. By calling cy.visit and cy.title, we add commands to a queue. The queue is processed later.

As a consequence, we do not need to wait for the result of cy.visit. Cypress automatically waits for the page to load before proceeding with the next command.

For the same reason, cy.title does not immediately return a string, but a Chainer that allows more declarations.

In the Jasmine unit and integration tests we wrote, we had to manage time ourselves. When dealing with asynchronous commands and values, we had to use async / await, fakeAsync and other means explicitly.

This is not necessary when writing Cypress tests. The Cypress API is designed for expressiveness and readability. Cypress hides the fact that all commands take time.

Automatic retries and waiting

A key feature of Cypress is that it retries certain commands and assertions.

For example, Cypress queries the document title and compares it with the expected title. If the title does not match instantly, Cypress will retry the cy.title command and the should assertion for four seconds. When the timeout is reached, the spec fails.

Other commands are not retried, but have a built-in waiting logic. For example, we are going to use Cypress’ click method to click on an element.

Cypress automatically waits for four seconds for the element to be clickable. Cypress scrolls the element into view and checks if it is visible and not disabled. After several other checks, the Cypress performs the click.

The retry and waiting timeout can be configured for all tests or individual commands.

If a spec fails despite these retries and waiting, Cypress can be configured to retry the whole spec. This is the last resort if a particular spec produces inconsistent results.

These features makes end-to-end tests more reliable, but also easier to write. In other frameworks, you have to wait manually and there is no automatic retry of commands, assertions or specs.

Testing the counter increment

In our first Cypress test, we have checked the page title successfully. Let us test the counter’s increment feature.

The test needs to perform the following steps:

  1. Navigate to “/”.
  2. Find the element with the current count and read its text content.
  3. Expect that the text is “5”, since this is the start count for the first counter.
  4. Find the increment button and click it.
  5. Find the element with the current count and read its text content (again).
  6. Expect that the text now reads “6”.

We have used cy.visit('/') to navigate to an address. The path “/” translates to http://localhost:4200/ since this is the configured baseUrl.

Finding elements

The next step is to find an element in the current page. Cypress provides several ways to find elements. We are going to use the cy.get method to find an element by CSS selector.

cy.get('.example')

cy.get returns a Chainer, an asynchronous wrapper around the found elements, enriched with useful methods.

Just like with unit and integration test, the immediate question is: How should we find an element – by id, name, class or by other means?

As discussed in querying the DOM with test ids, this guide recommends to mark elements with test ids.

These are data attributes like data-testid="example". In the test, we use a corresponding attribute selector to find the elements, for example:

cy.get('[data-testid="example"]')

Test ids are recommended, but other ways to find elements are still useful in some cases. For example, you might want to check the presence and the content of an h1 element. This element has a special meaning and you should not find it with by arbitrary test id.

The benefit of a test id is that it can be used on any element. Using a test id means ignoring the element type (like h1) and other attributes. The test does not fail if those change.

But if there is a reason for this particular element type or attribute, your test should verify the usage.

Interacting with elements

To test the counter Component, we want to verify that the start count for the first counter is “5”. The current count lives in an element with the test id count. So the element finder is:

cy.get('[data-testid="count"]')

The cy.get command already has an assertion built-in: It expects to find at least one element matching the selector. Otherwise, the test fails.

Next, we check the element’s text content to verify the start count. Again, we use the should method to create an assertion.

cy.get('[data-testid="count"]').should('have.text', '5');

The have.text assertion compares the text content with the given string.

We did it! We have found an element and checked its content.

Now let us increment the count. We find and click on the increment button (test id increment-button). Cypress offers the cy.click method for this purpose.

cy.get('[data-testid="increment-button"]').click();

The Angular code under test handles the click event. Finally, we verify that the visible count has increased by one. We repeat the should('have.text', …) command, but expect a higher number.

The test suite now looks like this:

describe('Counter', () => {
  beforeEach(() => {
    cy.visit('/');
  });

  it.only('has the correct title', () => {
    cy.title().should('equal', 'Angular Workshop: Counters');
  });

  it('increments the count', () => {
    cy.get('[data-testid="count"]').should('have.text', '5');
    cy.get('[data-testid="increment-button"]').click();
    cy.get('[data-testid="count"]').should('have.text', '6');
  });
});

The next feature we need to test is the decrement button. The spec works similar to the increment spec. It clicks on the decrement button (test id decrement-button) and checks that the count has decreased.

it('decrements the count', () => {
  cy.get('[data-testid="decrement-button"]').click();
  cy.get('[data-testid="count"]').should('have.text', '4');
});

Last but not least, we test the reset feature. The user can enter a new count into a form field (test id reset-input) and click on the reset button (test id reset-button) to set the new count.

The Cypress Chainer has a generic method for sending keys to an element that is keyboard-interactable: type.

To enter text into the form field, we pass a string to the type method.

cy.get('[data-testid="reset-input"]').type('123');

Next, we click on the reset button and finally expect the change.

it('resets the count', () => {
  cy.get('[data-testid="reset-input"]').type('123');
  cy.get('[data-testid="reset-button"]').click();
  cy.get('[data-testid="count"]').should('have.text', '123');
});

This is full test suite:

describe('Counter', () => {
  beforeEach(() => {
    cy.visit('/');
  });

  it('has the correct title', () => {
    cy.title().should('equal', 'Angular Workshop: Counters');
  });

  it('increments the count', () => {
    cy.get('[data-testid="count"]').should('have.text', '5');
    cy.get('[data-testid="increment-button"]').click();
    cy.get('[data-testid="count"]').should('have.text', '6');
  });

  it('decrements the count', () => {
    cy.get('[data-testid="decrement-button"]').click();
    cy.get('[data-testid="count"]').should('have.text', '4');
  });

  it('resets the count', () => {
    cy.get('[data-testid="reset-input"]').type('123');
    cy.get('[data-testid="reset-button"]').click();
    cy.get('[data-testid="count"]').should('have.text', '123');
  });
});

On the start page of the counter project, there are in fact nine counters instance. The cy.get command therefore returns nine elements instead of one.

Commands like type and click can only operate on one element, so we need to reduce the element list to the first result. This is achieved by Cypress’ first command inserted in the chain.

it('increments the count', () => {
  cy.get('[data-testid="count"]').first().should('have.text', '5');
  cy.get('[data-testid="increment-button"]').first().click();
  cy.get('[data-testid="count"]').first().should('have.text', '6');
});

This also applies to the other specs. If the element under test only appears once, the first command is not necessary, of course.

All counter features are now tested. In the next chapters, we will refactor the code to improve its readability and maintainability.

Custom Cypress commands

The test we wrote is quite repetitive. The pattern cy.get('[data-testid="…"]') is repeated over and over.

The first improvement is to write a helper that hides this detail. We have already written two similar functions as unit testing helpers, findEl and findEls.

The easiest way to create a Cypress helper for finding elements is a function.

function findEl(testId: string): Cypress.Chainable<JQuery<HTMLElement>> {
  return cy.get(`[data-testid="${testId}"]`);
}

This would allow us to write findEl('count') instead of cy.get('[data-testid="count"]').

This works fine, but we opt for a another way. Cypress supports adding custom commands to the cy namespace. We are going to add the command byTestId so we can write cy.byTestId('count').

Custom commands are placed in cypress/support/commands.ts created by the Angular schematic. Using Cypress.Commands.add, we add our own command as a method of cy. The first parameter is the command name, the second is the implementation as a function.

The simplest version looks like this:

Cypress.Commands.add(
  'byTestId',
  (id: string) =>
    cy.get(`[data-testid="${id}"]`)
);

Now we can write cy.byTestId('count'). We can still fall back to cy.get if we want to find an element by other means.

cy.byTestId should have the same flexibility as the generic cy.get. So we add the second options parameter as well. We borrow the function signature from the official cy.get typings.

Cypress.Commands.add(
  'byTestId',
  // Borrow the signature from cy.get
  <E extends Node = HTMLElement>(
    id: string,
    options?: Partial<
      Cypress.Loggable & Cypress.Timeoutable & Cypress.Withinable & Cypress.Shadow
    >,
  ): Cypress.Chainable<JQuery<E>> =>
    cy.get(`[data-testid="${id}"]`, options),
);

For proper type checking, we need to tell the TypeScript compiler that we have extended the cy namespace. In commands.ts, we extend the Chainable interface with a method declaration for byTestId.

declare namespace Cypress {
  interface Chainable {
    /**
     * Get one or more DOM elements by test id.
     *
     * @param id The test id
     * @param options The same options as cy.get
     */
    byTestId<E extends Node = HTMLElement>(
      id: string,
      options?: Partial<
        Cypress.Loggable & Cypress.Timeoutable & Cypress.Withinable & Cypress.Shadow
      >,
    ): Cypress.Chainable<JQuery<E>>;
  }
}

You do not have to understand these type definitions in detail. They simply make sure that you can pass the same options to cy.byTestId that you can pass to cy.get.

Save commands.ts, then edit cypress/support/index.ts and activate the line that imports command.ts.

import './commands';

This is it! We now have a strictly typed command cy.byTestId. Using the command, we can declutter the test.

describe('Counter (with helpers)', () => {
  beforeEach(() => {
    cy.visit('/');
  });

  it('has the correct title', () => {
    cy.title().should('equal', 'Angular Workshop: Counters');
  });

  it('increments the count', () => {
    cy.byTestId('count').first().should('have.text', '5');
    cy.byTestId('increment-button').first().click();
    cy.byTestId('count').first().should('have.text', '6');
  });

  it('decrements the count', () => {
    cy.byTestId('decrement-button').first().click();
    cy.byTestId('count').first().should('have.text', '4');
  });

  it('resets the count', () => {
    cy.byTestId('reset-input').first().type('123');
    cy.byTestId('reset-button').first().click();
    cy.byTestId('count').first().should('have.text', '123');
  });
});

Keep in mind that all these first calls are only necessary since there are multiple counters on the example page under test. If there is only one element with the given test id on the page, you do not need them.

We have learned the basics of Cypress by testing the counter app. Let us delve into end-to-end testing with Cypress by testing a more complex app, the Flickr search.

Before writing any code, let us plan what the end-to-end test needs to do:

  1. Navigate to “/”.
  2. Find the search input field and enter a search term, e.g. “flower”.
  3. Find the submit button and click on it.
  4. Expect photo item links to flickr.com to appear.
  5. Click on a photo item.
  6. Expect the full photo details to appear.

The application under test queries a third-party API with production data. The test searches for “flower” and with each test run, Flickr returns potentially different results.

There are two ways to deal with this dependency during testing:

  1. Test against the real Flickr API
  2. Fake the Flickr API and return a fixed response

If we test against the real Flickr API, we cannot be specific in our expectations due to changing search results. We can superficially test the search results and the full photo. We merely know that the clicked photo has “flower” in its title or tags.

This has pros and cons. Testing against the real Flickr API makes the test realistic, but less reliable. If the Flickr API has a short hiccup, the test fails although there is no bug in our code.

Running the test against a fake API allows us to inspect the application deeply. Did the application render the photos the API returned? Are the photo details shown correctly?

Keep in mind that unit, integration and end-to-end tests complement each other. The Flickr search is also tested extensively using unit and integration tests.

Each type of test should do what it does best. The unit tests already put the different photo Components through their paces. The end-to-end test does not need to achieve that level of detail.

With Cypress, both type of tests are possible. For a start, we will test against the real Flickr API. Then, we will fake the API.

Testing the search form

We create a file called cypress/integration/flickr-search.ts. We start with a test suite.

describe('Flickr search', () => {
  const SEARCH_TERM = 'flower';

  beforeEach(() => {
    cy.visit('/');
  });

  it('searches for a term', () => {
    /* … */
  });
});

We instruct the browser to enter “flower” into the search field (test id search-term-input). Then we click on the submit button (test id submit-search).

it('searches for a term', () => {
  cy.byTestId('search-term-input')
    .first()
    .clear()
    .type(SEARCH_TERM);
  cy.byTestId('submit-search').first().click();
  /* … */
});

The type command does not overwrite the form value with a new value, but sends keyboard input, key by key.

Before entering “flower”, we need to clear the field since it already has a pre-filled value. Otherwise we would append “flower” to the existing value. We use Cypress’ clear method for that purpose.

Clicking on the submit button starts the search. When the Flickr API has responded, we expect the search results to be appear.

A search result consists of a link (a element, test id photo-item-link) and an image (img element, test id photo-item-image).

We expect 15 links to appear since this is amount of results requested from Flickr.

cy.byTestId('photo-item-link')
  .should('have.length', 15)

By writing should('have.length', 15), we assert that there are 15 elements.

Each link needs to have an href containing https://www.flickr.com/photos/. We cannot check for an exact URL since results are the dynamic. But we know that all Flickr photo URLs have the same structure.

There is no direct Chai assertion for checking that each link in the list has an href attribute containing https://www.flickr.com/photos/. We need to check each link in the list individually.

The Chainer has an each method to call a function for each element. This works similar to JavaScript’s forEach array method.

cy.byTestId('photo-item-link')
  .should('have.length', 15)
  .each((link) => {
    /* Check the link */
  });

Cypress has three surprises for us.

  1. link is a synchronous value. Inside the each callback, we are in synchronous JavaScript land. (We could do asynchronous operations here, but there is no need.)

  2. link has the type JQuery<HTMLElement>. This is an element wrapped with the popular jQuery library. Cypress chose jQuery because many JavaScript developers are already familiar with it. To read the href attribute, we use link.attr('href').

  3. We cannot use Cypress’ should method since it only exists on Cypress Chainers. But we are dealing with a jQuery object here. We have to use a standard Chai assertion – expect or assert style. We use expect together with to.contain.

This brings us to:

cy.byTestId('photo-item-link')
  .should('have.length', 15)
  .each((link) => {
    expect(link.attr('href')).to.contain(
      'https://www.flickr.com/photos/'
    );
  });

The test now looks like this:

describe('Flickr search', () => {
  const SEARCH_TERM = 'flower';

  beforeEach(() => {
    cy.visit('/');
  });

  it('searches for a term', () => {
    cy.byTestId('search-term-input')
      .first()
      .clear()
      .type(SEARCH_TERM);
    cy.byTestId('submit-search').first().click();

    cy.byTestId('photo-item-link')
      .should('have.length', 15)
      .each((link) => {
        expect(link.attr('href')).to.contain(
          'https://www.flickr.com/photos/'
        );
      });
    cy.byTestId('photo-item-image').should('have.length', 15);
  });
});

To start the tests, we run the shell command:

ng run flickr-search:cypress-open

This opens the test runner where we can click on the test flickr-search.ts to run it.

Alternatively, you can start the development server (ng serve) in one shell and the test runner (npx cypress open) in a second shell.

Testing the full photo

When the user clicks on a link in the result list, the click event is caught and the full photo details are shown next to the list. (If the user clicks with the control/command key pressed or right-clicks, they can follow the link to flickr.com.)

In the end-to-end test, we add a spec to verify this behavior.

it('shows the full photo', () => {
  /* … */
});

First, it searches for “flower”, just like the spec before.

cy.byTestId('search-term-input').first().clear().type(SEARCH_TERM);
cy.byTestId('submit-search').first().click();

Then we find all photo item links, but not to inspect them, but to click on the first on:

cy.byTestId('photo-item-link').first().click();

The click lets the photo details appear. As mentioned above, we cannot check for a specific title, a specific photo URL or specific tags. The clicked photo might be a different one with each test run.

Since we have searched for “flower”, the term is either in the photo title or tags. We check the text content of the wrapper element with the test id full-photo.

cy.byTestId('full-photo').should('contain', SEARCH_TERM);

The contain assertion checks whether the given string is somewhere in the element’s text content. (In contrast, the have.text assertion checks whether the content equals the given string. It does not allow additional content.)

Next, we check that a title and some tags are present and not empty.

cy.byTestId('full-photo-title').should('not.have.text', '');
cy.byTestId('full-photo-tags').should('not.have.text', '');

The image itself needs to be present. We cannot check the src attribute in detail.

cy.byTestId('full-photo-image').should('exist');

The spec now looks like this:

it('shows the full photo', () => {
  cy.byTestId('search-term-input').first().clear().type(SEARCH_TERM);
  cy.byTestId('submit-search').first().click();

  cy.byTestId('photo-item-link').first().click();
  cy.byTestId('full-photo').should('contain', SEARCH_TERM);
  cy.byTestId('full-photo-title').should('not.have.text', '');
  cy.byTestId('full-photo-tags').should('not.have.text', '');
  cy.byTestId('full-photo-image').should('exist');
});

The assertions contain, text and exist are defined by Chai-jQuery, an assertion library for checking jQuery element lists.

Congratulations, we have successfully tested the Flickr search! This example demonstrates several Cypress commands and assertions. We also caught a glimpse of Cypress internals.

Page objects

The Flickr search end-to-end test we have written is fully functional. We can improve the code further to increase clarity and maintainability.

We introduce a design pattern called page object. A design pattern is a proven code structure, a best practice to solve a common problem.

A page object represents the web page that is scrutinized by an end-to-end test. The page object provides a high-level interface for interacting with the page.

So far, we have written low-level end-to-end tests. They find individual elements by hard-coded test id, check their content and click on them. This is fine for small tests.

But if the page logic is complex and there are diverse cases to test, the test becomes an unmanageable pile of low-level instructions. It is hard to find the gist of these tests and they are hard to change.

A page object organizes numerous low-level instructions into a few high-level interactions. What are the high-level interactions in the Flickr search app?

  1. Search photos using a search term
  2. Read the photo list and interact with the items
  3. Read the photo details

Where possible, we group these interactions into methods of the page object.

A page object is merely an abstract pattern – the exact implementation is up to you. Typically, the page object is declared as a class that is instantiated when the test starts.

Let us call the class FlickrSearch and save it in a separate file, cypress/pages/flickr-search.page.ts. The directory pages is reserved for page objects, and the .page.ts suffix marks the page object.

export class FlickrSearch {
  public visit(): void {
    cy.visit('/');
  }
}

The class has a visit method that opens the page that the page object represents.

In the test, we import the class and create an instance in a beforeEach block.

import { FlickrSearch } from '../pages/flickr-search.page';

describe('Flickr search (with page object)', () => {
  const SEARCH_TERM = 'flower';

  let page: FlickrSearch;

  beforeEach(() => {
    page = new FlickrSearch();
    page.visit();
  });

  /* … */
});

The FlickrSearch instance is stored in a variable declared in the describe scope. This way, all specs can access the page object.

Let us implement the first high-level interaction on the page object: searching for photos. We move the relevant code from the test into a method of the page object.

public searchFor(term: string): void {
  cy.byTestId('search-term-input').first().clear().type(term);
  cy.byTestId('submit-search').first().click();
}

The searchFor method expects a search term and performs all necessary steps.

Other high-level interactions, like reading the photo list and the photo details, cannot be translated into page object methods. But we can move the test ids and element queries to the page object.

public photoItemLinks(): Cypress.Chainable<JQuery<HTMLElement>> {
  return cy.byTestId('photo-item-link');
}

public photoItemImages(): Cypress.Chainable<JQuery<HTMLElement>> {
  return cy.byTestId('photo-item-image');
}

public fullPhoto(): Cypress.Chainable<JQuery<HTMLElement>> {
  return cy.byTestId('full-photo');
}

public fullPhotoTitle(): Cypress.Chainable<JQuery<HTMLElement>> {
  return cy.byTestId('full-photo-title');
}

public fullPhotoTags(): Cypress.Chainable<JQuery<HTMLElement>> {
  return cy.byTestId('full-photo-tags');
}

public fullPhotoImage(): Cypress.Chainable<JQuery<HTMLElement>> {
  return cy.byTestId('full-photo-image');
}

These methods return element Chainers.

Next, we rewrite the end-to-end test to use the page object methods.

import { FlickrSearch } from '../pages/flickr-search.page';

describe('Flickr search (with page object)', () => {
  const SEARCH_TERM = 'flower';

  let page: FlickrSearch;

  beforeEach(() => {
    page = new FlickrSearch();
    page.visit();
  });

  it('searches for a term', () => {
    page.searchFor(SEARCH_TERM);
    page
      .photoItemLinks()
      .should('have.length', 15)
      .each((link) => {
        expect(link.attr('href')).to.contain(
          'https://www.flickr.com/photos/'
        );
      });
    page.photoItemImages().should('have.length', 15);
  });

  it('shows the full photo', () => {
    page.searchFor(SEARCH_TERM);
    page.photoItemLinks().first().click();
    page.fullPhoto().should('contain', SEARCH_TERM);
    page.fullPhotoTitle().should('not.have.text', '');
    page.fullPhotoTags().should('not.have.text', '');
    page.fullPhotoImage().should('exist');
  });
});

For the Flickr search above, a page object is probably too much of a good thing. Still the example demonstrates the key ideas of page objects:

When writing end-to-end tests, you get lost in technical details quickly: finding elements, clicking them, filling out form fields, checking fields values and text content. But end-to-end tests should not revolve around these low-level details. They should describe the user journey on a high level.

The goal of this refactoring is not brevity. Using page objects does not necessarily lead to less code. The purpose of page objects is to separate low-level details – like finding elements by test ids – from the high-level user journey through the application. This makes the specs easier to read and the easier to maintain.

You can use the page object pattern when you feel the need to tidy up complex, repetitive tests. Once you are familiar with the pattern, it also helps you to avoid writing such tests in the first place.

Faking the Flickr API

The end-to-end test we wrote for the Flickr search uses the real Flickr API. As discussed, this makes the test realistic.

The test provides confidence that the application works hand in hand with the third-party API. But it makes the test slower and only allows unspecific assertions.

With Cypress, we can uncouple the dependency. Cypress allows us to intercept HTTP requests and respond with fake data.

First of all, we need to set up the fake data. We have already created fake photo objects for the FlickrService unit test. For simplicity, we just import them:

import {
  photo1,
  photo1Link,
  photos,
  searchTerm,
} from '../../src/app/spec-helpers/photo.spec-helper';

Using the fake photos, we create a fake response object that mimics the relevant part of the Flickr response.

const flickrResponse = {
  photos: {
    photo: photos,
  },
};

Now we instruct Cypress to intercept the Flickr API request and answer it with fake data. This setup happens in the test’s beforeEach block. The corresponding Cypress command is cy.intercept.

beforeEach(() => {
  cy.intercept(
    {
      method: 'GET',
      url: 'https://www.flickr.com/services/rest/*',
      query: {
        tags: searchTerm,
        method: 'flickr.photos.search',
        format: 'json',
        nojsoncallback: '1',
        tag_mode: 'all',
        media: 'photos',
        per_page: '15',
        extras: 'tags,date_taken,owner_name,url_q,url_m',
        api_key: '*',
      },
    },
    {
      body: flickrResponse,
      headers: {
        'Access-Control-Allow-Origin': '*',
      },
    },
  ).as('flickrSearchRequest');

  cy.visit('/');
});

cy.intercept can be called in different ways. Here, we pass two objects:

  1. A route matcher describing the requests to intercept. It contains the HTTP GET method, the base URL and a whole bunch of query string parameters. In the URL and the api_key query parameter, the * character is a wildcard that matches any string.
  2. A route handler describing the response Cypress should send. As JSON response body, we pass the flickrResponse fake object.

    Since the request to Flickr is cross-origin, we need to set the Access-Control-Allow-Origin: * header. This allows our Angular application at the origin http://localhost:4200 to read the response from the origin https://www.flickr.com/.

Finally, we give the request an alias by calling .as('flickrSearchRequest'). This makes it possible to refer to the request later using the @flickrSearchRequest alias.

After this setup, Cypress intercepts the request to Flickr and handles it by itself. The original Flickr API is not reached.

The existing, rather generic specs still pass. Before we make them more specific, we need to verify that Cypress found a match and intercepted the HTTP request. Because if it did not, the test would still pass.

We can achieve this by explicitly waiting for the request after starting the search.

it('searches for a term', () => {
  cy.byTestId('search-term-input').first().clear().type(searchTerm);
  cy.byTestId('submit-search').first().click();

  cy.wait('@flickrSearchRequest');

  /* … */
});

cy.wait('@flickrSearchRequest') tells Cypress to wait for a request that matches the specified criteria. @flickrSearchRequest refers to the alias we have defined above.

If Cypress does not find a matching request until a timeout, the test fails. If Cypress caught the request, we know that the Angular application received the photos specified in the photos array.

By faking the Flickr API, we gain complete control over the response. We chose to return fixed data. The application under test processes the data deterministically. As discussed, this allows us to verify that the application correctly renders the photos the API returned.

Let us write specific assertions that compare the photos in the result list with those in the photos array.

it('searches for a term', () => {
  cy.byTestId('search-term-input').first().clear().type(searchTerm);
  cy.byTestId('submit-search').first().click();

  cy.wait('@flickrSearchRequest');

  cy.byTestId('photo-item-link')
    .should('have.length', 2)
    .each((link, index) => {
      expect(link.attr('href')).to.equal(
        `https://www.flickr.com/photos/${photos[index].owner}/${photos[index].id}`,
      );
    });
  cy.byTestId('photo-item-image')
    .should('have.length', 2)
    .each((image, index) => {
      expect(image.attr('src')).to.equal(photos[index].url_q);
    });
});

Here, we walk through the links and images to ensure that the URLs originate from the fake data. Previously, when testing against the real API, we tested the links only superficially. We could not test the image URLs at all.

Likewise, for the full photo spec, we make the assertions more specific.

it('shows the full photo', () => {
  cy.byTestId('search-term-input').first().clear().type(searchTerm);
  cy.byTestId('submit-search').first().click();

  cy.wait('@flickrSearchRequest');

  cy.byTestId('photo-item-link').first().click();
  cy.byTestId('full-photo').should('contain', searchTerm);
  cy.byTestId('full-photo-title').should('have.text', photo1.title);
  cy.byTestId('full-photo-tags').should('have.text', photo1.tags);
  cy.byTestId('full-photo-image').should('have.attr', 'src', photo1.url_m);
  cy.byTestId('full-photo-link').should('have.attr', 'href', photo1Link);
});

The specs now ensure that the application under test outputs the data from the Flickr API. have.text checks an element’s text content, whereas have.attr checks the src and href attributes.

We are done! Our end-to-end test intercepts an API request and responds with fake data in order to inspect the application deeply.

In the case of the Flickr search, we have intercepted an HTTP request to a third-party API. Cypress allows to fake any request, including those to your own HTTP APIs.

This is useful for returning deterministic responses crucial for the feature under test. But it is also useful for suppressing requests that are irrelevant for your test, like marginal images and web analytics.

End-to-end testing: Summary

End-to-end tests used to be expensive while the outcome was poor. It was hard to write tests that pass reliably when the application is working correctly. This time could not be invested in writing useful tests that uncover bugs and regressions.

For years, Protractor was the end-to-end testing framework most Angular developers relied on. With Cypress, a framework arose that sets new standards. Compared to Protractor, Cypress excels in developer experience and cost-effectiveness.

While this guide recommends to start with Cypress, WebDriver-based frameworks are still useful if you need to test a broad range of browsers. For all Cypress tests in this guide, you will find equivalent Protractor tests in the example repositories.

Even with Cypress, end-to-end tests are much more complex and error-prone than unit and integration tests with Jasmine and Karma. Then again, end-to-end tests are highly effective to test a feature under realistic circumstances.