End-to-end tests: how to get off on the right foot
End-to-end tests should be used sparingly, as they are costly to write (Page Model) and costly to run. Here’s how to proceed.
Definition
End-to-end (e2e) tests replicate the behavior of a user interacting with the application being tested. The term can be understood in two complementary ways, both of which are useful:
-
In breadth: since we are simulating a user’s behavior, we want to stay as close as possible to their actual experience (their User Journey). This means replicating a real and complete use case, starting with the action the user wants to perform (Job To Be Done), as opposed to a unit test, which focuses on a localized aspect of the application’s behavior. Example of an e-commerce site: from anonymously browsing the catalog to a successful payment.
-
In depth: we want the test to “traverse” as many application layers as possible, from the UI (necessarily) to the database (ideally). This reinforces our confidence that these layers communicate effectively with one another. This is therefore useful, but it’s also virtually essential whenever we perform true end-to-end, breadth-of-coverage testing, because it becomes very difficult to mock these layers.
So, to put it simply, we want to use as few mocks as possible.
Using a custom attribute
An end-to-end (e2e) testing framework operates on the DOM or reads from the DOM using selectors. These selectors can be based on the HTML structure or on class names, but a common practice is to assist the process by defining a custom attribute, such as data-test-id. That’s because this approach is the one that ties your tests least closely to the application’s implementation.
As a reminder, we want your tests to fail only because the application’s behavior has changed. We do not want tests to fail because the application’s implementation has changed, provided the behavior remains the same.
Creating a Page Model
When writing E2E tests, we want to abstract away as much of the implementation as possible, so that the tests are readable by humans (cy.get(selector('guess')).eq(count - 1).find(selector('letter')).eq(index).contains(letter);, yay!). We should, in fact, keep this in mind for all types of tests and for our code in general, but in the specific case of end-to-end tests, this translates to creating a Page Model, which allows us to write tests like the one in this snippet. You have to admit it’s definitely more readable, and actually quite readable in its own right.
In practical terms, a Page Model is nothing more than a set of functions or methods designed to hide the test implementation and abstract away the testing framework. Structuring it takes some effort, especially if you want to be able to chain actions and assertions (Player2.Guess(N._2nd).Letter(N._4th).click({ times: 2 }).expectStatusToBe('VALIDATED');). It’s work, sure, but nothing too crazy: we’re not sending people to the moon.
Tests that are costly to write and execute
Whether or not a Page Model is used, these tests are particularly time-consuming to write (typically several hours for the first test if you want a real business scenario), but also particularly long to run (on the order of a second, vs. 10 to 100 ms for unit tests). This is because they involve making actual network calls and, ideally, using a real database.
You should therefore write only a few E2E tests, focusing on the application’s happy paths and critical paths. You can think of E2E tests as smoke tests, in line with the proverb “where there’s smoke, there’s fire”: if there’s smoke, it means there’s a fire somewhere in the application (a necessary condition); but the absence of smoke does not imply that there is no fire (not a sufficient condition).
Consequently, your testing strategy should not rely solely on end-to-end (e2e) tests, as is unfortunately often the case. Make smart use of the entire testing pyramid, starting with unit tests. Really, as with everything, be mindful of the law of the instrument.
Granted, vendors of testing frameworks, eager to promote adoption of their product, have an unfortunate tendency to tout that their tool does “everything under the sun.” Cypress is no exception, claiming to be the best for component testing. My take? These tests would benefit from mocking the DOM using testing libraries such as Testing Library.