Cypress vs Selenium
This article shows you the differences between Selenium library and Cypress framework. Cypress is a modern end to end testing library – a competitor of Selenium. I recommend you to get familiar with the general description of Cypress and description of Selenium problems. I wrote about it earlier (http://www.diwebsity.com/2019/07/16/why-choose-cypress/).
Below is a table with a comparison between these two approaches (libraries) line by line for each category. I tried to be as objective as possible. This table is intended to be a help for your decision process.
Cypress vs Selenium – Comparision
| Selenium | Cypress | |
| Type | It is simply one purpose library with a strong community and additional packages. | Complete framework using many front end test libraries. |
| Languages | C# Java Java Script TypeScript Python Ruby Perl PHP Haskell Objective-C R Dart Elixir | JavaScript TypeScript |
| Selectors | Id ClassName CSS selectors XPath By text and text fragments | CSS selectors By text and text fragments By regex |
| Selecting a unique element | There is a problem with uniqueness. When there are many elements matching a particular selector, then the first one will be returned without any notice. | Trying to select a single element with not a unique selector will result in an error. This is good because it gives instant feedback about the test inaccuracy. |
| Browsers Multiple browser tests | Chrome Opera Firefox Safari Internet Explorer Support of mobile browsers testing and testing on the browsers grind in the cloud (e.g. Browserstack or Appium) | Chrome Chromium Electron |
| Debugging | Depends on language: C#, Java tests are executed in a synchronous way, so it is easy to debug code the same as regular programming code. However, JavaScript and TypeScript tests are executed asynchronously, which makes the debugging very problematic. | Very good with time travel option in UI. Tests are executed in special chrome browser version with test steps explorer and preview window. It allows to select a particular step in the test case and see the state of a page at the time of executing this step. |
| Waiting for async operations | With additional libraries. Selenium doesn’t wait for any actions. It doesn’t have knowledge about any async operations. However, there exist support libraries to handle all kind of waiting for popular frontend frameworks. | Automatically. Cypress is run under the same browser process, so it knows when Java Script code is working. In that way it can automatically wait for finishing all front-end processing before going on with test execution. |
| Speed | Fast All web page operations have a small delay due to the driver working as a proxy between test instructions and browser. | Very fast It operates on the same process in the browser. |
| Cross application testing | Yes You can freely navigate from one application to any other URL | No You can test only a single domain and any number of subdomain inside one domain. |
| Integration with CI | Possible You can use any testing library, test reports, and execution patterns. You can easily adjust it to your needs. | Possible Using command line and npm library – MochaThis is the only one option. The CI service has to support npm. Today almost all of them support it.Test recording on the CI server is a paid option for a high number of recordings. |
| Parallelization | Yes, easily | Yes, from version 3.1.0 |
| Test confidence level | High Selenium stimulates the same actions as a user. | Medium Let more actions than a user can perform. Especially it let to perform all actions (also disabled in UI) using { force: true } which make the tests more flexible but less reliable in the context of repeating user actions. |
| Troubleshooting | Custom-developed We have to manage it manually. We can take screenshots, detailed error information or call stacks. | Very easy Cypress offer UI to debug tests with the time travel option (the ability to choose particular test instruction and see, how the tested page looks before this line execution). It also supports test recording and taking screenshots. |
| Documentation | Medium Base documentation is medium, but there are plenty of learning resources on the internet | Good |
| Work with custom elements | Need custom code There exist multiple additional libraries to work with custom controls | Need custom code |
| Execute JS | Possible We can do anything on the page and get elements in that way. JS code is written as a string. | Possible JS code can be written as code using a window object as a |
| Integrations | Big number On many places | Medium number Centralized https://docs.cypress.io/plugins/ |
| Manipulate browser options | Cookie LocalStorage Screen Sizes Extensions Command-line options | Network options |
| Browsers extensions | Yes Can be automated | Yes Need manual installation |
| General purpose | End to end testing Web automation Web scraping For both: developers and QA team | End to end testing Mostly for developers |
iframe support | Yes | No But there is an open issue for this. |
| Test case architecture | Custom, but very flexible. You can do whatever you need. | Hard It is hard to create layered architecture to support test case readability (like in BDD test cases). That’s because selected elements are not persistent. We can’t select element and operate on it multiple times later in code. The code below will not work correctly: let poemElement = cy.get('.poem1');poemElement.get('.list-group > .list-group-item.selected') .should('contain', 'And there is another sunshine,'); poemElement.get('.header').should('contain', 'Other header,') |
| Other | The most important advantage of Selenium is a great community which results in a huge number of resources to learn and resolve any problems. Additionally, there exists a lot of libraries for Selenium. It is related with the fact that Selenium is only a simple purpose library, that cannot work without additional libraries (like for test execution). | Cypress have additional features: – the ability to mock API calls – the ability to control network traffic speed This is relatively young framework and we should keep an eye on its growth. It has a great potential, but also has some serious problems/blockers for some applications. |
| Learning resources | https://www.lynda.com/Selenium-tutorials/Learning-Selenium/656801-2.html https://www.udemy.com/selenium-training/ | https://docs.cypress.io https://github.com/cypress-io/cypress-example-recipes#xhr-web-formshttps://github.com/cypress-io/testing-workshop-cypress |
I hope this comparison will help you to your own assessment of the end to end test approach. Remember to fit the testing approach to your and your organization needs (skills in the team, team structure, type of applications you create, test cases inside applications), not only following the hype. Cypress is a very promising library, but still, it has some blockers which can be problematic for your need (like single domain constraint or lack of iframe testing).