Cypress & Rails and the mysterious error
Disclaimer: This is a story of investigation and perseverance rather than a technical walkthrough.
A couple of months ago my team decided to start implementing E2E tests for our Ruby on Rails application, using Cypress which looked like as a quite promising and easy to use tool that should help us to getting the job done.
To provide some context, so far we have been testing using rspec + Factorybot (to create our test data) for our Rails components, so finding a way to re use all these factories was very important for us.
After some research we decided to go with the cypress-on-rails gem, which allowed us to re-use our existing factories on our tests scenarios and use Database Cleaner gem to wipe our Database before each test.
We began with small tests, everything was going well, so we integrated the E2E tests to our pipeline.
Then as our E2E test suite started to grow, we detected random errors on the test execution, not always on the same tests and not all the time, so yeah, very random.
One thing we noticed is that the failure was happening more often when the machine running the test suite was stressed, so we started to suspect of some kind of race condition.
Several lines of investigations were conducted (and later discarded) such as:
- Omniauth Integration Testing (which we love and still use)
- rspec mocks outside rspec (which we ended up discarding for other reasons…. bummer…)
- Checked each test like…. 1000 times…
No matter what we did, we always ended up with some error message similar to this one:
Nothing but that error over and over again, nothing on the web console or the rails logs to help us.
After some time banging our heads against the walls, we ended up adding this javascript code only for test environment:
window.onerror = function myErrorHandler(errorMsg, url, lineNumber) {
debugger;
return false;
}That small awesome snippet (documentation) allowed us to stop the execution right when the test failed, and thanks to the information provided we were able to pin point the root cause, which I bet you wouldn’t guess…. we were missing a css class in our sidebar, which made a class in our UI framework to raise an exception.
After fixing the sidebar, our tests started to work as a charm, yay!
Our team is really happy on how well Cypress integrates with our rails app, so I really recommend trying it if you haven’t done so yet.
I want to thank specially N N who really helped me on this journey of ups and downs that concluded with a happy ending (and a mystery solved).
If you want new stories on how we integrated Cypress with Rails, please let me know in the comments.
