ruby on rails - What is a systematic approach to debug intermittently failing specs? -

i have 4 tests in capybara/rspec suite keep failing (a real problem ci deployment).

the worst thing, these tests failing intermittently, , when entire suite run, making difficult debug.

they ajax requests, either submitting remote form or clicking remote link, followed expect(page).to have_content 'my flash message'.

these tests fail intermittently within same test cycle. example, have several models behave similarly, iterating through them test.

e.g.,  ['country', 'state', 'city'].each |object|   let(:target) { create object.to_sym }   'runs frustrating test'    end end 

sometimes country fails, state, passes.

i have tried adding wait: 30 expect statement. have tried adding sleep 30 before expect statement. i'm still getting intermittent passes.

there quite bit of information out there describing finicky ajax tests, have not found how debug , fix these kinds of problems.

i'm grateful advise or pointers others, before pull hair out!!

update

thank these excellent responses. it's been useful see others have grappled similar issues, , i'm not alone.

so, there solution?

the suggestions use debugging tools such pry, byebug, poltergeist's debug feature (thanks @jay-ar polidario, @tomwalpole) have been useful confirm thought knew — namely, , suggested @bm5k) features work consistently in browser, , errors lie within tests.

i experimented with adjusting timeouts , retries (@jay-ar polidario, @bm5k), , while improvement these still not consistent fix. more importantly, approach felt patching holes rather proper fix, , not entirely comfortable.

ultimately went major rewrite of these tests. has entailed breaking multi-step features, , setting , testing each step individually. while purists may claim not testing user's perspective, there sufficient overlap between each test i'm comfortable result.

in going through process, did notice of these errors related "clicking on things, or filling forms", @borama suggested. though in case experience reversed — we had adopted .trigger('click') syntax because capybara + poltergeist reporting errors clicking on elements using click_link or find(object).click, , these tests problematic.

to avoid these problems i've removed js tests as possible. i.e., testing majority of feature without js enabled, , creating short, targeted js specs test specific js responses, features or user feedback.

so there not 1 single fix. major refactoring that, in honesty, needed happen , valuable exercise. tests have lost features by breaking individual tests, whole has made tests easier read , maintain.

there still couple of tests showing red, , need more work. overall great improvement.

thank great guidance, , reassuring me interactions in testing environment root cause.

intermittently failing tests pain troubleshoot, there things can make life easier. first remove looping or shared examples. explicitly stating each expectation should make more clear example combination failing (or make more obvious indeed random).

over course of several runs, track tests failing. in same context group?

are mixing , matching javascript tests , non-javascript tests? if are, may running database issues (i've seen problems caused switching database cleaner strategies mid context block).

make sure consider parent context blocks tests in.

and if none of narrows down search, use gem allows retry failing tests.

i used respec-retry in past, have found unreliable lately. i've switched rspec-repeat. leave these off in development (configured 1 try) , run multiple tries on ci (usually 3). way can feel tests wobbly locally, not let tests break build (unless fail consistently).

tl;dr

most of intermittently failing tests encounter have lot of moving pieces (rails, capybara, database cleaner, factory girl, phantomjs, rspec name few). if code tested , specs pass , feature consistently works in browser chances interaction in testing environment root cause of intermittent failures. if can't track down, retry failing specs couple of times.