#1059 Intermittent Tests

[orenyk]

Resolves #1059, currently not doing the trick :-\

[orenyk]

Ok, the last commit passed on its first run. I think we can merge this in ASAP and see if it helps across other branches / builds.

[orenyk]

Spoke too soon, I guess :-\

[mnquintana]

It looks like it's passing now

[orenyk]

Yea, after I re-ran the build five or six times. Clearly not the fix :-\

[orenyk]

I'm going to try to deal with this so we can get accurate builds again.

[orenyk]

Ok, when I run just that file the test passes, but not when I run the whole test suite. The plot thickens...

[mnquintana]

@orenyk If it comes to it, Travis did offer last summer to send us a copy of the VM our tests are running on so we could test for system config issues that might be affecting the build.

[orenyk]

Alright, binding.pry confirms that subject.invoke does actually change Reservation.count as expected. I really can't figure out why this is failing.

[orenyk]

It's not only a Travis issue, it also happens locally. Running rspec spec/lib/tasks/delete_missed_reservations_rake_spec.rb passes but just running rspec fails on that test. Still debugging...

[orenyk]

Ok, this is weird. It appears as though the order of the tests matters; when it runs at the start of testing, it passes, so we must have some weird db thing going. I'm going to re-run and see what happens.

[mnquintana]

I was actually gonna ask about that – I was wondering if one of our tests was making some lasting db change that were affecting this test.

[orenyk]

Ok, this is even weirder. I've sandwiched the subject.invoke call between two binding.pry calls to verify that Reservation.count actually changed, and it passed even when it was later in the test suite.

RSpec shouldn's allow for that to happen, lasting DB changes should be impossible.

[orenyk]

On the other hand, sandwiching the call between two puts Reservation.count calls shows that, indeed, the reservation count isn't actually changing. I don't know what the difference is between that and binding.pry, binding.pry shouldn't make any difference...

[orenyk]

Huh, if I simply exit out of the binding.pry breakpoint without calling anything, the tests fail. WTF.

[orenyk]

Oooooooooook, so it's definitely order-dependent, b/c now I can't replicate my 'passing when I call Reservation.count in a breakpoint' solution. So weird.

[orenyk]

WOOOOOOOOO! Figured it out. For some reason we were creating multiple App Configs and therefore when the rake task checked AppConfig.first it wasn't necessarily getting the right one. Whew. Cleaning this up and then ready for review.

[orenyk]

DONE (build 2001, btw)!

[mnquintana]

@orenyk ZOMG

[mnquintana]

@orenyk @squidgetx That reminds me though – AppConfig should really be a singleton class... thoughts? (ie. there should never be more than one AppConfig instance)

[orenyk]

Yes please :-). That totally makes sense, we're always checking for AppConfig.first anyway, so if we can make that more rigorous that would be awesome. I don't know much about singleton classes but I'll look into it (and open an issue).

[squidgetx]

yesssssssssssssss

[squidgetx]

code looks solid

[orenyk]

great squashing / merging

[orenyk]

woot green build!