Hi Mike,

Hi, we are using rspec + Selenium to test our applicaiton.

Congratulations, you are very close to the best setup. Add DeepTest (http://deep-test.rubyforge.org) to the equation and you've got the ultimate Selenium killing machine

Locally, when we test the applicaiton with Selenium Grid, we get nondeterministic results. Tests seem to fail randomly. Messing with the number of nodes in the grid seems to help, but its really annoying that we can't seem to get consistent results.

Most likely some tests are timing out in a non-deterministic manner because your CPU or Network is over-utilized. Monitor your CPU and Network activity on all the machines involved. Once you find the bottleneck launch fewer processes. For instance if your load average is way higher than the number of CPUs on the machine running the remote controls, cut the number of remote controls you launch by two until you get to a sustainable machine load.

I'm hoping that this is a resource issue.

It is very likely indeed!

To that end, we are in the process of integrating Selenium Grid into our continuous integration server build that has a lot more resources than a typical developers machine.

The best advice I can give you is to properly monitor resource utilization when you dimension you selenium grid setup.

Does anyone else suffer from nondeterministic results locally?

My team has been using Selenium Grid in a reliable manner for about a year. However, we did spend some time figuring out the optimal number of concurrent test runners and remote controls to run in parallel on each machine.

Good luck, and thank you for giving Selenium Grid a try!

- Philippe Hanrigou

  http://ph7spot.com