Speed-testing

[squidgetx]

We should do some serious performance profiling. While I don't have the evidence on hand, I would bet one of my kidneys that we have massive N+1 problems. (Actually, now that I think back on it, I'm pretty sure I'm responsible for at least some of the N+1 code...) So, at some juncture, we should:

• use Rack Mini Profiler and YSlow to identify the source of slowness;
• use Bullet to reduce the number of N+1 queries;
• perform other optimizations that are, at this point, definitely not premature anymore.

Speed has, so far, been the chief complaint of almost every MT I talked to, so there's some case to make this top priority -- but it should probably wait for the next milestone, i.e. after we finalize the full testing coverage.

[mnquintana]

In the spirit of automating stuff, do you think it might be worthwhile to write some speed tests (as in, not benchmarks, but adding to our test suite)? ie. a test should fail if the catalog takes longer than 5 secs to render, or something like that?

[squidgetx]

Let's just use this PR to get the profiling gems into our workflow. Testing speed sounds like a good idea but I think it would depend on a) ability to write integration tests and b) the power of the working machine

[dgoerger]

I believe the usual method for benchmarking speed is to always use the same machine for the tests. My setup is ridiculously fast. The production server at Science Park is by comparison absurdly slow. @squidgetx's machine is in-between. We would need to pick a consistent environment (hardware, software environment) for benchmarks to be remotely meaningful between version numbers, or between benchmarks of the same point-release even.

[mnquintana]

That makes a lot of sense. Maybe it would be valuable to create an environment mirroring (as best we can) the resources of the Science Park VM our production instances are running on?

[dgoerger]

I'll ask Mike about the VM specs when i call re installing Ruby 2.1.1 on the server for v3.3.x. We can probably emulate it with VirtualBox, if not import a copy of the VM directly, and store it in the STC Box folder or.. a Dev Box folder since we're leaving STC.

[orenyk]

That makes sense to me. We have the development instance, although we're already using it for user testing / feedback purposes. Can we get another instance up for benchmarking purposes?

@dgoerger don't we also need to be on the same hardware to appropriately test speed?