Interesting tests using Magma
Last updated at 12:41 am UTC on 17 January 2006
The Commit Benchmark
I wanted to have some sort of reproducible benchmark that stresses as much of the code as possible. Commiting a change to a single object over and over fit the bill. It's good for commits to be small and frequent, though they don't need to be this small, and usually won't.
A - AMD XP2000+, 512MB RAM, Win98SE
B - 700MHz IBM A20m laptop, 512MB RAM, Windows 2000
C - 700MHz IBM A20m laptop, 512MB RAM, Windows 2000
D - 350MHz Pentium II, 384MB RAM, Win98SE
Test 1 (Server A, client B, 802.11b Wi-Fi) ~ 16 commits / sec
Test 2 (Server A, client B, 100Mbps ethernet) ~ 18 commits / sec
Test 3 (Server A, client D, 100Mbps ethernet) ~ 11 commits / sec
Test 4 (Server B, client A, 100Mbps ethernet) ~ 22 commits / sec
Test 5 (Server B, client C, 802.11b Wi-Fi) ~ 17 commits / sec
Test 6 (Local connection on B) ~ 100 commits / sec (wow!)
It's interesting to note that the results were more dependent on the speed of the client than the server. Knowing the implementation, this doesn't surprise me. The server only deals with Integer oid relationships and so has less serialization work.
Keep in mind that these are somewhat of a "peak" commit rates. Your commits will be larger than one object and so there'll be a little more work to do, but it should be pretty close.
The Stress Test
I also decided to see how well the server could hold up to two clients performing the commit test at the same time. This is a great way to force the "Ma client server" framework to actually have to queue some requests. In these tests, I left one computer in a [ true ] whileTrue: [ commit something ] loop to constantly change the value in an Array back and forth. I call this computer the stressor. I then moved to the other computer and repeated the bench tests, above:
Test 1 (Server A, client B, stressor D) ~ client B reports 12 commits / sec
