SqueakDBX - Benchmarks


	links to this page:

Last updated at 12:30 am UTC on 12 April 2009

We are starting on benchmarks. You can execute it yourself and get some conclusions. Till now, we have some real information, this will be here. We have 2 types of benchmarks: tiny and big. And, for each of them, we have many tests.

Settings

Intel Core 2 duo, 2 GB RAM.
All databases running in localhost
In Linux SQL server case, the squeakDBX was in linux and SQL Server was running in a VirtualBox VM with Windows XP.

Tini benchmarks

How to run them

DBXBenchmark facility: DBXPostgreFacility facilityForBenchmark.

DBXTinyBenchmarks new runAll.

You can set the facility for the backend you want. This test, will run 10 times and then, obtain the AVG of those run. If you want to change the number of runs, you can execute: DBXBenchmark defaultRuns: 100. The important things about tiny benchmarks is that each test is run after other. There is no fork at all. No parallelism here.

To run native driver benchmarks, you can do:

PGTinyBenchmarks new runAll.
MysqlTinyBenchmarks new runAll.

Remember you must have the native driver install it. You can install them trough Universes.

Results in Linux

PostgreSQL

In this case, we not only do benchmarks, but also compare the results with squeak native PostgreSQL driver.
PostgreSQL 8.3, localhost

Benchmark	SqueakDBX Time (millis)	Native driver Time (millis)	Explanation
benchmarkInsert	773.8 AVG	1097.63 AVG	inserting 800 rows, all strings
benchmarkInsertConverted	1352.5 AVG	1793.03 AVG	inserting 800 rows, squeak types
benchmarkSelect10	5.0 AVG	5.67 AVG	selecting 10 rows, squeak types
benchmarkSelect100	44.3 AVG	42.32 AVG	selecting 100 rows, squeak types
benchmarkSelect1000	430.1 AVG	428.82 AVG	selecting 1000 rows, squeak types

MySQL

Benchmark	SqueakDBX Time (millis)	Native driver Time (millis)	Explanation
benchmarkInsert	196.5 AVG	278.9 AVG	inserting 800 rows, all strings
benchmarkInsertConverted	748.9 AVG	740.2 AVG	inserting 800 rows, squeak types
benchmarkSelect10	5.4 AVG	7.8 AVG	selecting 10 rows, squeak types
benchmarkSelect100	49.2 AVG	55.9 AVG	selecting 100 rows, squeak types
benchmarkSelect1000	485.5 AVG	537.4 AVG	selecting 1000 rows, squeak types

All backends together

Benchmark	PostgreSQL	MySQL	Oracle	SQL Server	Sqlite3
benchmarkInsert	773.8	196.5	2510.7	1240.8	3843.9
benchmarkInsertConverted	1352.5	748.9	3568.5	1655.4	4141.6
benchmarkSelect10	5.0	5.4	12.3	10.8	2.9
benchmarkSelect100	44.3	49.2	67.8	32.2	24.5
benchmarkSelect1000	430.1	485.5	630.7	301.8	204.6

Results in Windows

Here, all backends were running in localhost with a VirtualBox VM.

All backends together

Benchmark	PostgreSQL	MySQL	Oracle	SQL Server	Sqlite3
benchmarkInsert	1408.1			1649.5
benchmarkInsertConverted	1924.6			2276.8
benchmarkSelect10	3.1			2.1
benchmarkSelect100	15.1			14.4
benchmarkSelect1000	148.2			144.2

Big benchmarks

We added some benchmarking for big operations. The important things about big benchmarks is that each test (insert test and select test) is run simultaneously with the other. The test are fork. This is parallelism. Selects and inserts are beeing executed parallel.
The inserts is about 100.000 rows. The select is about 100.000 rows with squeak types conversion. You can test it running:

"Setting the platform"

DBXBenchmark facility: DBXPostgreFacility facilityForBenchmark.

"No fork"

DBXBigBenchmarks new runAll.

"Default priority fork"

DBXBigBenchmarks new runAllForked.

"Custom priority fork"

DBXBigBenchmarks new runAllForked: Processor userSchedulingPriority.

You can set the facility for the backend you want.

Results

Thread priority	PostgreSQL inserts	PostgreSQL selects	MySQL inserts	MySQL selects	Oracle inserts	Oracle selects	MSSQL inserts	MSSQL selects	Sqlite3 inserts	Sqlite3 selects
No thread	132941.0	38784.0	28612.0	39754.0	325784.0	57304.0	215713.0	42591.0	470728.0	15481.0
timingPriority	113272.0	37825.0	26078.0	38882.0	208195.0	58973.0	180674.0	45184.0	443758.0	16406.0
userSchedulingPriority	223221.0	267620.0	54630.0	132746.0	283811.0	347656.0	323739.0	389788.0
userBackgroundPriority	216585.0	257007.0	55812.0	138922.0	402037.0	465609.0	285176.0	326647.0

(1) This is the best time reached (with no optimizations), but of course, the image freezes until the job is done.
(2) The results for this level priority is almost the same as for no fork at all, and the image still freezes. This behaivor is expected.
(3) Here the performance begin to degrade, but the image retains some control, not very good for UI, but maybe is a good choice for seaside applications -we need more tests here-
(4) This is the worst performance result, but as expected, the best for avoiding image freezing (no block is perceived)