Cosmo 0.7 Peformance

Setup

Al tests are were performed on my local laptop, which is dual core 1.83Ghz Intel processor with 2GB of RAM running Windows XP Pro. The client,server, and database are all running on the same machine. This isn't a typical setup for a server, but it allowed me to run in a consistent environment and switch variables (jvm and database) easily. The client is the cosmo stress testing client written in Ruby. Each test run consisted of 15 client threads (10 morse code, 3 atom, and 2 CalDAV) run for 100 seconds and then repeated 10 times for a total run time of 1000 seconds. Each user thread continuously runs operations based on the following probabilities:

Morse Code

Operation	Probability
publish	0.5%
sync	76.5%
update	15%
subscribe	7.75%
delete	0.25%

Atom

Operation	Probability
time-range feed	60%
full feed	5%
create event	15%
update event	5%
dashboard query	15%

CalDAV

Operation	Probability
create calendar	0.5%
put event	25%
delete event	0.5%
time-range report	69%
get event	5%

Variables

The test runs were run against different jvms and databases.

JVM	Database
Sun JDK 1.5	MySQL 5.0.41
Sun JDK 1.6	MySQL 5.0.41
BEA jrockit-R27.2.0-jdk1.6.0	MySQL 5.0.41
Sun JDK 1.6	Postgres 8.2

The one exception is that the same test was run against an instance on lab.osaf.us, but the client was run on a different machine (people.osafoundation.org) which is in the same subnet.

Results

Here are the results for total throughput and selected operations. I also included the results for the tests run against lab.osaf.us but they can't really be compared to the other results because the client was run on a different machine.

Notes

• All test runs completed with no errors.
• request times include client time to transmit/receive request/response
• sync (no changes) means a sync against a collection using the most updated sync token (results in no changes)
• jdk 1.6 seems to offer better performance on win32
• Postgres seems to have better write performance, worse read performance than MySQL (at least on win32). Because the majority of operations are read operations, total throughput is worse. I'd like to run this test again on a Unix server with 4+ cpus.
• cpu time was dominated by the app server (java process)

lab.osaf.us

Using a separate client machine I was able to run a 17,7,4 thread mix (17 morse code threads/7 atom threads/4 caldav threads) and get around 50 ops/sec on lab.osaf.us.

-- RandyLetness - 09 Jul 2007