The Volano Report
- Last updated on May 31, 1999. The most recent Volano Report is available here.
The tests presented in this report look at the performance and network scalability of 11 Java virtual machines on six operating systems using a common Intel hardware platform running VolanoMark™ 2.1. VolanoMark attempts to answer two questions about Java virtual machines used in network server environments -- especially those environments involving a large number of active socket connections:
When run at high connection counts, VolanoMark essentially measures the consequences of omitting non-blocking input/output operations from the Java platform, as summarized by this Java Developer Connection request for enhancement (free registration required):
For background information and download locations, please see the Java Benchmarks page on Volano's Web site.
Is it fast?
Run rules
The performance test was executed with the following commands on a local loopback connection, using the heap size options shown below where possible:
- Server
java -ms8m -mx64m COM.volano.Main
- Client
java -ms8m -mx64m COM.volano.Mark -count 100
The operating system was rebooted before each set of tests for a particular Java virtual machine, and the first test result was discarded. The server side was restarted before each run of the client benchmark. The final score is the average of the best two out of three subsequent results.
Results
Scores are the throughput of the server in messages per second. Bigger numbers are faster. See the Environments section for details on the hardware platform, operating systems, and Java virtual machine environments.
Java Platform | Score |
---|---|
Tower TowerJ 3.0.1 Linux IBM JDK 1.1.7 OS/2 IBM JDK 1.1.7 Windows Sun HotSpot 1.0 Windows Microsoft SDK 3.2 Windows Sun JDK 1.2.1 Windows Novell JDK 1.1.5 NetWare Sun JDK 1.2 Solaris Transvirtual Kaffe 1.0b4 Linux Blackdown JDK 1.1.7 Linux JDK 1.1.7 FreeBSD |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
Java Platform | Operating System | Results | Score |
---|---|---|---|
![]() |
Red Hat Linux 6.0 Intel | 1897, 1900, 1893 | 1899 |
![]() |
OS/2 Warp Server for e-business | 1849, 1777, 1833 | 1841 |
![]() |
Windows NT Workstation 4.0 | 1652, 1664, 1656 | 1660 |
![]() |
Windows NT Workstation 4.0 | 1398, 1444, 1425 | 1435 |
![]() |
Windows NT Workstation 4.0 | 1408, 1391, 1388 | 1400 |
![]() |
Windows NT Workstation 4.0 | 1344, 1344, 1348 | 1346 |
![]() |
NetWare 5 | 1216, 1216, 1220a | 1218 |
![]() |
Solaris 7 Desktop Intel Platform Edition | 881, 885, 827 | 883 |
![]() |
Red Hat Linux 6.0 Intel | 378, 380, 371 | 379 |
![]() |
Red Hat Linux 6.0 Intel | 264, 264, 262 | 264 |
![]() |
FreeBSD 3.1-RELEASE | 171, 172, 171 | 172 |
Notes:
- Received
"java.net.SocketException: Bad file number"
36 times on the third run, but the test completed successfully.
Does it scale?
Run rules
The network scalability test was executed with the following commands over an isolated 10-Mbps Ethernet connection with a 10-Mbps hub, using the heap and stack size options shown below where possible:
- Server
java -ms8m -mx128m -ss32k COM.volano.Main
- Client
-
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 15
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 30
...
java -ms8m -mx128m -ss32k COM.volano.Mark -host xxx.yyy.zzz -rooms 105
See the COM.volano.Mark command synopsis for a complete description of all options.
The client test driver was executed under the IBM JDK 1.1.7 Java virtual machine using Windows NT 4.0 on a 200-MHz Intel Pentium Pro with a 256-kilobyte L2 cache and 256 megabytes of RAM. The operating systems on both sides were rebooted before each set of tests for a particular Java virtual machine. The VolanoMark server was not restarted between client test runs except where noted.
Results
Scores are the throughput of the server in messages per second based on the total number of concurrent connections. Bigger numbers are faster. See the Environments section for details on the hardware platform, operating systems, and Java virtual machine environments.
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() ![]() |
|||
300 | 600 | 900 | 1200 | 1500 | 1800 | 2100 | |||
---|---|---|---|---|---|---|---|---|---|
|
Java Platform | 300 | 600 | 900 | 1200 | 1500 | 1800 | 2100 | Limit |
---|---|---|---|---|---|---|---|---|
![]() |
2661 | 2387 | 2206 | ----h | ---- | ---- | ---- | 1014 |
![]() |
2157 | 1901 | 473 | ----a | ---- | ---- | ---- | 1102 |
![]() |
2535 | 2181 | 1753 | 1499 | 1452 | 755c | 498c | ~ 3600 |
![]() |
2260 | 2081 | ----a | ---- | ---- | ---- | ---- | 899 |
![]() |
2367 | 1927 | 1529 | 1203 | 973 | 697 | 552 | ~ 3600 |
![]() |
2404 | 1788 | 1516 | ----a | ---- | ---- | ---- | 922 |
![]() |
2487 | 2014 | 1844 | 1293 | 1505 | ----d | ---- | ~ 1500 |
![]() |
1612 | 1513 | 1448 | 1385 | 1321 | 1248 | 1184 | ~ 4000 |
![]() |
897 | 732 | 600 | ----e | ---- | ---- | ---- | 1016 |
![]() |
324 | 172 | 149 | ----f | ---- | ---- | ---- | 1016 |
![]() |
296 | 189 | 122 | ----g | ---- | ---- | ---- | 1023 |
Notes:
- Failed with
"java.lang.OutOfMemoryError"
. - Failed with
"java.net.SocketException: accept: Invalid argument"
. - Failed on the first try at 1800 and 2100 connections with
"java.net.SocketException: Socket read failed: 10055"
and"java.net.SocketException: Connection shutdown"
. Restarted the server and it worked on the second try. - Took too long to complete at 1800 connections (thrashing).
- Failed with
"java.io.IOException: Too many open files"
. - Failed with
"java.net.SocketException: Too many open files"
. - Failed with
"java.net.SocketException: Too many open files in system"
. - Failed with
"java.net.SocketException: accept: Too many open files"
.
Solaris Intel vs. SPARC
The following tests compare VolanoMark 2.1.2, JDK 1.2, and Solaris 7 on two different hardware architectures:
- a 200 MHz Intel Pentium Pro processor with 256 MB of RAM, and
- a 200 MHz Sun UltraSPARC processor with 256 MB of RAM.
Java Platform | Score |
---|---|
Sun JDK 1.2 Solaris SPARC Sun JDK 1.2 Solaris Intel |
![]() ![]() |
Java Platform | Operating System | Results | Score |
---|---|---|---|
![]() |
Solaris 7 Desktop SPARC Platform Edition | 1473, 1613, 1693 | 1653 |
![]() |
Solaris 7 Desktop Intel Platform Edition | 881, 885, 827 | 883 |
![]() ![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
![]() ![]() |
|
300 | 600 | 900 | 1200 | 1500 | 1800 | 2100 | |
---|---|---|---|---|---|---|---|
|
Java Platform | 300 | 600 | 900 | 1200 | 1500 | 1800 | 2100 | Limit |
---|---|---|---|---|---|---|---|---|
![]() |
2631 | 2311 | 2130 | 1894 | 1780 | 1628 | 1488 | ~ 4000 |
![]() |
1612 | 1513 | 1448 | 1385 | 1321 | 1248 | 1184 | ~ 4000 |
Environments
All tests ran identical copies of VolanoMark 2.1.2 on identical hardware—a 200-MHz Intel Pentium Pro processor with a 256-kilobyte L2 cache and 256 megabytes of RAM.
- Tower TowerJ 3.0.1 Linux
-
- 10 May 1999—Tower Technology TowerJ Compiler 3.0.1.0
- Red Hat Linux Intel 6.0 (Linux 2.2.5, glibc 2.1.1)
- TowerJ Compiler
"version 3.0.1.0 x86-linux"
- Installed from
TowerJ_3_0_1_0_x86_linux.class
(13,116,025 bytes). - Uses user-level threads and static native pre-compilation.
- Used the experimental command line option
-Dtowerj.poll
to increase the network scalability on Linux. - Built executable with TowerJ project files
Main.tj
andMark.tj
. - Did not increase the per-process file descriptor hard limit beyond its default of 1024 since it requires rebuilding the Linux 2.2 kernel.
- IBM JDK 1.1.7 OS/2
-
- 06 May 1999—IBM OS/2 Warp Developer Kit, Java Edition, Version 1.1.7
- IBM OS/2 Warp Server for e-business (Internal revision
14.039F_UNI
) - java full version
"JDK 1.1.7A IBM build o117-19990506 (JIT enabled: javax)"
- Installed from the
runtime.exe
update (5,540,519 bytes) on top of the Java runtime shipped with OS/2 Warp Server for e-business. - Uses native threads and
javax
just-in-time compiler. - Modified the
CONFIG.SYS
file to increase the thread limit from 1024 to 4095(THREADS=4095)
and the initial swap file size from 2 MB to 32 MB(SWAPPATH=C:\OS2\SYSTEM 2048 32768)
.
- IBM JDK 1.1.7 Windows
-
- 28 January 1999—IBM Win32 Java Virtual Machine Version 1.1.7
- Microsoft Windows NT Workstation Version 4.0 (Build 1381: Service Pack 4)
- java full version
"JDK 1.1.7 IBM build n117p-19990128 (JIT enabled: ibmjitc)"
- Installed from
ibm117gm.exe
(10,132,439 bytes). - Uses native threads and
ibmjitc
just-in-time compiler.
- Sun HotSpot 1.0 Windows
-
- 30 April 1999—Sun HotSpot VM 1.0 FCS
- Microsoft Windows NT Workstation Version 4.0 (Build 1381: Service Pack 4)
- Sun JDK 1.2.1 base with
HotSpot VM (1.0fcs, mixed mode, build E)
. - Installed from
hotspot1_0-win.zip
(1,622,666 bytes). - Uses native threads and an adaptive compiler.
- The heap options must be preceded by the capital letter
"X"
, and the stack options are not available.
- Microsoft SDK 3.2 Windows
-
- 05 April 1999—Microsoft SDK 3.2 Windows Build 3177
- Microsoft Windows NT Workstation Version 4.0 (Build 1381: Service Pack 4)
- jview version
5.00.3177
- Installed from
MSJavx86.exe
(6,631,656 bytes) andClassR.exe
(1,593,064 bytes). - Uses native threads and just-in-time compiler.
- Heap and stack command line options are not available.
- Sun JDK 1.2.1 Windows
-
- 27 March 1999—Sun JDK 1.2.1 Windows 95/98/NT Production Release
- Microsoft Windows NT Workstation Version 4.0 (Build 1381: Service Pack 4)
- java full version
"JDK-1.2.1-A"
- Installed from
jdk1_2_1-win.exe
(20,514,654 bytes). - Uses native threads and the
symcjit
Symantec Just-in-Time compiler. - The heap options must be preceded by the capital letter
"X"
, and the stack options are not available.
- Novell JDK 1.1.5 NetWare
-
- 25 January 1999—Novell JVM for NetWare
- Novell NetWare Version 5.00
- java full version
"1.1.5"
(JVM dated 01/08/99) - Installed from
jvm.exe
(18,107,065 bytes). - Uses native threads and
symcjit
just-in-time compiler (Symantec Java! JustInTime Compiler Version 3.00.040(x) for JDK 1.1.x). - Set Maximum Packet Receive Buffers = 1000 (default is 500).
- Modified the
java.cfg
file to enable the just-in-time compiler(JAVA_COMPILER=symcjit)
.
- Sun JDK 1.2 Solaris
-
- 26 February 1999—Sun JDK 1.2 Solaris Production Release
- Sun Solaris 7 Desktop Intel Platform Edition with patches 106981-04 and 107079-02.
- java full version
"Solaris_JDK_1.2_01"
- Installed from
Solaris_JDK_1.2_01_i386.tar.Z
(26,165,449 bytes). - Uses native threads and
sunwjit
just-in-time compiler. - The heap and stack options must be preceded by the capital letter
"X"
. - Increased the per-process file descriptor hard limit to 4096 by
setting the
rlim_fd_max
variable in/etc/system
.
- Transvirtual Kaffe 1.0b4 Linux—beta Java VM
-
- 09 April 1999—Transvirtual Technologies Kaffe OpenVM 1.0 Beta 4
- Red Hat Linux Intel 6.0 (Linux 2.2.5, glibc 2.1.1)
- Kaffe Virtual Machine
"Engine: Just-in-time Version: 1.0b4 Java Version: 1.1"
- Installed from the
kaffe-1.0.b4-2
package shipped with Red Hat Linux 6.0. - Uses
jthreads
user-level threads andkaffe.jit
just-in-time compiler. - Did not increase the per-process file descriptor hard limit beyond its default of 1024 since it requires rebuilding the Linux 2.2 kernel.
- Blackdown JDK 1.1.7 Linux
-
- 04 November 1998—Blackdown JDK 1.1.7 Linux
- Red Hat Linux Intel 6.0 (Linux 2.2.5, glibc 2.1.1)
- java full version
"Linux_JDK_1.1.7B_v3_green_threads"
- Installed from
jdk_1.1.7-v3-glibc-x86.tar.bz2
(10,130,082 bytes). - Uses user-level threads and no just-in-time compiler.
- Did not increase the per-process file descriptor hard limit beyond its default of 1024 since it requires rebuilding the Linux 2.2 kernel.
- JDK 1.1.7 FreeBSD
-
- 21 December 1998—JDK 1.1.7 for FreeBSD
- FreeBSD 3.1-RELEASE
- java full version
"jdk1.1.7-FreeBSD:1998/12/21"
- Installed from
jdk1.1.7.V98-12-21.tar.gz
(12,920,606 bytes). - Uses user-level threads and no just-in-time compiler.
- Did not increase the per-process file descriptor hard limit beyond its default of 1024 since the performance at high connection levels did not justify running the tests.


Contact John Neffenger with questions or comments.