[benchmarks main page] [whole program benchmarks] [expression benchmarks] [Clojure version history] [hardware and software used]

Clojure whole program benchmarks

For each program, the tables below include:
All Clojure programs were compiled AOT (Ahead Of Time) before they were executed.  The compilation times are not included in the run time measurements.  What is included in the run time measurements are:
In other words, what you would get from the elapsed time measurement printed when you run the command:

    /usr/bin/time java -server -Xmixed -XX:+TieredCompilation -XX:+AggressiveOpts  extra-jvm-options  -classpath classpath  class_name  additional_args  > output_file

Each combination of (benchmark, Clojure version, platform) was run 3 times, and only the smallest elapsed time among them is shown in the graph.  By "platform" I mean the JVM, OS, and hardware running beneath Clojure.

If you are interested in what the JVM and Clojure startup times are by themselves, see the "hello" benchmark, which does nothing but print "Hello, world!".  This ranges from about 1 to 2 seconds depending upon the Clojure version and JVM used, versus about 200 millisec for the Java hello program.

See Hardware and software used for details on what was running beneath the JVM in these measurements.  The graphs themselves give information about the JVM and OS used.

Benchmark name
Description of what the program should do
This benchmark stresses ...
Source code and charts of results
Extra options to JVM besides "-server -Xmixed -XX:+TieredCompilation -XX:+AggressiveOpts -classpath <path>"
Additional arguments (after class_name, before "> output_file")
Input file size (bytes)
Output file size (bytes)
binarytrees
[link]
Memory allocation and deallocation.  Inherently sequential (true?) except if GC can run in parallel.
[Java]
[Clojure]
[Results]
 (none)
20 N/A
422
fannkuchredux
[link]
Generating and counting permutations.  Lots of parallelism possible.
[Java]
[Clojure]
[Results]
 -Xmx16m
12
N/A 29
fasta
[link]
Generating pseudo-random numbers via a particular method using integer arithmetic, and writing output.  Pseudo-random generation portion is inherently sequential.
[Java]
[Clojure]
[Results]
 -Xmx16m
25000000
N/A 254,166,745
hello
Print "Hello, world!"
Startup time
[Java]
[Clojure]
[Results]
 -Xmx16m

N/A 14
knucleotide
[link]
Updating hash tables, from small ones to fairly large ones
[Java]
[Clojure]
[Results]
 -Xmx1024m
< input_file
254,166,745 259
mandelbrot
[link]
Double precision arithmetic, embarassingly parallel
[Java]
[Clojure]
[Results]
 -Xmx16m
16000
N/A 32,000,015
nbody
[link]
Double precision arithmetic, inherently sequential
[Java]
[Clojure]
[Results]
 -Xmx8m
50000000
N/A 26
pidigits
[link]
Arbitrary precision integer arithmetic
[Java]
[Clojure]
[Results]
 -Xmx16m
10000
N/A 16,893
regexdna
[link] Regex searching and replacement, some parallelism possible
[Java]
[Clojure]
[Results]
 -Xmx616m
< input_file
50,833,411
298
revcomp
[link] Reading input, simple transformation of input strings, writing output
[Java]
[Clojure]
[Results]
 -Xmx384m
< input_file
254,166,745 254,166,745
spectralnorm
[link] Double precision arithmetic, lots of parallelism possible
[Java]
[Clojure]
[Results]
 -Xmx8m
5500
N/A 12

Why is -Xmx used?

The JVM command line option -Xmx<megabytes>m is frequently used to specify the maximum heap size for these programs, because it helps reduce the maximum resident memory used by the program.  This is another prominent factor measured and reported on the Computer Language Benchmarks game web site, in addition to the elapsed time and total CPU time used.  Many Clojure programs that can run fast with a small maximum heap size often use a much larger amount of resident memory if permitted, because they allocate memory that quickly becomes collectible garbage.  The JVM will not collect the garbage unless explicitly triggered by the program, or forced to by a smaller-than-default maximum heap size.

AOT compilation commands used

The following commands were used for AOT compilation of all Clojure programs, where CLJ_SOURCE_FILE is the original source file name, CLASSNAME is the same as the "Benchmark name" in the table above, and CLJ_JAR_FILE varied depending upon the Clojure version being measured:
mkdir ./obj
cp ${CLJ_SOURCE_FILE} ./obj/${CLASSNAME}.clj
java -Dclojure.compile.path=./obj/clj -classpath ${CLJ_JAR_FILE} clojure.lang.Compile ${CLASSNAME}

Number of Clojure function calls made

The table below tells how many calls were made to each Clojure function, usually with some indication of where most of the execution time is spent.

Benchmark / class name
Clojure source code
Number of times each Clojure function is called
binarytrees
binarytrees.clojure-rh.clojure
Most time is spent in:
bottom-up-tree: 607,475,038
item-check: 607,475,038

Other calls:
iterate-trees: 9
main: 1
-main: 1
fannkuchredux
fannkuchredux.clj-12.clj
Most time is spent in:
fannkuch-of-permutation: 301,297,917
next-permutation!: 296,930,829

Other calls:
init-permutations: 1
partial-fannkuch: 12
fannkuch: 1
fasta
fasta.clj-11.clj

(TBD: update for fasta.clj-8.clj source file)
Most time is spent in:
gen-random-fast: 200,000,000

Other calls:
find-index: 444,000
make-cumulative: 2
make-lookup-table: 2
cycle-bytes: 1
fasta-repeat: 1
fasta-random: 2
write-line: 3
hello
hello.clj
-main: 1
knucleotide
knucleotide.clojure-7.clojure
Most time is spent in:
tally-dna-subs-with-len: 16
fasta-description-line: 4,166,671
fasta-description-line-beginning: 2,083,337

Other calls with a small fraction of total run time:
medusa-future-thunk: 16
medusa-pmap: 1
fasta-dna-str-with-desc-beginning: 1
dna-str-to-key: 21
key-to-dna-str: 20
getcnt: 148
tally-total: 2
all-tally-to-str: 2
one-tally-to-str: 5
piece-sizes: 7
break-work-into-pieces: 7
do-one-piece: 16
add-tally-hashmaps!: 2
combine-pieces: 7
run: 1
mandelbrot
mandelbrot.clj-9-par-1.3.clj
Most time is spent in:
dot: 256,000,000

Other calls:
index-to-val: 32,000
ubyte: 32,000,000
compute-row: 16,000
compute-rows: 1
do-mandelbrot: 1
nbody
nbody.clj-14.clj
Most time is spent in:
v+!: 500,000,000
p-dt!: 250,000,000
v-dt! 500,000,000
advance: 50,000,000

Other calls:
v!: 1
energy (method of type Body): 10
mk-body: 5
init-state: 1
energy (function): 12
pidigits
pidigits.clojure-2.clj
floor-ev: 86,418
ncomp: 43,209
digit: 43,210
regexdna
regexdna.clj-3.clj
slurp-std-input: 1
count-regex-occurences: 9
revcomp
revcomp.clj-15.clj
Most time is spent in:
reverse-and-complement!: 3

Other calls:
rba-reverse: 4
find-first-byte-idx: 3
ubyte: 256
make-array-char-mapper: 1
spectralnorm
spectralnorm.clojure-6-corrected.clj
multiply-a-v: 160
multiply-at-v: 160
multiply-at-a-v: 20
run-game: 1