[benchmarks main page] [whole program benchmarks] [expression benchmarks] [Clojure version history] [hardware and software used]

Clojure whole program benchmarks

For each program, the tables below include:

a description of what the program should do.
a link to the source code of the Clojure and Java programs.
the command line arguments used.
any versions of Clojure that were not measured, and why.
some statistics about the amount of input read, output written, and in the last table the number of function calls performed during an execution of the program.

All Clojure programs were compiled AOT (Ahead Of Time) before they were executed. The compilation times are not included in the run time measurements. What is included in the run time measurements are:

the JVM startup time
Clojure startup time
"JIT warmup time"
time to read the input file, if any
time to write the output file

In other words, what you would get from the elapsed time measurement printed when you run the command:

/usr/bin/time java -server -Xmixed -XX:+TieredCompilation -XX:+AggressiveOpts extra-jvm-options -classpath classpath class_name additional_args > output_file

Each combination of (benchmark, Clojure version, platform) was run 3 times, and only the smallest elapsed time among them is shown in the graph. By "platform" I mean the JVM, OS, and hardware running beneath Clojure.

If you are interested in what the JVM and Clojure startup times are by themselves, see the "hello" benchmark, which does nothing but print "Hello, world!". This ranges from about 1 to 2 seconds depending upon the Clojure version and JVM used, versus about 200 millisec for the Java hello program.

See Hardware and software used for details on what was running beneath the JVM in these measurements. The graphs themselves give information about the JVM and OS used.

Benchmark name	Description of what the program should do	This benchmark stresses ...	Source code and charts of results	Extra options to JVM besides "-server -Xmixed -XX:+TieredCompilation -XX:+AggressiveOpts -classpath <path>"	Additional arguments (after class_name, before "> output_file")	Input file size (bytes)	Output file size (bytes)
binarytrees	[link]	Memory allocation and deallocation. Inherently sequential (true?) except if GC can run in parallel.	[Java] [Clojure] [Results]	(none)	20	N/A	422
fannkuchredux	[link]	Generating and counting permutations. Lots of parallelism possible.	[Java] [Clojure] [Results]	-Xmx16m	12	N/A	29
fasta	[link]	Generating pseudo-random numbers via a particular method using integer arithmetic, and writing output. Pseudo-random generation portion is inherently sequential.	[Java] [Clojure] [Results]	-Xmx16m	25000000	N/A	254,166,745
hello	Print "Hello, world!"	Startup time	[Java] [Clojure] [Results]	-Xmx16m		N/A	14
knucleotide	[link]	Updating hash tables, from small ones to fairly large ones	[Java] [Clojure] [Results]	-Xmx1024m	< input_file	254,166,745	259
mandelbrot	[link]	Double precision arithmetic, embarassingly parallel	[Java] [Clojure] [Results]	-Xmx16m	16000	N/A	32,000,015
nbody	[link]	Double precision arithmetic, inherently sequential	[Java] [Clojure] [Results]	-Xmx8m	50000000	N/A	26
pidigits	[link]	Arbitrary precision integer arithmetic	[Java] [Clojure] [Results]	-Xmx16m	10000	N/A	16,893
regexdna	[link]	Regex searching and replacement, some parallelism possible	[Java] [Clojure] [Results]	-Xmx616m	< input_file	50,833,411	298
revcomp	[link]	Reading input, simple transformation of input strings, writing output	[Java] [Clojure] [Results]	-Xmx384m	< input_file	254,166,745	254,166,745
spectralnorm	[link]	Double precision arithmetic, lots of parallelism possible	[Java] [Clojure] [Results]	-Xmx8m	5500	N/A	12

Why is -Xmx used?

The JVM command line option -Xmx<megabytes>m is frequently used to specify the maximum heap size for these programs, because it helps reduce the maximum resident memory used by the program. This is another prominent factor measured and reported on the Computer Language Benchmarks game web site, in addition to the elapsed time and total CPU time used. Many Clojure programs that can run fast with a small maximum heap size often use a much larger amount of resident memory if permitted, because they allocate memory that quickly becomes collectible garbage. The JVM will not collect the garbage unless explicitly triggered by the program, or forced to by a smaller-than-default maximum heap size.

AOT compilation commands used

The following commands were used for AOT compilation of all Clojure programs, where CLJ_SOURCE_FILE is the original source file name, CLASSNAME is the same as the "Benchmark name" in the table above, and CLJ_JAR_FILE varied depending upon the Clojure version being measured:

mkdir ./obj

cp ${CLJ_SOURCE_FILE} ./obj/${CLASSNAME}.clj

java -Dclojure.compile.path=./obj/clj -classpath ${CLJ_JAR_FILE} clojure.lang.Compile ${CLASSNAME}

Number of Clojure function calls made

The table below tells how many calls were made to each Clojure function, usually with some indication of where most of the execution time is spent.

Benchmark / class name	Clojure source code	Number of times each Clojure function is called
binarytrees	binarytrees.clojure-rh.clojure	Most time is spent in: bottom-up-tree: 607,475,038 item-check: 607,475,038 Other calls: iterate-trees: 9 main: 1 -main: 1
fannkuchredux	fannkuchredux.clj-12.clj	Most time is spent in: fannkuch-of-permutation: 301,297,917 next-permutation!: 296,930,829 Other calls: init-permutations: 1 partial-fannkuch: 12 fannkuch: 1
fasta	fasta.clj-11.clj (TBD: update for fasta.clj-8.clj source file)	Most time is spent in: gen-random-fast: 200,000,000 Other calls: find-index: 444,000 make-cumulative: 2 make-lookup-table: 2 cycle-bytes: 1 fasta-repeat: 1 fasta-random: 2 write-line: 3
hello	hello.clj	-main: 1
knucleotide	knucleotide.clojure-7.clojure	Most time is spent in: tally-dna-subs-with-len: 16 fasta-description-line: 4,166,671 fasta-description-line-beginning: 2,083,337 Other calls with a small fraction of total run time: medusa-future-thunk: 16 medusa-pmap: 1 fasta-dna-str-with-desc-beginning: 1 dna-str-to-key: 21 key-to-dna-str: 20 getcnt: 148 tally-total: 2 all-tally-to-str: 2 one-tally-to-str: 5 piece-sizes: 7 break-work-into-pieces: 7 do-one-piece: 16 add-tally-hashmaps!: 2 combine-pieces: 7 run: 1
mandelbrot	mandelbrot.clj-9-par-1.3.clj	Most time is spent in: dot: 256,000,000 Other calls: index-to-val: 32,000 ubyte: 32,000,000 compute-row: 16,000 compute-rows: 1 do-mandelbrot: 1
nbody	nbody.clj-14.clj	Most time is spent in: v+!: 500,000,000 p-dt!: 250,000,000 v-dt! 500,000,000 advance: 50,000,000 Other calls: v!: 1 energy (method of type Body): 10 mk-body: 5 init-state: 1 energy (function): 12
pidigits	pidigits.clojure-2.clj	floor-ev: 86,418 ncomp: 43,209 digit: 43,210
regexdna	regexdna.clj-3.clj	slurp-std-input: 1 count-regex-occurences: 9
revcomp	revcomp.clj-15.clj	Most time is spent in: reverse-and-complement!: 3 Other calls: rba-reverse: 4 find-first-byte-idx: 3 ubyte: 256 make-array-char-mapper: 1
spectralnorm	spectralnorm.clojure-6-corrected.clj	multiply-a-v: 160 multiply-at-v: 160 multiply-at-a-v: 20 run-game: 1