?

Log in

No account? Create an account

GCC 4.3.2 Vs LLVM-GCC 2.5 - leonardo
View:Recent Entries.
View:Archive.
View:Friends.
View:Profile.
View:Website (My Website).

Tags:, , ,
Security:
Subject:GCC 4.3.2 Vs LLVM-GCC 2.5
Time:02:14 am
This is a follow-up of this post:
http://leonardo-m.livejournal.com/73732.html

I have redone the benchmarks using more recent versions of the compilers (LLVM 2.5 and MinGW GCC 4.3.2), the source code is the same (but I have added the new "puzzle" benchmark).



C souce code (from the Shootout site), and timings in OpenOffice format (look for updated versions for LLVM 2.5):
http://www.fantascienza.net/leonardo/js/llvm_vs_gcc.zip
Compilers used:
  LLVM-gcc V. 2.5
  MinGW GCC V. 4.3.2

Compiler options used:
  GCC:      -O3 -s -fomit-frame-pointer -msse3 -march=core2
  GCCb:     -O3 -s -fomit-frame-pointer
  LLVM-GCC: -O3 -s -fomit-frame-pointer -msse3 -march=core2  

CPU used: Intel Core2, 2 GHz (32-bit mode).
All benchmarks use only 1 core.
OS: WindowsXP SP2

Timings (seconds), best of 4    GCC    GCCb   LLVM-GCC
  bintrees, N=15:               4.14   4.15   4.41
  fannkuck, N=11:               5.64   5.09   5.16
  fasta, N=9_000_000, >NUL:     3.68   3.83   2.81
  fasta, N=9_000_000, >file:    4.28   4.45   3.44
  knucleotide (a):              5.02   4.98   5.20
  mandelbrot, N=4_000, >NUL:    3.81   6.14   3.83
  meteor_contest_c, N=2_098:    0.19   0.18   0.16
  meteor_contest_c++, N=2_098:  0.14   0.14   0.15
  nbody, N=10_000_000:          5.91   5.88   5.08
  nsieve, N=12:                 6.19   6.20   6.21
  nsieve_bits, N=13:            4.77   4.70   4.78
  partial_sums, N=7_000_000:    5.77   6.22   5.75
  recursive, N=12:              4.17   4.18   6.38
  reverse_complement (b) >NUL:  1.81   1.80   1.92
  reverse_complement (b) >file: 3.40   2.59   3.47
  spectral_norm, N=3_000:       6.75   6.82   5.72
  sum_file (c):                 3.31   3.32   3.33
  puzzle:                       8.96   8.94  11.16
  
Key:
  (a) = from fasta with N=1_000_000
  (b) = from fasta with N=9_000_000
  (c) input file=71_974_912 bytes, 16384 times 
      the original given file.
In the meantime the llvm developers have fixed the bug shown by LLVM 2.4 for the 'nbody' benchmark.

Now GCC supports the Core2 CPU too.

There are few benchmarks where LLVM is significantly slower:
recursive, N=12
puzzle
reverse_complement (b) >file

Regarding the 'recursive', from other benchmarks I think LLVM may inline recursive functions less than GCC, producing slower code.

I have used the packaged version 4.0 of this compiled MinGW (GCC 4.3.2):
http://nuwen.net/mingw.html
But in the meantime MinGW GCC 4.3.3 is available (package V.4.1). I have done few focused benchmarks with 4.3.3, and the timings are similar (but occasionally a little worse than 4.3.2).
comments: Leave a comment Previous Entry Share Next Entry

the_resistor
Subject:Puzzle benchmark
Link:(Link)
Time:2009-03-07 05:16 am (UTC)
Where is the puzzle benchmark available?
(Reply) (Thread)

the_resistor
Subject:Re: Puzzle benchmark
Link:(Link)
Time:2009-03-07 05:16 am (UTC)
Nevermind. :-)
(Reply) (Parent) (Thread)

the_resistor
Subject:Results differ by platform
Link:(Link)
Time:2009-03-07 07:57 am (UTC)
The results of these benchmarks seem to vary wildly from platform to platform. I just re-ran the tests on X86/Darwin, and got completely different results. For instance, puzzle performs significantly BETTER with LLVM on this platform, while nsieve-bits is noticeably worse with it.
(Reply) (Thread)


leonardo_m
Subject:Re: Results differ by platform
Link:(Link)
Time:2009-03-07 11:29 am (UTC)
Thank you for your info. What are your timings and system? When in doubt the asm source code (for the puzzle and nsieve-bits benchmarks) is a good point to understand what's going on.
(Reply) (Parent) (Thread)

(Anonymous)
Subject:Compile times
Link:(Link)
Time:2009-03-13 05:20 pm (UTC)
It would be interesting to also measure compile times. Is LLVM a faster compiler?
(Reply) (Thread)


leonardo_m
Subject:Re: Compile times
Link:(Link)
Time:2009-03-13 06:01 pm (UTC)
Yes, timing the compile times is quite interesting.
But maybe the little benchmarks I have used here are too much small to test the compiler speed (I may take the total compile time of them all).
The Clang-LLVM front-end is probably a better and more fair way to benchmark the speed of LLVM (I'm using LLVM-GCC). Some benchmarks show that it's indeed faster and more efficient if your CPU has more than one core and your code stresses the front-end more than back-end:
http://clang.llvm.org/performance.html
(Reply) (Parent) (Thread)

dnet_for_gpe
Subject:Re: Puzzle benchmark and compile times
Link:(Link)
Time:2009-05-03 07:20 am (UTC)
Please may you also try run these benchmarks algorithms with GCC + libJIT 0.1.2, and then post us results for libJIT and this research branch of libJIT 0.1.2.5+ too:

http://code.google.com/p/libjit-linear-scan-register-allocator/

with various optimization levels enabled from 0 to 4 (jit_function_set_optimization_level), and at least for different two different ABIs used (jit_abi_internal, jit_abi_cdecl).

So we may see results of LLVM vs (GCC + libJIT) for these benchmarks:

Documentation, and tutorials with libJIT are at this page:

http://www.gnu.org/software/dotgnu/libjit-doc/libjit.html
(Reply) (Thread)


leonardo_m
Subject:Re: Puzzle benchmark and compile times
Link:(Link)
Time:2009-05-03 11:07 am (UTC)
Why do you say me four times the same thing? I am not a robot.
Please, feel free to test it yourself, and post the timing results here in my blog.
(Reply) (Parent) (Thread)

(Anonymous)
Subject:Updated benchmarks
Link:(Link)
Time:2010-06-03 11:30 pm (UTC)
These benchmarks are very interesting, is there any chance of some updated ones for gcc 4.5 and llvm 2.7?
(Reply) (Thread)


leonardo_m
Subject:Re: Updated benchmarks
Link:(Link)
Time:2010-06-03 11:45 pm (UTC)
Thank you. In my website there is already a partial comparison, but it's not easy to find it among all the other stuff.

I have all the code still, and both compilers installed, so I just need an hour or so to find the updated timings. But so far I have not written such updated comparison because now it isn't so interesting, the latest versions of gcc and llvm-gcc give very similar timings with those benchmarks.

I have found other programs that show more variance between gcc and llvm-gcc, but they are often using specialized features that those two compilers optimize better or worse (vectorizations, computed gotos, link time optimizations, etc).
(Reply) (Parent) (Thread)

GCC 4.3.2 Vs LLVM-GCC 2.5 - leonardo
View:Recent Entries.
View:Archive.
View:Friends.
View:Profile.
View:Website (My Website).