Doing Micro benchmarks on the JVM is very hard, due to the Hotspot compiler (assuming you are on a SUN/SAP JVM) doing very advanced optimizations at runtime.
I usually recommend to be very careful about doing any micro benchmarks without having a real JVM hacker nearby ;)
Brent Boyer has written an article series at developerworks that explains how to do it correctly and he also presents a framework to make it easier to write robust microbenchmarks :
Robust Java benchmarking, Part 1
Robust Java benchmarking, Part 2
Also I have not yet tried the framework, it looks promising.