[CIG-SEISMO] SPECFEM3D Globe verification

[Eric Heien]

Hi Ping,

We have found that SPECFEM3D (and actually all CIG codes, and probably most scientific codes in general) can return slightly different results such as this depending on the platform, compiler, hardware, etc.  We're looking into how much difference is acceptable and what is the best way to measure that difference.  It appears that the largest absolute difference in your results is about 9e-6, but the largest relative difference is nearly 200% (mean is about 16% difference).  The SPECFEM developers will have to weigh in regarding whether this is high.

I've found that for SPECFEM, cross correlation with 0 time lag between the results is generally a good measure of how similar they are.  This will be sensitive to random noise and changing time offsets between the signals, but will not be too sensitive to minor fluctuations like you see caused by differences in floating point arithmetic.

Overall I think this result wouldn't be considered invalid and SPECFEM3D is still a good measure for testing relative performance between clusters.  If you are concerned about exact reproducibility for results you might look at the following resources:

http://software.intel.com/en-us/articles/introduction-to-the-conditional-numerical-reproducibility-cnr
http://software.intel.com/en-us/articles/run-to-run-reproducibility-of-floating-point-calculations-for-applications-on-intel-xeon
http://www.nccs.nasa.gov/images/FloatingPoint_consistency.pdf

Hope that answers your question,

-Eric

On Jun 25, 2013, at 11:23 AM, Luo, Ping wrote:

> Hello,
> 
> We are going to use SPECFEM3D Globe as a benchmark for new cluster purchase. I have built SPECFEM3D Globe one a Sandy Bridge cluster and am running "small_benchmark_run_to_test_a_new_machine" from the EXAMPLES directory. In the README file, it mentions that "If you notice any difference on any of the 387 pages, the code on your new machine (or your new machine, or both) has a problem." I indeed find my simulation results are a little different from the reference values. I am attaching one graph where the two sets of results seems off the most. The files attached are:
> 
>     BBSR.IU.MXE.sem.ascii
>     BBSR.IU.MXE.sem.ascii-ref
>     BBSR_IU_MXE.seismograms_comparison.pdf
> 
> 
> Do you think there is something wrong with my simulation? What could be the problem? Since we are only interested in relative performance among different clusters, is it OK to use SPECFEM3D Globe as a benchmark of cluster selection even though the results are not correct?
> 
> I used the Intel XE compilers and the intel MPI for compilation. I didn't change the default compilation flags.
> 
> Thank you for help,
> 
> Sincerely,
> 
> Ping
> Lead Systems Administrator
> Supercomputing Facility
> Texas A&M University<BBSR_IU_MXE_seismograms_comparison.pdf><BBSR.IU.MXE.sem.ascii-ref><BBSR.IU.MXE.sem.ascii>_______________________________________________
> CIG-SEISMO mailing list
> CIG-SEISMO at geodynamics.org
> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo