[CIG-MC] time information in citcoms

[Eh Tan]

Tobias Hoeink wrote:
> Hi Eh,
>
> We are trying to scale citcoms' performance on a NERSC computer, and
> wonder which information in the output would be a useful quantity.
>
> Does the CPU total = ??? & line in the log file deliver the actual
> time spend on the CPU, or the walltime? Same for the last line in the
> log file: "Average cpu time taken for velocity step = 6.071693".?
>
> If I understand correctly, Adrian Lenardic and Mark Richards (cc'd)
> got different numbers for the same input file on the same machine.
> This suggests to me that the times outputted depend on the overal node
> usage, not on the CPU cycle for our process.
>
> Which number should we use?
>
> Thanks,
> Tobias
>
>
> ps. We got the restart to work on franklin at NERSC. Thanks for your
> help!


Hi Tobias,
(cc'ing to cig-mc)

The timing information is obtained through MPI_Wtime(). The MPI
documentation says this function returns the wall clock time, not cpu
time. You can change the function CPU_time0() in lib/Parallel_util.c to
get the cpu time.

Don't trust the last line "Average cpu time taken for velocity step
=..." in the log file and stderr. I am sorry the number reported is
broken, and I never got a chance to fix it. You can trust the line "CPU
total = ..." though. A similar timing information can be found in the
*.time file.

The convergence parameter in CitcomS is mesh dependent, especially the
divergence term. To conduct a meaningful scaling test, there are a few
parameters must be changed.

   1. "tic_method": The initial temperature field should be mesh
      independent. This precludes "tic_method=0". "tic_method=3" should
      be fine.
   2. "tole_compressibility" should be a very small value e.g. 1e-20.
      The current CitcomS computes norm(div(u)) in a mesh-dependent way.
      The computed norm is scaled with 1/(the number of elements). As a
      result, norm(div(u)) is not suitable as a convergence criterion in
      the scaling test. Setting "tole_compressibility" to a small value
      would walk around the problem. This problem will be fixed in the
      next release (v3.1).


-- 
Eh Tan
Staff Scientist
Computational Infrastructure for Geodynamics
2750 E. Washington Blvd. Suite 210
Pasadena, CA 91107
(626) 395-1693
http://www.geodynamics.org