[cig-commits] r5338 - mc/3D/CitcomS/trunk/doc/manual

Tue Nov 21 15:03:39 PST 2006

Author: luis
Date: 2006-11-21 15:03:38 -0800 (Tue, 21 Nov 2006)
New Revision: 5338

Modified:
   mc/3D/CitcomS/trunk/doc/manual/citcoms.lyx
Log:
Expanded section on optimizing parallel I/O.


Modified: mc/3D/CitcomS/trunk/doc/manual/citcoms.lyx
===================================================================

--- mc/3D/CitcomS/trunk/doc/manual/citcoms.lyx	2006-11-21 22:20:41 UTC (rev 5337)
+++ mc/3D/CitcomS/trunk/doc/manual/citcoms.lyx	2006-11-21 23:03:38 UTC (rev 5338)
@@ -1,4 +1,4 @@
-#LyX 1.4.2 created this file. For more info see http://www.lyx.org/
+#LyX 1.4.1 created this file. For more info see http://www.lyx.org/
 \lyxformat 245
 \begin_document
 \begin_header
@@ -1405,7 +1405,7 @@
 \end_layout
 
 \begin_layout Standard
-On Unix or Linux systems, there is a high likelyhood that a useable C compiler
+On Unix or Linux systems, there is a high likelihood that a useable C compiler
  is already installed.
  To check, type 
 \family typewriter
@@ -5046,65 +5046,176 @@
 \end_layout
 
 \begin_layout Standard
-There are several platform-dependent parameters that are used by the HDF5
- library and the underlying MPI-IO routines to optimize the performance
- of parallel I/O.
+There are several platform-dependent parameters used by the HDF5 library
+ and the underlying MPI-IO routines to optimize the performance of parallel
+ I/O.
  The optimal values for these parameters may vary from file system to file
- system, and ideally you would hardcode these values for your installation
- of CitComS on a particular parallel file system.
- For illustration purposes we have exposed a relevant subset of these parameters
- to the input file.
- You can use these to tune the performance of the parallel I/O on your system,
- although to change any parameters not specified here, you will have to
- modify parts of the CitComS source code.
- Example values are given below:
+ system.
+ Ideally, before compiling CitComS, the build procedure would configure
+ these parameters based on your platform.
+ 
 \end_layout
 
-\begin_layout LyX-Code
+\begin_layout Standard
+In order to facilitate the process of gathering I/O performance data from
+ a variety of parallel file systems such as GPFS, PVFS, IBRIX FusionFS,
+ you can specify the following parameters in the CitComS input file.
+ You may use these to tune the performance of the parallel I/O on your system,
+ although in future versions of CitComS this might become unnecessary for
+ supported file systems.
+\end_layout
 
+\begin_layout Standard
+All values are assumed to be specified in bytes, unless otherwise indicated.
 \end_layout
 
+\begin_layout Enumerate
+MPI file hints for collective buffering file access.
+\end_layout
+
 \begin_deeper
-\begin_layout Standard
+\begin_layout Enumerate
 
-\family roman
-To control the size of the buffer used in collective I/O
+\family typewriter
+\size small
+cb_block_size
+\family default
+\size default
+: Target nodes will access data in chunks of this size.
 \end_layout
 
-\begin_layout LyX-Code
+\begin_layout Enumerate
 
 \family typewriter
-[CitcomS.solver.output]
+\size small
+cb_buffer_size
+\family default
+\size default
+: Specifies the total buffer space on each target node.
+ Set this parameter to a multiple of 
+\family typewriter
+\size small
+cb_block_size
+\family default
+\size default
+.
 \end_layout
 
-\begin_layout LyX-Code
-cb_block_size=1048576
+\end_deeper
+\begin_layout Enumerate
+HDF5 file access properties.
 \end_layout
 
-\begin_layout LyX-Code
-cb_buffer_size=4194304
+\begin_deeper
+\begin_layout Enumerate
+
+\family typewriter
+\size small
+sieve_buf_size
+\family default
+\size default
+: Maximum size of data sieve buffer.
 \end_layout
 
-\begin_layout Standard
+\begin_layout Enumerate
 
-\family roman
-To change the size of the buffer used by the sieve algorithm
+\family typewriter
+\size small
+output_alignment
+\family default
+\size default
+: Alignment interval size in bytes.
 \end_layout
 
-\begin_layout LyX-Code
+\begin_layout Enumerate
 
 \family typewriter
-[CitcomS.solver.output]
+\size small
+output_alignment_threshold
+\family default
+\size default
+: Allocation requests above this size will be aligned on a memory address
+ that is a multiple of 
+\family typewriter
+\size small
+output_alignment
+\family default
+\size default
+.
 \end_layout
 
+\begin_layout Enumerate
+
+\family typewriter
+\size small
+cache_rdcc_nelmts
+\family default
+\size default
+: Maximum number of chunks that can be stored in the raw data chunk cache.
+\end_layout
+
+\begin_layout Enumerate
+
+\family typewriter
+\size small
+cache_rdcc_nbytes
+\family default
+\size default
+: Size of raw data chunk cache in bytes.
+\newline
+
+\end_layout
+
+\end_deeper
+\begin_layout Standard
+For more details, you can refer to the following references: The list of
+ MPI-IO reserved file hints is available on Section 7.2.8 in the MPI-2 reference
+ book, which can also be accessed online from 
+\begin_inset LatexCommand \url{http://www-unix.mcs.anl.gov/mpi/mpi-standard/mpi-report-2.0/node182.htm}
+
+\end_inset
+
+.
+ For a list of HDF5 file access properties, refer to Chapter 2, Section
+ 7.3, in the 
+\begin_inset LatexCommand \url[HDF5 documentation]{http://hdf.ncsa.uiuc.edu/HDF5/doc/UG/08_TheFile.html}
+
+\end_inset
+
+.
+ Finally, for a short explanation of raw data chunk caching, you can refer
+ to 
+\begin_inset LatexCommand \url{http://hdf.ncsa.uiuc.edu/HDF5/doc/Caching.html}
+
+\end_inset
+
+.
+\end_layout
+
+\begin_layout Standard
+You should also refer to the documentation for your particular parallel
+ file system and check if there are any other MPI or HDF5 hints that could
+ improve your parallel I/O performance.
+\end_layout
+
+\begin_layout Standard
+(
+\series bold
+TODO: clean up references
+\series default
+)
+\end_layout
+
 \begin_layout LyX-Code
-sieve_buf_size=1048576
+
 \end_layout
 
+\begin_deeper
 \begin_layout Standard
 
 \family roman
-To change the data alignment parameters
+Finally, here is an example section that would appear in a CitComS input
+ file:
 \end_layout
 
 \begin_layout LyX-Code
@@ -5114,23 +5225,23 @@
 \end_layout
 
 \begin_layout LyX-Code
-output_alignment=262144
+cb_block_size=1048576              # 1 MiB
 \end_layout
 
 \begin_layout LyX-Code
-output_alignment_threshold=524288
+cb_buffer_size=4194304             # 4 MiB
 \end_layout
 
-\begin_layout Standard
+\begin_layout LyX-Code
+sieve_buf_size=1048576             # 1 MiB
+\end_layout
 
-\family roman
-To change the size of the raw data chunk cache
+\begin_layout LyX-Code
+output_alignment=262144            # 256 KiB
 \end_layout
 
 \begin_layout LyX-Code
-
-\family typewriter
-[CitcomS.solver.output]
+output_alignment_threshold=524288  # 512 KiB
 \end_layout
 
 \begin_layout LyX-Code
@@ -5146,19 +5257,6 @@
 
 \end_layout
 
-\begin_layout Standard
-For more details, refer to
-\series bold
- (TODO: insert reference to chapter)
-\series default
- in the HDF5 documentation, the 
-\series bold
-(TODO: insert reference to chapter)
-\series default
- in the MPI-2 reference book, and the documentation for your parallel file
- system.
-\end_layout
-
 \begin_layout Section
 Accessing Data
 \end_layout
@@ -5172,7 +5270,7 @@
 \family typewriter
 h5ls
 \family default
- from the HDF5 library
+ which is included with the HDF5 software:
 \end_layout
 
 \begin_layout LyX-Code

