[cig-commits] r18151 - in seismo/3D/SPECFEM3D_GLOBE/trunk: EXAMPLES/noise_examples/global_long EXAMPLES/noise_examples/global_short EXAMPLES/noise_examples/regional doc/USER_MANUAL src/specfem3D
yangl at geodynamics.org
yangl at geodynamics.org
Sun Mar 27 14:01:03 PDT 2011
Author: yangl
Date: 2011-03-27 14:01:03 -0700 (Sun, 27 Mar 2011)
New Revision: 18151
Modified:
seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/global_long/NOISE123.submit_atten
seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/global_short/NOISE123.submit_atten
seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/regional/NOISE123.submit_atten
seismo/3D/SPECFEM3D_GLOBE/trunk/doc/USER_MANUAL/manual_SPECFEM3D_GLOBE.pdf
seismo/3D/SPECFEM3D_GLOBE/trunk/doc/USER_MANUAL/manual_SPECFEM3D_GLOBE.tex
seismo/3D/SPECFEM3D_GLOBE/trunk/src/specfem3D/noise_tomography.f90
Log:
another update on NOISE_TOMOGRAPHY in SPECFEM3D_GLOBE
Modified: seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/global_long/NOISE123.submit_atten
===================================================================
--- seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/global_long/NOISE123.submit_atten 2011-03-27 12:15:09 UTC (rev 18150)
+++ seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/global_long/NOISE123.submit_atten 2011-03-27 21:01:03 UTC (rev 18151)
@@ -42,9 +42,6 @@
######################################## simulation ###############################################
################################################################### step 1: ensemble forward source
cp DATA/Par_file_NOISE_1_attenuation DATA/Par_file
-cp DATA/STATIONS_NOISE DATA/STATIONS
-cp DATA/STATIONS_NOISE DATA/STATIONS_ADJOINT
-cp DATA/CMTSOLUTION_NOISE DATA/CMTSOLUTION
sleep 2
mpiexec -np $numnodes -hostfile zzz_job_info/compute_nodes_all $PWD/xmeshfem3D_attenuation
sleep 2
@@ -107,7 +104,7 @@
## collect outputs (e.g., kernels, meshes) from local nodes to global disk
d=`date`
echo "Start collecting outputs from nodes, $d"
-pbsdsh $PBS_O_WORKDIR/NOISE_collect $LOCAL_PATH_ALL OUTPUT_FILES/
+pbsdsh $PBS_O_WORKDIR/NOISE_collect $LOCAL_PATH_ALL $PWD/OUTPUT_FILES/
d=`date`
echo "Finish collecting outputs from nodes, $d"
Modified: seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/global_short/NOISE123.submit_atten
===================================================================
--- seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/global_short/NOISE123.submit_atten 2011-03-27 12:15:09 UTC (rev 18150)
+++ seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/global_short/NOISE123.submit_atten 2011-03-27 21:01:03 UTC (rev 18151)
@@ -42,9 +42,6 @@
######################################## simulation ###############################################
################################################################### step 1: ensemble forward source
cp DATA/Par_file_NOISE_1_attenuation DATA/Par_file
-cp DATA/STATIONS_NOISE DATA/STATIONS
-cp DATA/STATIONS_NOISE DATA/STATIONS_ADJOINT
-cp DATA/CMTSOLUTION_NOISE DATA/CMTSOLUTION
sleep 2
mpiexec -np $numnodes -hostfile zzz_job_info/compute_nodes_all $PWD/xmeshfem3D_attenuation
sleep 2
@@ -107,7 +104,7 @@
## collect outputs (e.g., kernels, meshes) from local nodes to global disk
d=`date`
echo "Start collecting outputs from nodes, $d"
-pbsdsh $PBS_O_WORKDIR/NOISE_collect $LOCAL_PATH_ALL OUTPUT_FILES/
+pbsdsh $PBS_O_WORKDIR/NOISE_collect $LOCAL_PATH_ALL $PWD/OUTPUT_FILES/
d=`date`
echo "Finish collecting outputs from nodes, $d"
Modified: seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/regional/NOISE123.submit_atten
===================================================================
--- seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/regional/NOISE123.submit_atten 2011-03-27 12:15:09 UTC (rev 18150)
+++ seismo/3D/SPECFEM3D_GLOBE/trunk/EXAMPLES/noise_examples/regional/NOISE123.submit_atten 2011-03-27 21:01:03 UTC (rev 18151)
@@ -42,9 +42,6 @@
######################################## simulation ###############################################
################################################################### step 1: ensemble forward source
cp DATA/Par_file_NOISE_1_attenuation DATA/Par_file
-cp DATA/STATIONS_NOISE DATA/STATIONS
-cp DATA/STATIONS_NOISE DATA/STATIONS_ADJOINT
-cp DATA/CMTSOLUTION_NOISE DATA/CMTSOLUTION
sleep 2
mpiexec -np $numnodes -hostfile zzz_job_info/compute_nodes_all $PWD/xmeshfem3D_attenuation
sleep 2
@@ -107,7 +104,7 @@
## collect outputs (e.g., kernels, meshes) from local nodes to global disk
d=`date`
echo "Start collecting outputs from nodes, $d"
-pbsdsh $PBS_O_WORKDIR/NOISE_collect $LOCAL_PATH_ALL OUTPUT_FILES/
+pbsdsh $PBS_O_WORKDIR/NOISE_collect $LOCAL_PATH_ALL $PWD/OUTPUT_FILES/
d=`date`
echo "Finish collecting outputs from nodes, $d"
Modified: seismo/3D/SPECFEM3D_GLOBE/trunk/doc/USER_MANUAL/manual_SPECFEM3D_GLOBE.pdf
===================================================================
(Binary files differ)
Modified: seismo/3D/SPECFEM3D_GLOBE/trunk/doc/USER_MANUAL/manual_SPECFEM3D_GLOBE.tex
===================================================================
--- seismo/3D/SPECFEM3D_GLOBE/trunk/doc/USER_MANUAL/manual_SPECFEM3D_GLOBE.tex 2011-03-27 12:15:09 UTC (rev 18150)
+++ seismo/3D/SPECFEM3D_GLOBE/trunk/doc/USER_MANUAL/manual_SPECFEM3D_GLOBE.tex 2011-03-27 21:01:03 UTC (rev 18151)
@@ -1869,8 +1869,8 @@
Users are recommended to familiarize themselves first with the procedures for running regular earthquake
simulations (Chapters \ref{cha:Running-the-Mesher}--\ref{cha:Regional-Simulations}) and
adjoint simulations (Chapter \ref{cha:Adjoint-Simulations}).
-Also, make sure you read the paper `Noise cross-correlation sensitivity kernels'
-by {(REF: Tromp et al., 2010)} in order to understand noise simulations from a theoretical perspective.
+Also, make sure you read the paper `Noise cross-correlation sensitivity kernels' \citep{trompetal2010}
+in order to understand noise simulations from a theoretical perspective.
\section{New Requirements on `Old' Input Parameter Files}
@@ -1880,12 +1880,15 @@
\texttt{DATA/CMTSOLUTION} is required for all simulations. However, it may seem unexpected to have it listed here, since the
noise simulations should have nothing to do with the earthquake -- hence the \texttt{DATA/CMTSOLUTION}.
For noise simulations, it is critical to have no earthquakes. In other words, the moment tensor
-specified in \texttt{DATA/CMTSOLUTION} must be set to zero! \\
+specified in \texttt{DATA/CMTSOLUTION} must be set to ZERO! \\
\texttt{DATA/STATIONS} remains the same as in previous earthquake simulations,
except that the order of stations listed in \texttt{DATA/STATIONS} is now important.
-The order will be used to define the `master' receiver,
-i.e., the one that simultaneously cross correlates with the others.\\
+The order will be used later to identify the `master' receiver,
+i.e., the one that simultaneously cross correlates with the others.
+Please be noted that the actual station file used in the simulation is \texttt{OUTPUT\_FILES/STATIONS\_FILTERED},
+which is generated when you run your simulations. (e.g., in regional simulations we may have included stations out of the
+region of our interests in \texttt{DATA/STATIONS}, so we have to get rid of them.)\\
\texttt{DATA/Par\_file} also requires careful attention.
New to Version 5.1.0, a parameter called \texttt{NOISE\_TOMOGRAPHY} has been added that specifies the type of simulation to be run.
@@ -1893,13 +1896,13 @@
For example, when \texttt{NOISE\_TOMOGRAPHY} equals 0, a regular earthquake simulation will be run.
When \texttt{NOISE\_TOMOGRAPHY} is equal to 1/2/3, you are about to run
step 1/2/3 of the noise simulations respectively.
-Should you be confused by the three steps, refer to {(REF: Tromp et al., 2010)} for details.\\
+Should you be confused by the three steps, refer to \citet{trompetal2010} for details.\\
Another change to \texttt{DATA/Par\_file} involves the parameter \texttt{RECORD\_LENGTH\_IN\_MINUTES}.
While for regular earthquake simulations this parameter specifies the length of synthetic seismograms generated,
for noise simulations it specifies the length of the seismograms used to compute cross correlations.
The actual cross correlations are thus twice this length.
-The code automatically makes the modification accordingly, if \texttt{NOISE\_TOMOGRAPHY} is not zero.\\
+The code automatically makes modification accordingly, if \texttt{NOISE\_TOMOGRAPHY} is not zero.\\
There are other parameters in \texttt{DATA/Par\_file} which should be given specific values.
For instance, \texttt{NUMBER\_OF\_RUNS} and \texttt{NUMBER\_OF\_THIS\_RUN} must be 1;
@@ -1917,7 +1920,9 @@
please make sure that the \texttt{LOCAL\_PATH} in \texttt{DATA/Par\_file} is in fact local,
not globally shared. Because we have to save the wavefields at the earth's surface at every time step,
it is quite problematic to have a globally shared \texttt{LOCAL\_PATH}, in terms of both disk storage and
-I/O speed.
+I/O speed. {\color{red}(the new implementation uses the same way we save absorbing boundaries, and thus may be
+more effective and less architecture demanding. ask Qinya to test the new version on her cluster, which has a
+global storage system)}
\section{Noise Simulations: Step by Step}
@@ -1927,7 +1932,7 @@
for example, the ensemble-averaged noise spectrum, the noise distribution etc.
However, since there are a few `new' files, it is better to introduce them sequentially.
Read through this section even if you don't understand some parts temporarily,
-since some examples will be provided to you at the end of this chapter.
+since some examples you can go through are provided in this package.
\subsection{Pre-simulation}
@@ -1948,7 +1953,8 @@
Before compilation, the \texttt{DATA/Par\_file} must be specified correctly, e.g.,
\texttt{NOISE\_TOMOGRAPHY} shouldn't be zero; \texttt{RECORD\_LENGTH\_IN\_MINUTES},
\texttt{NEX\_XI} and \texttt{NEX\_ETA} must be what you want in your real simulations.
-Otherwise you may get wrong informations which will cause problems later. \\
+Otherwise you may get wrong informations which will cause problems later.
+(it is good to always re-complie the code before you run simulations)\\
\item
After compiling, you will find two important numbers besides the needed executables:
@@ -1958,8 +1964,9 @@
\texttt{time\_stepping of the solver will be: 0.19000}\\
The first number will be denoted as \texttt{NSTEP} from now on, and the second one as \texttt{dt}.
-The two numbers are essential to calculate the ensemble-averaged noise spectrum from Peterson's noise
-model {(REF: Peterson, 1993)}. Should you miss the two numbers, you can run
+The two numbers are essential to calculate the ensemble-averaged noise spectrum from either Peterson's noise
+model or just a simple flat power spectrum (corresponding to 1-bit preprocessing).
+Should you miss the two numbers, you can run
\texttt{./xcreate\_header\_file} to bring them up again (with correct \texttt{DATA/Par\_file}!).
FYI, \texttt{NSTEP} is determined by \texttt{RECORD\_LENGTH\_IN\_MINUTES} in \texttt{DATA/Par\_file},
which is automatically doubled in noise simulations; whereas \texttt{dt} is derived from
@@ -1988,13 +1995,13 @@
\texttt{the source time function has been saved in:}\\
\texttt{/data2/yangl/3D\_NOISE/S\_squared} (note this path must be different)\\
\texttt{S\_squared should be put into directory:}\\
-\texttt{./NOISE\_TOMOGRAPHY/ in the SPECFEM3D package}\\
+\texttt{./NOISE\_TOMOGRAPHY/ in the SPECFEM3D\_GLOBE package}\\
In other words, the Matlab script creates a file called \texttt{S\_squared},
which is the first `new' input file we encounter for noise simulations.
One may choose a flat noise spectrum rather than Peterson's noise model.
-This can be done easily by modifying the Matlab script a little.
+This can be done easily by modifying the Matlab script a little bit.
\item
Create a new directory in the SPECFEM3D\_GLOBE package, name it as \texttt{NOISE\_TOMOGRAPHY}.
@@ -2009,28 +2016,29 @@
\item
Create a file called \texttt{NOISE\_TOMOGRAPHY/irec\_master\_noise}. Note that this file should be put
in directory \texttt{NOISE\_TOMOGRAPHY} as well. This file contains only one integer, which is the ID
-of the `master' receiver. For example, if this file contains 5, it means that the fifth receiver
+of the `master' receiver. For example, if in this file shows 5, it means that the fifth receiver
listed in \texttt{DATA/STATIONS} becomes the `master'. That's why we mentioned previously that
the order of receivers in \texttt{DATA/STATIONS} is important.
Note that in regional (1- or 2-chunk) simulations, the \texttt{DATA/STATIONS}
may contain receivers not within the selected chunk(s).
Therefore, the integer in \texttt{NOISE\_TOMOGRAPHY/irec\_master\_noise}
-is actually the ID in \texttt{DATA/STATIONS\_FILTERED} (which is generated by \texttt{xmeshfem3D}).
+is actually the ID in \texttt{DATA/STATIONS\_FILTERED} (which is generated by \texttt{xspecfem3D}).
\item
Create a file called \texttt{NOISE\_TOMOGRAPHY/nu\_master}. This file holds three numbers,
forming a (unit) vector. It describes which component we are cross-correlating at the `master'
-receiver, i.e., ${\hat{\bf \nu}}^{\alpha}$ in {(REF: Tromp et al., 2010)}. The three numbers correspond to
+receiver, i.e., ${\hat{\bf \nu}}^{\alpha}$ in \citet{trompetal2010}. The three numbers correspond to
N/E/Z components respectively.
\item
-Describe the noise direction and distributions in \texttt{noise\_tomography.f90}.
+Describe the noise direction and distributions in \texttt{src/specfem3d/noise\_tomography.f90}.
Search for a subroutine called \texttt{noise\_distribution\_direction}
in \texttt{noise\_tomography.f90}. It is actually located at the very beginning
of \texttt{noise\_tomography.f90}. The default assumes vertical noises and a uniform distribution
-across the whole model domain. It should be quite self-explanatory for modifications.
-Should you modify this part, you have to re-compile the source code.
+across the whole physical domain. It should be quite self-explanatory for modifications.
+Should you modify this part, you have to re-compile the source code. (again, that's why we recommend
+that you always re-compile the code before you run simulations)
\end{itemize}
@@ -2044,24 +2052,24 @@
It is quite problematic to have a globally shared \texttt{LOCAL\_PATH}, in terms of both disk storage and
speed of I/O (we have to save the wavefields at the earth's surface at every time step).
-As discussed in {(REF: Tromp at al., 2010)}, it takes three simulations to obtain one contribution
+As discussed in \citet{trompetal2010}, it takes three simulations to obtain one contribution
of the ensemble sensitivity kernels:
\begin{itemize}
\item
-Step 1: simulation for generating wavefields\\
+Step 1: simulation for generating wavefield\\
\texttt{SIMULATION\_TYPE=1}\\
\texttt{NOISE\_TOMOGRAPHY=1}\\
\texttt{SAVE\_FORWARD} not used, can be either .true. or .false.
\item
-Step 2: simulation for ensemble forward wavefields\\
+Step 2: simulation for ensemble forward wavefield\\
\texttt{SIMULATION\_TYPE=1}\\
\texttt{NOISE\_TOMOGRAPHY=2}\\
\texttt{SAVE\_FORWARD=.true.}
\item
-Step 3: simulation for ensemble adjoint wavefields and sensitivity kernels\\
+Step 3: simulation for ensemble adjoint wavefield and sensitivity kernels\\
\texttt{SIMULATION\_TYPE=3}\\
\texttt{NOISE\_TOMOGRAPHY=3}\\
\texttt{SAVE\_FORWARD=.false.}
@@ -2072,15 +2080,15 @@
\end{itemize}
It's better to run the three steps continuously within the same job, otherwise you have to collect the saved surface movies
-from the old nodes/CPUs to the new nodes/CPUs.\\
+from the old nodes to the new nodes.\\
\subsection{Post-simulation}
After those simulations, you have all stuff you need, either in the \texttt{OUTPUT\_FILES} or
-in the directory specified by \texttt{DATA/LOCAL\_PATH} (most probably on local nodes).
+in the directory specified by \texttt{LOCAL\_PATH} in \texttt{DATA/Par\_file} (most probably on local nodes).
Collect whatever you want from the local nodes to your workstation, and then visualize them.
This process is the same as what you may have done for regular earthquake simulations.
-Refer to other chapters if you have problems.\\
+Refer Chapter~\ref{cha:graphics} if you have problems.\\
Simply speaking, two outputs are the most interesting: the simulated ensemble cross correlations and
one contribution of the ensemble sensitivity kernels.\\
@@ -2094,19 +2102,20 @@
for different components as well, not necessarily the cross correlations between vertical components.\\
One contribution of the ensemble cross-correlation sensitivity kernels are obtained after Step 3,
-stored in the \texttt{DATA/LOCAL\_PATH} on local nodes.
+stored in the \texttt{LOCAL\_PATH} on local nodes.
The ensemble kernel files are named the same as regular earthquake kernels.\\
You need to run another three simulations to get the other contribution of the ensemble kernels,
-using different forward and adjoint sources given in {(REF: Tromp et al., 2010)}.
+using different forward and adjoint sources given in \citet{trompetal2010}.
\section{Examples}
In order to illustrate noise simulations in an easy way, three examples are provided in
\texttt{EXAMPLES/noise\_examples/}.
Note however that they are created for a specific cluster (SESAME at PRINCETON).
+You have to modify them to fit your own cluster.
-The three examples are (contained in directory \texttt{EXAMPLES/noise\_examples/}):\\
+The three examples can be executed using (in directory \texttt{EXAMPLES/noise\_examples/}):\\
\texttt{./pre-processing regional}
@@ -2117,14 +2126,14 @@
Each corresponds to one example, but they are pretty similar.
Although the job submission only works on SESAME at PRINCETON, the procedure itself is universal.
-You may review the whole process described in the last section by following the commands in those examples.
+You may review the whole process described in the last section by following commands in those examples.
Finally, note again that those examples show only one contribution of the ensemble kernels!
%>YANGL
-\chapter{Graphics}
+\chapter{\label{cha:graphics}Graphics}
\section{\label{sec:Meshes}Meshes}
Modified: seismo/3D/SPECFEM3D_GLOBE/trunk/src/specfem3D/noise_tomography.f90
===================================================================
--- seismo/3D/SPECFEM3D_GLOBE/trunk/src/specfem3D/noise_tomography.f90 2011-03-27 12:15:09 UTC (rev 18150)
+++ seismo/3D/SPECFEM3D_GLOBE/trunk/src/specfem3D/noise_tomography.f90 2011-03-27 21:01:03 UTC (rev 18151)
@@ -742,7 +742,6 @@
! write(outputname,"('/proc',i6.6,'_surface_movie',i6.6)") myrank, it
! open(unit=IIN_NOISE,file=trim(LOCAL_PATH)//outputname,status='old',form='unformatted',action='read',iostat=ios)
! if( ios /= 0) call exit_MPI(myrank,'file '//trim(outputname)//' does NOT exist!')
-! call read_abs(9,SURFACE_MOVIE,CUSTOM_REAL*NDIM*NGLLX*NGLLY*nspec_top,it)
!
! read(IIN_NOISE) store_val_ux
! read(IIN_NOISE) store_val_uy
More information about the CIG-COMMITS
mailing list