[CIG-SEISMO] BG/P support for SPECFEM3d

[Peter Danecek]

On 27 Feb 2010, at 09:37, Mark Cheeseman wrote:

> Sorry, I forgot to include the error messages I receive:
>
> When I first try to compile with the IBM XLF90 compiler, I first get  
> the error message about the xcreate_header_file binary
>
> 1501-510  Compilation successful for file exit_mpi.f90.
> mpixlf90  -O3 -qsave -qstrict -qtune=450 -qarch=450d -qfree=f90 - 
> qsuffix=f=f90 -qhalt=e -qsigtrap - 
> qinitauto 
> =7Fswap_all,mxm_m1_m2_5points,mxm_m1_m1_5points,mxm_m2_m1_5points -C  
> -qddim -qfloat=nans -qfullpath -o xmeshfem3D j/exit_mpi.o obj/ 
> libspecfem.a
> mkdir -p OUTPUT_FILES
> ./xcreate_header_file
> make: *** [OUTPUT_FILES/values_from_mesher.h] Illegal instruction
>
> Running the ./xcreate_header_file seems to work fine with the  
> required header file being generated in the OUTPUT_FILES  
> subdirectory.  Next, I comment out the call to the  
> xcreate_header_file binary in the main Makefile and rerun make.  The  
> result is a large number of error messages about zero-sized arrays  
> in the following source code files
>
>     compute_forces_crust_mantle.f90
>     compute_forces_inner_core.f90
>     compute_forces_outer_core.f90
>     write_movie_volume.f90
>
> with the error messages being
> .
> .
> "./write_movie_volume.f90", line 425.14: 1511-013 (W) The value of  
> the DO-loop increment should be negative when initial value is  
> greater than the terminal value.
> "./write_movie_volume.f90", line 429.12: 1516-152 (S) Zero-sized  
> arrays must not be subscripted.
> "./write_movie_volume.f90", line 431.18: 1516-152 (S) Zero-sized  
> arrays must not be subscripted.
> "./write_movie_volume.f90", line 432.28: 1516-152 (S) Zero-sized  
> arrays must not be subscripted.
> ** write_movie_volume_vector   === End of Compilation 4 ===
> "./write_movie_volume.f90", line 495.16: 1511-013 (W) The value of  
> the DO-loop increment should be negative when initial value is  
> greater than the terminal value.
> "./write_movie_volume.f90", line 502.52: 1516-152 (S) Zero-sized  
> arrays must not be subscripted.
> ** write_movie_volume_divcurl   === End of Compilation 5 ===
> 1501-511  Compilation failed for file write_movie_volume.f90.
> .
> .
> .
> The actual offending parameters (after some tracking) are those  
> defined in the header file
>
>     ./OUTPUT_FILES/values_from_mesher.h
>
> and include
>
>    NSPEC_CRUST_MANTLE, NSPEC_OUTER_CORE, NSPEC_INNER_CORE
>
> It looks like I am missing an important configuration step or  
> datafile.  Any assistance would be greatly appreciated.
>


Hi Mark,

The xcreate_header_file executable is used to create some header file  
(values_from_mesher.h) which otherwise would be created by the mesher  
exec. But in contrast to the mesher itself, it does not need to be run  
in parallel though the queuing system. Instead xcreate_header_file  
calculates/estimates the size of several arrays immediately and can be  
executed during compilation process.

 From what you are writing, i would guess as well that the problem  
arrises from the cross compiler environment. The xcreate_header_file  
binary is created for the target machine, but executed on the  
compiling machine itself. That's why the makefile fails. So you could  
try two things:

(1) compile the xcreate_header_file exec for the local compiling  
machine (not with the cross compiler) to generate the  
values_from_mesher.h header file during compilation. You probably need  
to compile some dependences too and it is probably not straight  
forward to get the Makefile working properly.
NOTE: I am not very sure to which extend architectural differences  
will influences the resulting estimates in values_from_mesher.h file.  
So be careful with this approach.

(2) use the more conservative way: First (cross-)compile the mesher  
executable only (make target: meshfem3D) and run it through the queue  
system. Than use the generated values_from_mesher.h file to compile  
the solver (make target: specfem3D) .

Running xcreate_header_file through the queue system (serial job) --  
as I understand you have tried -- might also work, though I do not see  
any benefits with respect to approach (2). From the error messages,  
however, it looks like the generated  values_from_mesher.h is not  
valid or incomplete. Check if the content looks reasonable. (does the  
execution really complete correctly, is your Par_file reasonable)

An other problem might arrise from running the ./configure script (on  
the compiling machine, i suppose). To my understanding this should  
fail for the same reason the xcreate_... binary does, or at best give  
results relative to the compiling machine and not relative to the  
target machine. You might want to check this.

In conclusion I would suggest you:
- check if your configure really work fine, and if the results are  
relative to the target machine
- check your configuration in Par_file
- recompile the mesher (make meshfem3D)
- run it
- check and copy the values_from_mesher.h file
- compile the solver

Hope this could be helpful ....
~petr


---------------------------------------------------------
Peter Danecek, PhD -- Ricercatore
Istituto Nazionale di Geofisica e Vulcanologia (INGV)
Sezione di Bologna
via Donato Creti, 12, I-40128 Bologna

tel: (+39) 051 4151-458
fax: (+39) 051 4151-499
---------------------------------------------------------