[CIG-SEISMO] BG/P support for SPECFEM3d
Dimitri Komatitsch
dimitri.komatitsch at univ-pau.fr
Mon Mar 1 02:50:11 PST 2010
Dear all,
I agree, the problem comes from the fact that the code is compiled
on a frontend machine that cannot run the executable code, and we need
"xcreate_header_file" to be able to run on the machine on which we compile.
I see two options:
1/ if the xlf compiler is also installed on the Blue Gene nodes,
create a script that runs "make all" on one node instead of on the frontend
2/ otherwise, this will work:
- create two copies of the Makefile:
mv Makefile Makefile1
cp Makefile1 Makefile2
In Makefile1 put this instead of the current values:
FLAGS_CHECK = -O0
FLAGS_NO_CHECK = -O0
and replace
create_header_file: $O/program_create_header_file.o $(LIBSPECFEM)
${FCCOMPILE_CHECK} -o xcreate_header_file
$O/program_create_header_file.o $(LIBSPECFEM)
with
create_header_file: $O/program_create_header_file.o $O/exit_mpi.o
$(LIBSPECFEM)
${MPIFCCOMPILE_CHECK} -o xcreate_header_file
$O/program_create_header_file.o $O/exit_mpi.o $(LIBSPECFEM)
In Makefile2 comment out these two lines:
#OUTPUT_FILES/values_from_mesher.h: xcreate_header_file
# ./xcreate_header_file
Then:
make -f Makefile1 clean
make -f Makefile1 create_header_file
./xcreate_header_file
make -f Makefile2 clean
make -f Makefile2 meshfem3D
make -f Makefile2 specfem3D
should work.
I am going to cut and paste this in the user manual.
(or maybe change the "configure" script)
Best regards,
Dimitri.
Peter Danecek wrote:
> On 27 Feb 2010, at 09:37, Mark Cheeseman wrote:
>
>> Sorry, I forgot to include the error messages I receive:
>>
>> When I first try to compile with the IBM XLF90 compiler, I first get
>> the error message about the xcreate_header_file binary
>>
>> 1501-510 Compilation successful for file exit_mpi.f90.
>> mpixlf90 -O3 -qsave -qstrict -qtune=450 -qarch=450d -qfree=f90 -
>> qsuffix=f=f90 -qhalt=e -qsigtrap -
>> qinitauto
>> =7Fswap_all,mxm_m1_m2_5points,mxm_m1_m1_5points,mxm_m2_m1_5points -C
>> -qddim -qfloat=nans -qfullpath -o xmeshfem3D j/exit_mpi.o obj/
>> libspecfem.a
>> mkdir -p OUTPUT_FILES
>> ./xcreate_header_file
>> make: *** [OUTPUT_FILES/values_from_mesher.h] Illegal instruction
>>
>> Running the ./xcreate_header_file seems to work fine with the
>> required header file being generated in the OUTPUT_FILES
>> subdirectory. Next, I comment out the call to the
>> xcreate_header_file binary in the main Makefile and rerun make. The
>> result is a large number of error messages about zero-sized arrays
>> in the following source code files
>>
>> compute_forces_crust_mantle.f90
>> compute_forces_inner_core.f90
>> compute_forces_outer_core.f90
>> write_movie_volume.f90
>>
>> with the error messages being
>> .
>> .
>> "./write_movie_volume.f90", line 425.14: 1511-013 (W) The value of
>> the DO-loop increment should be negative when initial value is
>> greater than the terminal value.
>> "./write_movie_volume.f90", line 429.12: 1516-152 (S) Zero-sized
>> arrays must not be subscripted.
>> "./write_movie_volume.f90", line 431.18: 1516-152 (S) Zero-sized
>> arrays must not be subscripted.
>> "./write_movie_volume.f90", line 432.28: 1516-152 (S) Zero-sized
>> arrays must not be subscripted.
>> ** write_movie_volume_vector === End of Compilation 4 ===
>> "./write_movie_volume.f90", line 495.16: 1511-013 (W) The value of
>> the DO-loop increment should be negative when initial value is
>> greater than the terminal value.
>> "./write_movie_volume.f90", line 502.52: 1516-152 (S) Zero-sized
>> arrays must not be subscripted.
>> ** write_movie_volume_divcurl === End of Compilation 5 ===
>> 1501-511 Compilation failed for file write_movie_volume.f90.
>> .
>> .
>> .
>> The actual offending parameters (after some tracking) are those
>> defined in the header file
>>
>> ./OUTPUT_FILES/values_from_mesher.h
>>
>> and include
>>
>> NSPEC_CRUST_MANTLE, NSPEC_OUTER_CORE, NSPEC_INNER_CORE
>>
>> It looks like I am missing an important configuration step or
>> datafile. Any assistance would be greatly appreciated.
>>
>
>
> Hi Mark,
>
> The xcreate_header_file executable is used to create some header file
> (values_from_mesher.h) which otherwise would be created by the mesher
> exec. But in contrast to the mesher itself, it does not need to be run
> in parallel though the queuing system. Instead xcreate_header_file
> calculates/estimates the size of several arrays immediately and can be
> executed during compilation process.
>
> From what you are writing, i would guess as well that the problem
> arrises from the cross compiler environment. The xcreate_header_file
> binary is created for the target machine, but executed on the
> compiling machine itself. That's why the makefile fails. So you could
> try two things:
>
> (1) compile the xcreate_header_file exec for the local compiling
> machine (not with the cross compiler) to generate the
> values_from_mesher.h header file during compilation. You probably need
> to compile some dependences too and it is probably not straight
> forward to get the Makefile working properly.
> NOTE: I am not very sure to which extend architectural differences
> will influences the resulting estimates in values_from_mesher.h file.
> So be careful with this approach.
>
> (2) use the more conservative way: First (cross-)compile the mesher
> executable only (make target: meshfem3D) and run it through the queue
> system. Than use the generated values_from_mesher.h file to compile
> the solver (make target: specfem3D) .
>
> Running xcreate_header_file through the queue system (serial job) --
> as I understand you have tried -- might also work, though I do not see
> any benefits with respect to approach (2). From the error messages,
> however, it looks like the generated values_from_mesher.h is not
> valid or incomplete. Check if the content looks reasonable. (does the
> execution really complete correctly, is your Par_file reasonable)
>
> An other problem might arrise from running the ./configure script (on
> the compiling machine, i suppose). To my understanding this should
> fail for the same reason the xcreate_... binary does, or at best give
> results relative to the compiling machine and not relative to the
> target machine. You might want to check this.
>
> In conclusion I would suggest you:
> - check if your configure really work fine, and if the results are
> relative to the target machine
> - check your configuration in Par_file
> - recompile the mesher (make meshfem3D)
> - run it
> - check and copy the values_from_mesher.h file
> - compile the solver
>
> Hope this could be helpful ....
> ~petr
>
>
> ---------------------------------------------------------
> Peter Danecek, PhD -- Ricercatore
> Istituto Nazionale di Geofisica e Vulcanologia (INGV)
> Sezione di Bologna
> via Donato Creti, 12, I-40128 Bologna
>
> tel: (+39) 051 4151-458
> fax: (+39) 051 4151-499
> ---------------------------------------------------------
>
> _______________________________________________
> CIG-SEISMO mailing list
> CIG-SEISMO at geodynamics.org
> http://geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo
>
--
Dimitri Komatitsch - dimitri.komatitsch aT univ-pau.fr
Professor, University of Pau, Institut universitaire de France,
CNRS and INRIA Magique3D, France http://www.univ-pau.fr/~dkomati1
More information about the CIG-SEISMO
mailing list