[CIG-SEISMO] specfem3d_globe with CUDA

[Phil Cummins]

Hello,

I'm would like to use specfem3d_globe with my university's GPU cluster. 
I am trying to compile specfem3d_globe-7.0.0 after having configured it 
using:
./configure --with-cuda=cuda6
I am using the ifort fortran compiler version 12.1.9.293, and the 
version of cuda I have loaded is 6.5.
The "make all" command does a lot of compilation using ifort and nvcc, 
but eventually fails when trying to link xspecfem3D itself with the error:

ifort: error #10236: File not found:  './obj/cuda_device_obj.o'
make: *** [bin/xspecfem3D] Error 1

Delving into what happened during the make (see below), I noticed that 
after using nvcc to compile crust_mantle_impl_kernel_adjoint.cu, it 
attempts to use the command DUSE_OLDER_CUDA4_GPU to compile 
cuda_device_obj, which of course doesn't work because 
DUSE_OLDER_CUDA4_GPU is a compiler option not a command. It seems that 
make has somehow lost the beginning of this particular compilation command.
Has anyone else seen or know what to do about it?
Thanks,

- Phil

P.S. Some of the hardware on our cluster can be used only with cuda 
version 8. Does anyone know if specfem3d will work with cuda8?

nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o 
obj/crust_ma\
ntle_impl_kernel_adjoint.cuda-kernel.o   --cudart=shared 
-I/apps/openmpi/1.7.5/\
include -DUSE_OLDER_CUDA4_GPU 
-gencode=arch=compute_20,code=\"sm_20,compute_20\\
" -x cu -I./setup -I./src/gpu/kernels.gen  -DUSE_CUDA -include 
src/gpu/mesh_con\
stants_gpu.h
DUSE_OLDER_CUDA4_GPU -gencode=arch=compute_20,code=\"sm_20,compute_20\" 
-o obj/\
cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o 
obj/assemble_MPI_vector_gp\
u.cuda.o obj/check_fields_gpu.cuda.o 
obj/compute_add_sources_elastic_gpu.cuda.o\
  obj/compute_coupling_gpu.cuda.o 
obj/compute_forces_crust_mantle_gpu.cuda.o obj\
/compute_forces_inner_core_gpu.cuda.o 
obj/compute_forces_outer_core_gpu.cuda.o \
obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o 
obj/compu\
te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o 
obj/helper_functions\
_gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o 
obj/prepa\
re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o 
obj/update_displace\
ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o 
obj/save_and_compare_cpu_vs_gp\
u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o 
obj/assemble_bound\
ary_potential_on_device.cuda-kernel.o 
obj/prepare_boundary_potential_on_device.\
cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o 
obj/get_maximu\
m_scalar_kernel.cuda-kernel.o 
obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\
ompute_add_sources_adjoint_kernel.cuda-kernel.o 
obj/compute_add_sources_kernel.\
cuda-kernel.o
nel.cuda-kernel.o 
obj/write_seismograms_transfer_strain_from_device_kernel.cuda\
-kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o 
obj/noise_add\
_source_master_rec_kernel.cuda-kernel.o 
obj/noise_add_surface_movie_kernel.cuda\
-kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o 
obj/compute_stacey_a\
coustic_backward_kernel.cuda-kernel.o 
obj/compute_stacey_elastic_kernel.cuda-ke\
rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o 
obj/update_disp\
_veloc_kernel.cuda-kernel.o obj/update_potential_kernel.cuda-kernel.o 
obj/updat\
e_accel_elastic_kernel.cuda-kernel.o 
obj/update_veloc_elastic_kernel.cuda-kerne\
l.o obj/update_accel_acoustic_kernel.cuda-kernel.o 
obj/update_veloc_acoustic_ke\
rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o 
obj/compute_iso_kernel.\
cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o 
obj/compute_hess_kernel.cuda\
-kernel.o obj/compute_acoustic_kernel.cuda-kernel.o 
obj/compute_strength_noise_\
kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o 
obj/compute_i\
so_undoatt_kernel.cuda-kernel.o obj/compute_strain_kernel.cuda-kernel.o 
obj/out\
er_core_impl_kernel_forward.cuda-kernel.o 
obj/outer_core_impl_kernel_adjoint.cu\
da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o 
obj/inner_core_imp\
l_kernel_adjoint.cuda-kernel.o 
obj/crust_mantle_impl_kernel_forward.cuda-kernel\
.o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o
make: DUSE_OLDER_CUDA4_GPU: Command not found
make: [obj/cuda_device_obj.o] Error 127 (ignored)

-- 

Phil Cummins
Prof. Natural Hazards
Research School of Earth Sciences
Australian National University

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.geodynamics.org/pipermail/cig-seismo/attachments/20170423/ab030559/attachment.html>