[CIG-SEISMO] specfem3d_globe with CUDA

Wed Apr 26 07:26:19 PDT 2017

Hi all

as far as I understand it should work with:

configure.... --with-cuda=cuda5

regards

Piero

Il 26/04/2017 16:04, Phil Cummins ha scritto:
> Hi Dimitri,
>
> Thanks for the kind words, but my approach was just a workaround. I 
> think someone smarter than me should try to fix the Makefile or 
> configure script. I'm still not sure what was wrong with it.
> Good luck!
>
> - Phil
>
>
>> Dimitri Komatitsch <mailto:komatitsch at lma.cnrs-mrs.fr>
>> 26 April 2017 at 8:54 PM
>>
>> Hi Phil, Hi all,
>>
>> Thanks! Let me cc Daniel Peter and Peter Messmer, who are very 
>> familiar with the CUDA version. Daniel, would you know how to commit 
>> Phil's fix to the official Git version of the code? (and I guess also 
>> for SPECFEM3D_Cartesian and for SPECFEM2D?).
>>
>> Thanks,
>> Best wishes,
>>
>> Dimitri.
>>
>>
>>
>> Phil Cummins <mailto:phil.cummins at anu.edu.au>
>> 23 April 2017 at 11:45 AM
>> Hello,
>>
>> I'm would like to use specfem3d_globe with my university's GPU 
>> cluster. I am trying to compile specfem3d_globe-7.0.0 after having 
>> configured it using:
>> ./configure --with-cuda=cuda6
>> I am using the ifort fortran compiler version 12.1.9.293, and the 
>> version of cuda I have loaded is 6.5.
>> The "make all" command does a lot of compilation using ifort and 
>> nvcc, but eventually fails when trying to link xspecfem3D itself with 
>> the error:
>>
>> ifort: error #10236: File not found: './obj/cuda_device_obj.o'
>> make: *** [bin/xspecfem3D] Error 1
>>
>> Delving into what happened during the make (see below), I noticed 
>> that after using nvcc to compile crust_mantle_impl_kernel_adjoint.cu, 
>> it attempts to use the command DUSE_OLDER_CUDA4_GPU to compile 
>> cuda_device_obj, which of course doesn't work because 
>> DUSE_OLDER_CUDA4_GPU is a compiler option not a command. It seems 
>> that make has somehow lost the beginning of this particular 
>> compilation command.
>> Has anyone else seen or know what to do about it?
>> Thanks,
>>
>> - Phil
>>
>> P.S. Some of the hardware on our cluster can be used only with cuda 
>> version 8. Does anyone know if specfem3d will work with cuda8?
>>
>> nvcc -c src/gpu/kernels.gen/crust_mantle_impl_kernel_adjoint.cu -o 
>> obj/crust_ma\
>> ntle_impl_kernel_adjoint.cuda-kernel.o   --cudart=shared 
>> -I/apps/openmpi/1.7.5/\
>> include -DUSE_OLDER_CUDA4_GPU 
>> -gencode=arch=compute_20,code=\"sm_20,compute_20\\
>> " -x cu -I./setup -I./src/gpu/kernels.gen  -DUSE_CUDA -include 
>> src/gpu/mesh_con\
>> stants_gpu.h
>> DUSE_OLDER_CUDA4_GPU 
>> -gencode=arch=compute_20,code=\"sm_20,compute_20\" -o obj/\
>> cuda_device_obj.o obj/assemble_MPI_scalar_gpu.cuda.o 
>> obj/assemble_MPI_vector_gp\
>> u.cuda.o obj/check_fields_gpu.cuda.o 
>> obj/compute_add_sources_elastic_gpu.cuda.o\
>>  obj/compute_coupling_gpu.cuda.o 
>> obj/compute_forces_crust_mantle_gpu.cuda.o obj\
>> /compute_forces_inner_core_gpu.cuda.o 
>> obj/compute_forces_outer_core_gpu.cuda.o \
>> obj/compute_kernels_gpu.cuda.o obj/compute_stacey_acoustic_gpu.cuda.o 
>> obj/compu\
>> te_stacey_elastic_gpu.cuda.o obj/compute_strain_gpu.cuda.o 
>> obj/helper_functions\
>> _gpu.cuda.o obj/initialize_gpu.cuda.o obj/noise_tomography_gpu.cuda.o 
>> obj/prepa\
>> re_mesh_constants_gpu.cuda.o obj/transfer_fields_gpu.cuda.o 
>> obj/update_displace\
>> ment_gpu.cuda.o obj/write_seismograms_gpu.cuda.o 
>> obj/save_and_compare_cpu_vs_gp\
>> u.cuda.o obj/assemble_boundary_accel_on_device.cuda-kernel.o 
>> obj/assemble_bound\
>> ary_potential_on_device.cuda-kernel.o 
>> obj/prepare_boundary_potential_on_device.\
>> cuda-kernel.o obj/prepare_boundary_accel_on_device.cuda-kernel.o 
>> obj/get_maximu\
>> m_scalar_kernel.cuda-kernel.o 
>> obj/get_maximum_vector_kernel.cuda-kernel.o obj/c\
>> ompute_add_sources_adjoint_kernel.cuda-kernel.o 
>> obj/compute_add_sources_kernel.\
>> cuda-kernel.o
>> nel.cuda-kernel.o 
>> obj/write_seismograms_transfer_strain_from_device_kernel.cuda\
>> -kernel.o obj/noise_transfer_surface_to_host_kernel.cuda-kernel.o 
>> obj/noise_add\
>> _source_master_rec_kernel.cuda-kernel.o 
>> obj/noise_add_surface_movie_kernel.cuda\
>> -kernel.o obj/compute_stacey_acoustic_kernel.cuda-kernel.o 
>> obj/compute_stacey_a\
>> coustic_backward_kernel.cuda-kernel.o 
>> obj/compute_stacey_elastic_kernel.cuda-ke\
>> rnel.o obj/compute_stacey_elastic_backward_kernel.cuda-kernel.o 
>> obj/update_disp\
>> _veloc_kernel.cuda-kernel.o obj/update_potential_kernel.cuda-kernel.o 
>> obj/updat\
>> e_accel_elastic_kernel.cuda-kernel.o 
>> obj/update_veloc_elastic_kernel.cuda-kerne\
>> l.o obj/update_accel_acoustic_kernel.cuda-kernel.o 
>> obj/update_veloc_acoustic_ke\
>> rnel.cuda-kernel.o obj/compute_rho_kernel.cuda-kernel.o 
>> obj/compute_iso_kernel.\
>> cuda-kernel.o obj/compute_ani_kernel.cuda-kernel.o 
>> obj/compute_hess_kernel.cuda\
>> -kernel.o obj/compute_acoustic_kernel.cuda-kernel.o 
>> obj/compute_strength_noise_\
>> kernel.cuda-kernel.o obj/compute_ani_undoatt_kernel.cuda-kernel.o 
>> obj/compute_i\
>> so_undoatt_kernel.cuda-kernel.o 
>> obj/compute_strain_kernel.cuda-kernel.o obj/out\
>> er_core_impl_kernel_forward.cuda-kernel.o 
>> obj/outer_core_impl_kernel_adjoint.cu\
>> da-kernel.o obj/inner_core_impl_kernel_forward.cuda-kernel.o 
>> obj/inner_core_imp\
>> l_kernel_adjoint.cuda-kernel.o 
>> obj/crust_mantle_impl_kernel_forward.cuda-kernel\
>> .o obj/crust_mantle_impl_kernel_adjoint.cuda-kernel.o
>> make: DUSE_OLDER_CUDA4_GPU: Command not found
>> make: [obj/cuda_device_obj.o] Error 127 (ignored)
>>
>> _______________________________________________
>> CIG-SEISMO mailing list
>> CIG-SEISMO at geodynamics.org
>> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo
>
> -- 
> Phil Cummins
> Prof. Natural Hazards
> Research School of Earth Sciences
> Australian National University
>
>
> _______________________________________________
> CIG-SEISMO mailing list
> CIG-SEISMO at geodynamics.org
> http://lists.geodynamics.org/cgi-bin/mailman/listinfo/cig-seismo

-- 
Piero Lanucara
CINECA
SuperComputing Applications and Innovation Department - SCAI
Phone: +39 06 44486709
Fax: +39 06 4957083
Address: CINECA Sede di Roma Via dei Tizii,6/b - 00185 Roma - Italy
E-mail: p.lanucara at cineca.it
skype: lpeter175
site: http://www.hpc.cineca.it

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.geodynamics.org/pipermail/cig-seismo/attachments/20170426/7246c076/attachment.html>