NVCC does not define USE_MPI when compiling with omp+mpi+cuda
Created by: cypox
Branch
Occurs in branch gpu-hackathlon
Explanation The cuda compier is not seeing the USE_MPI flag when compiling ".cu" files. This raises a serious issue when using cuda+mpi. The ParticlesData struct compiled with mpic++ is different from the one compiled with nvcc. This is not shown as error nor as warning during compilation but is problematic when I added the DeviceParticleData.
How to reproduce?
-
Add:
printf("[DEBUG] -- sizeof: %ld\n", sizeof(Dataset));
To file: density.hpp At line: 109 (beforecuda::computeDensity<T>(l, taskList, d);
) -
Add:
printf("[DEBUG] -- sizeof: %ld\n", sizeof(ParticlesData));
To file: cudaDensity.cu At line: 53 (beforeconst int maz = d.bbox.PBCz ? 2 : 0;
) -
While we should get similar results (the two structures should be the same), we get different sized because the variables declared in ParticlesData.cpp after the USE_MPI definition are not included in the struct passed to the cuda file.
How to reproduce 2?
Just add:
#ifdef USE_MPI #warning "BROOKS WAS HERE" #endif
To one of the *.cu files and compile with "mpi+omp+cuda".