As the number of data elements in the application data gets bigger we're moving to use the ilp64 MKL routines.
I used the Intel Link Line Advisor to specify the required link libraries. Based on this advice, if I build and link an application thus:
icc -DMKL_ILP64 -w testlapack.cpp -L${MKLROOT}/lib/intel64 -lmkl_intel_ilp64 -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -lm -o testlapackp
and use ldd -v to examine what libraries are linked with the executable I find as expected:
linux-vdso.so.1 => (0x00007fffb8bff000)
libmkl_intel_ilp64.so => /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_ilp64.so (0x00007f111c837000)
libmkl_intel_thread.so => /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_thread.so (0x00007f111b878000)
libmkl_core.so => /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_core.so (0x00007f111a1ba000)
libiomp5.so => /opt/intel/composer_xe_2013_sp1.1.106/compiler/lib/intel64/libiomp5.so (0x00007f1119e9f000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1119c76000)
libm.so.6 => /lib64/libm.so.6 (0x00007f11199f2000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007f11196ec000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f11194d5000)
libc.so.6 => /lib64/libc.so.6 (0x00007f1119142000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f1118f3e000)
/lib64/ld-linux-x86-64.so.2 (0x00007f111cf53000)
ie the ilp64 library (highlighted) is properly linked. With > 2**31 array elements the application builds, runs and, parallelises as expected.
If I try to simplify the link line a bit using:
icc -DMKL_ILP64 -w testlapack.cpp -L${MKLROOT}/lib/intel64 -lmkl_intel_ilp64 -mkl=parallel -lm -o testlapackp
and examine the executable with ldd -v I get:
linux-vdso.so.1 => (0x00007fff1edff000)
libmkl_intel_ilp64.so => /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_ilp64.so (0x00007ff92cda4000)
libmkl_intel_lp64.so => /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_lp64.so (0x00007ff92c65f000)
libmkl_intel_thread.so => /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_intel_thread.so (0x00007ff92b6a1000)
libmkl_core.so => /opt/intel/composer_xe_2013_sp1.1.106/mkl/lib/intel64/libmkl_core.so (0x00007ff929fe3000)
libiomp5.so => /opt/intel/composer_xe_2013_sp1.1.106/compiler/lib/intel64/libiomp5.so (0x00007ff929cc7000)
libm.so.6 => /lib64/libm.so.6 (0x00007ff929a38000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00007ff929732000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007ff92951b000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ff9292fe000)
libc.so.6 => /lib64/libc.so.6 (0x00007ff928f6b000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007ff928d66000)
/lib64/ld-linux-x86-64.so.2 (0x00007ff92d4c0000)
ie both the ilp64 and lp64 libraries.
The application built with this linkline executes, parallelises etc OK with an array containing 2.5G elements (50k x 50k) but my question is why is the lp64 library being linked?
Thanks
David