Charm++ parallel programming system

["Van Der Wijngaart, Rob F" <rob.f.van.der.wijngaart AT intel.com>]

That looks great, thanks, Nikhil! I'll try this out.

Rob

-----Original Message-----
From: Nikhil Jain 
[mailto:nikhil.life AT gmail.com]
 
Sent: Monday, September 22, 2014 1:58 PM
To: Van Der Wijngaart, Rob F
Cc: Miller, Philip B; Kunzman, David M; Hammond, Jeff R; Mattson, Timothy G; 
charm AT cs.illinois.edu
Subject: Re: [charm] Using icc/icpc with Charm

Hi Rob,

Sorry for the trouble you are going through. In order to better
understand which step was failing, I installed an intel-based MPI and
redid what we had been suggesting you.

Unfortunately, it seems there was a long standing bug in the
cc-mpicxx.sh files we had that was preventing the proposed solutions
from working properly. I have submitted a patch to be reviewed for the
same. Have a look:

http://charm.cs.illinois.edu/gerrit/#/c/374/

Using this patch, I tried following with success:

MPICXX=/dcsdata/home/nikhil/installs/opempi-intel/bin/mpicxx
CXX=/dcsdata/home/nikhil/installs/opempi-intel/bin/mpicxx
MPICC=/dcsdata/home/nikhil/installs/opempi-intel/bin/mpicc
CC=/dcsdata/home/nikhil/installs/opempi-intel/bin/mpicc ./build
charm++  mpi-linux-x86_64 mpicxx -j8

where openmpi is installed for me at
/dcsdata/home/nikhil/installs/opempi-intel/bin and I get following
version:
bin@finesse$
 ./mpicxx --version
icpc (ICC) 14.0.1 20131008
Copyright (C) 1985-2013 Intel Corporation.  All rights reserved.

When I compile examples/charm++/jacobi2d in this build, I see following:

jacobi2d@finesse$
 make OPTS=-verbose
../../../bin/charmc -verbose  jacobi2d.ci
Verbose mode set
Adding file jacobi2d.ci...
OpenMP support not enabled
charmc> ----------- Module dependency search complete ------------
charmc: Executing ../../../bin/charmxi jacobi2d.ci
charmc> No target object-- finished
../../../bin/charmc -verbose -c jacobi2d.C
Verbose mode set
Adding file jacobi2d.C...
OpenMP support not enabled
charmc> ----------- Module dependency search complete ------------
Compiling jacobi2d.C
charmc: Executing rm -f jacobi2d.o jacobi2d.f.o
charmc: Executing
/dcsdata/home/nikhil/installs/opempi-intel/bin/mpicxx -m64
-I../../../bin/../include -D__CHARMC__=1 -fno-stack-protector -c
jacobi2d.C -o jacobi2d.o
charmc> No target object-- finished
../../../bin/charmc -verbose -language charm++ -o jacobi2d jacobi2d.o
Verbose mode set
Adding file jacobi2d.o...
OpenMP support not enabled
charmc> ----------- Module dependency search complete ------------
charmc: Executing
/dcsdata/home/nikhil/installs/opempi-intel/bin/mpicxx -m64
-D__CHARMC__=1 -fno-stack-protector -c moduleinit27345.C -o
moduleinit27345.o
charmc> All libraries are: -L../../../bin/../lib
-I../../../bin/../include ../../../bin/../lib/libldb-rand.o jacobi2d.o
moduleinit27345.o ../../../bin/../lib/libmemory-default.o
../../../bin/../lib/libthreads-default.o -lckmain -lck -lconv-cplus-y
-lconv-core -lconv-util -lconv-partition -lckqt -ldl -lm
charmc: Executing
/dcsdata/home/nikhil/installs/opempi-intel/bin/mpicxx -m64 -rdynamic
-o jacobi2d -L../../../bin/../lib -I../../../bin/../include
../../../bin/../lib/libldb-rand.o jacobi2d.o moduleinit27345.o
../../../bin/../lib/libmemory-default.o
../../../bin/../lib/libthreads-default.o -lckmain -lck -lconv-cplus-y
-lconv-core -lconv-util -lconv-partition -lckqt -ldl -lm
charmc: Executing rm -f charmrun
charmc: Executing cp -p ../../../bin/charmrun charmrun
rm -f moduleinit27345.C moduleinit27345.o

Hope this helps.

--Nikhil

On Mon, Sep 22, 2014 at 2:39 PM, Van Der Wijngaart, Rob F
<rob.f.van.der.wijngaart AT intel.com>
 wrote:
> Maybe I should explain why I want to use the Intel compiler. It is not just
> company loyalty J. In the jacobi2d example, which I am morphing into the
> Stencil Parallel Research Kernel
> (https://github.com/tgmattso/ParResKernels), I replaced the
> multi-dimensional array allocation with a one-dimensional version, plus a
> macro (e.g. allocate temperature and use the following macro for array
> references: #define temp(i,j) temperature[j+LDA*i]). That usually gives a
> substantial performance improvement, but in this case I noticed a severe
> performance penalty (it was not caused by the wrong stride, I made sure that
> my access was equivalent to that in the original code). Then I wanted to
> look at the compiler optimization report (including vectorization) and also
> wanted to inform the compiler that there was no aliasing between pointers.
> It was at that time that I found out that I was using g++.
>
>
>
> Rob
>
>
>
> From: Van Der Wijngaart, Rob F
> Sent: Monday, September 22, 2014 9:45 AM
> To: 'Phil Miller'
> Cc: Nikhil Jain; 
> charm AT cs.illinois.edu
> Subject: RE: [charm] Using icc/icpc with Charm
>
>
>
> Thanks, Phil. I could track the following dependencies: conv-mach.sh is
> called by conv-config.sh, which is called by special.sh, which is called by
> the Makefile in src/scripts/Makefile. As far as I can tell, cc-mpicxx.sh is
> not called by anything. I may be able to reverse engineer the scripts, but
> could I ask you to give me a pointer how to tweak the build line so I can
> use the Intel compilers? I assume someone else has also tried to do that in
> the past. Thanks!
>
>
>
> Rob
>
>
>
> From: 
> unmobile AT gmail.com
>  
> [mailto:unmobile AT gmail.com]
>  On Behalf Of Phil
> Miller
> Sent: Friday, September 19, 2014 2:21 PM
> To: Van Der Wijngaart, Rob F
> Cc: Nikhil Jain; 
> charm AT cs.illinois.edu
>
>
> Subject: Re: [charm] Using icc/icpc with Charm
>
>
>
> Have a look at src/arch/mpi-linux-x86_64/conv-mach.sh and
> src/arch/mpi-linux-x86_64/cc-mpicxx.sh. Between those two, you should be
> able to tweak the commands and arguments used by charmc to generate the
> overall backend compiler command line.
>
>
>
> On Fri, Sep 19, 2014 at 4:17 PM, Van Der Wijngaart, Rob F
> <rob.f.van.der.wijngaart AT intel.com>
>  wrote:
>
> Hi Nikhil,
>
> Sadly, this did not work, even though mpiicpc is indeed installed and in my
> path. The problem is, of course, that I do not issue the compilation
> commands myself, charmc does, so I cannot influence how mpiicpc is invoked.
> Overall, the charmc interface if fairly obscuring. I did not know I was
> using g++ behind the scenes, until I specified some compiler flags in my
> makefile that are Intel compiler specific.
>
> Rob
>
> ../../../bin/charmc -O3 -fno-alias  jacobi2d.ci
> ../../../bin/charmc -O3 -fno-alias -c jacobi2d.C
> g++: error: unrecognized command line option "-fno-alias"
>
> -----Original Message-----
> From: 
> nikhil.life AT gmail.com
>  
> [mailto:nikhil.life AT gmail.com]
>  On Behalf Of
> Nikhil Jain
> Sent: Thursday, September 18, 2014 5:11 PM
> To: Van Der Wijngaart, Rob F
> Cc: Phil Miller 
> (mille121 AT illinois.edu);
>  
> charm AT cs.illinois.edu
> Subject: Re: [charm] Using icc/icpc with Charm
>
> Jumping in to suggest what I think could be done.
>
> Do you have an mpi* wrappers that default to Intel C/C++ compilers? If
> yes, the following can be done:
>
> MPICXX=mpiicpc CXX=icpc ./build charm++  mpi-linux-x86_64 mpicxx ifort
> smp --with-production -j8 -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX
>
> assuming the mpi* wrapper that default to ICC is named mpiicpc, else
> it can be replaced with the right name for you. You can similarly also
> prefix MPICC= and CC= to let Charm pick the right names for C code.
>
> --Nikhil
>
> On Thu, Sep 18, 2014 at 6:28 PM, Van Der Wijngaart, Rob F
> <rob.f.van.der.wijngaart AT intel.com>
>  wrote:
>> Hi Phil,
>>
>>
>>
>> Do you know how I should modify my build line to be able to use the Intel
>> C/C++ compilers, in addition to mpicxx? This is what I have today:
>>
>> ./build charm++ mpi-linux-x86_64 ifort mpicxx smp -j8
>> -DMPICH_IGNORE_CXX_SEEK -DMPICH_SKIP_MPICXX
>>
>>
>>
>> Thanks!
>>
>>
>>
>> Rob
>>
>>
>> _______________________________________________
>> charm mailing list
>> charm AT cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/charm
>>
>
>
>
> --
> Nikhil Jain, 
> nikhil.jain AT acm.org,
>  http://charm.cs.uiuc.edu/people/nikhil
> Doctoral Candidate @ CS, UIUC
>
>



-- 
"Success may ditch you most of the times, but if you ditch it once :
your job is done."
Nikhil Jain, 
nikhil.life AT gmail.com,
 +1-217-979-0918