Charm++ parallel programming system

["Swartz, Brent A. (GSFC-610.3)[ADVANCED MANAGEMENT TECHNOLOGY]" <brent.a.swartz AT nasa.gov>]

Title: Re: [charm] Charm 6.2.2 build

Hi Jim,

It is not what I expected!:

baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort/tests/charm++/megatest> mpicxx -V
g++: couldn't run 'x86_64-suse-linux-gcc--I/usr/local/intel/mpi/3.2.2.006/include64': No such file or directory
baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort/tests/charm++/megatest> which mpicxx
/usr/local/intel/mpi/3.2.2.006/bin64/mpicxx

Apparently mpicxx is defaulting to g++. So I changed the modules I have loaded:

baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2> module load comp/intel-10.1.023 lib/mkl-10.1.2.024 mpi/impi-3.1.038
baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2> !mpi
mpicxx -V
g++: couldn't run 'x86_64-suse-linux-gcc--I/usr/local/intel/mpi/3.1.038/include64': No such file or directory
baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2> !whi
which mpicxx
/usr/local/intel/mpi/3.1.038/bin64/mpicxx

But it still defaults to g++. So I set the relevant env var to make it choose icc:

baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2> env I_MPI_CXX=icc mpicxx -V
Intel(R) C Compiler for applications running on Intel(R) 64, Version 10.1    Build 20090615 Package ID: l_cc_p_10.1.023
Copyright (C) 1985-2009 Intel Corporation.  All rights reserved.

Thus I rebuilt charm++:

baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2> env I_MPI_CXX=icc env MPICXX=mpicxx ./build charm++ mpi-linux-x86_64 ifort --no-build-shared --with-production >& build.out.5

In build.out.5:

gmake[1]: Entering directory `/gpfsm/dnb33/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort/tmp'
./configure
checking machine name... mpi-linux-x86_64-ifort
checking "cp command as"... cp -p
checking "C++ compiler as"... "mpicxx       -optimize  -nobs"
checking "whether C++ compiler works"... "ok"
checking "C++ linker as"... "mpicxx      "
checking "whether linker works"... "no"
Cannot link C++ programs with mpicxx
 (check your charm++ version)
*** Please find detailed output in charmconfig.out ***
gmake[1]: *** [conv-autoconfig.h] Error 1
gmake[1]: Leaving directory `/gpfsm/dnb33/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort/tmp'
gmake: *** [headers] Error 2
-------------------------------------------------
Charm++ NOT BUILT. Either cd into mpi-linux-x86_64-ifort/tmp and try
to resolve the problems yourself, visit
http://charm.cs.uiuc.edu/
for more information. Otherwise, email the developers at charm AT cs.uiuc.edu

So looking at mpi-linux-ifort-icc/tmp/charmconfig.out we have:

### whether ucontext uses uc_regs
#include <ucontext.h>

int main()
{
  ucontext_t context;
  context.uc_mcontext.uc_regs = 0;
} 
icpc -fpic -I../include -I. -c test.cpp -o test.o
test.cpp(6): error: class "mcontext_t" has no member "uc_regs"
    context.uc_mcontext.uc_regs = 0;
                        ^

compilation aborted for test.cpp (code 2)

So it appears it cannot find ucontext.h – Where is this normally found?
I could not find it in the charm subdir.

- Brent

On 9/28/10 3:32 PM, "Jim Phillips" <jim AT ks.uiuc.edu> wrote:

> Hi again,
>
> What does "mpicxx -V" give you?  It should be something like this:
>
> [jphillip@honest2 ~]$ mpicxx -V
> Intel(R) C++ Compiler for applications running on Intel(R) 64, Version
> 10.1    Build 20080602 Package ID: l_cc_p_10.1.017
> Copyright (C) 1985-2008 Intel Corporation.  All rights reserved.
>
> -Jim
>
>
> On Tue, 28 Sep 2010, Swartz, Brent A. (GSFC-610.3)[ADVANCED MANAGEMENT TECHNOLOGY] wrote:
>
> > Hi Phil et al,
> >
> > I successfully rebuilt using:
> >
> > baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2>
> > env MPICXX=mpicxx ./build charm++ mpi-linux-x86_64 ifort
> > --no-build-shared --with-production >& build.out.4
> >
> > But when I compile the megatest pgm I get:
> >
> > baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort/tests/charm++/megatest>
> > make pgm >& make.out.1
> >
> > ../../../bin/charmc  -cxx=icc -o pgm  megatest.o groupring.o nodering.o varsizetest.o varraystest.o groupcast.o groupmulti.o groupsectiontest.o multisectiontest.o nodecast.o synctest.o fib.o arrayring.o tempotest.o packtest.o queens.o migration.o marshall.o priomsg.o priotest.o rotest.o statistics.o templates.o inherit.o reduction.o bitvector.o immediatering.o callback.o inlineem.o  -language charm++ -lguide -cxx=icc
> > megatest.o:(.gnu.linkonce.d._ZTI4main[.gnu.linkonce.d._ZTI4main]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
> > megatest.o:(.gnu.linkonce.d.DW.ref.__gxx_personality_v0+0x0): undefined reference to `__gxx_personality_v0'
> > groupring.o: In function `groupring_group::~groupring_group()':
> > groupring.C:(.gnu.linkonce.t._ZN15groupring_groupD1Ev[.gnu.linkonce.t._ZN15groupring_groupD1Ev]+0x4a): undefined reference to `operator delete[](void*)'
> > groupring.C:(.gnu.linkonce.t._ZN15groupring_groupD1Ev[.gnu.linkonce.t._ZN15groupring_groupD1Ev]+0xf3): undefined reference to `operator delete[](void*)'
> > groupring.C:(.gnu.linkonce.t._ZN15groupring_groupD1Ev[.gnu.linkonce.t._ZN15groupring_groupD1Ev]+0x18e): undefined reference to `operator delete[](void*)'
> >
> > Then I edited the Makefile to remove all -cxx=icc references. Then I get:
> >
> > ../../../bin/charmc   -o inlineem.o inlineem.C
> > ../../../bin/charmc   -o pgm  megatest.o groupring.o nodering.o varsizetest.o varraystest.o groupcast.o groupmulti.o groupsectiontest.o multisectiontest.o nodecast.o synctest.o fib.o arrayring.o tempotest.o packtest.o queens.o migration.o marshall.o priomsg.o priotest.o rotest.o statistics.o templates.o inherit.o reduction.o bitvector.o immediatering.o callback.o inlineem.o  -language charm++ -lguide
> > ../../../bin/../lib/libthreads-default.o: In function `CthRegister':
> > threads.c:(.text+0xdb): undefined reference to `_intel_fast_memset'
> > ../../../bin/../lib/libthreads-default.o: In function `CthRegistered':
> > threads.c:(.text+0x16b): undefined reference to `_intel_fast_memset'
> > ../../../bin/../lib/libthreads-default.o: In function `CthSwitchThread':
> > threads.c:(.text+0x26a): undefined reference to `_intel_fast_memset'
> > ../../../bin/../lib/libthreads-default.o: In function `CthBaseResume':
> > threads.c:(.text+0x4da): undefined reference to `_intel_fast_memset'
> > ...
> > ../../../bin/../lib/libconv-core.a(debug-conv.o): In function `CpdDebugCallAllocationTree':
> > debug-conv.c:(.text+0x459): undefined reference to `__intel_sse2_strlen'
> > ../../../bin/../lib/libconv-core.a(debug-conv.o): In function `CpdDebugCallMemStat':
> > debug-conv.c:(.text+0x788): undefined reference to `__intel_sse2_strlen'
> > ...
> >
> > This is the problem I saw before. I thought these routines were in the -lguide compiler lib, as  I can see then in the nm output:
> >
> > baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort/tests/charm++/megatest> nm /usr/local/intel/Compiler/11.1/072/lib/intel64/libguide.a > guide.nm
> >
> > fastmemset.o:
> >                 U __kmp_external___intel_cpu_indicator
> >                 U __kmp_external___intel_cpu_indicator_init
> > 0000000000000000 T __kmp_external__intel_fast_memset
> > 0000000000000036 T __kmp_external__intel_fast_memset.A
> > 000000000000002e T __kmp_external__intel_fast_memset.J
> >                 U __kmp_external___intel_new_memset
> >                 U memset
> >
> > Although these symbols are not exactly _intel_fast_memset ... They look more like pthreads references.
> > I get the same undefined references with and without the -lguide.
> > So I guess the question here is what libs do I need to include to satisfy these undefined references?
> > Or should I build charm with different options (icc)?
> >
> > Thanks,
> >
> > - Brent
> >
> > On 9/28/10 11:18 AM, "Jim Phillips" <jim AT ks.uiuc.edu> wrote:
> >
> >
> >
> > Adding -cxx=icc is probably what broke it.  The mpi builds need to use
> > mpicc and mpicxx to get all of the libraries correctly.  Your mpicc and
> > mpicxx scripts need to use icc as the base compiler.
> >
> > The NAMD build uses icc directly and calls charmc to link, but everything
> > in the charm tree just uses charmc (and in this case mpicc/mpicxx).
> >
> > -Jim
> >
> >
> > On Tue, 28 Sep 2010, Swartz, Brent A. (GSFC-610.3)[ADVANCED MANAGEMENT
> > TECHNOLOGY] wrote:
> >
> >> Hi Phil,
> >>
> >> Yes I originally tried the mpi-linux-x86_64 build target, with and
> >> without icc and ifort, and charm++ built, but when I attempt to compile
> >> the megatest sample code (I added the -cxx=icc to the Makefile) I get:
> >>
> >> baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-mpicxx/tests/charm++/megatest>
> >> make pgm
> >>
> >> ...
> >> ../../../bin/charmc  -cxx=icc -c inlineem.ci
> >> ../../../bin/charmc  -cxx=icc -o inlineem.o inlineem.C
> >> icpc: command line warning #10156: ignoring option '-c'; no argument required
> >> ../../../bin/charmc  -cxx=icc -o pgm  megatest.o groupring.o nodering.o varsizetest.o varraystest.o groupcast.o groupmulti.o groupsectiontest.o multisectiontest.o nodecast.o synctest.o fib.o arrayring.o tempotest.o packtest.o queens.o migration.o marshall.o priomsg.o priotest.o rotest.o statistics.o templates.o inherit.o reduction.o bitvector.o immediatering.o callback.o inlineem.o  -language charm++ -lguide -cxx=icc
> >> Fatal Error by charmc in directory /discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/tests/charm++/megatest
> >>   charmc : No such thread mode default
> >> charmc exiting...
> >> make: *** [pgm] Error 1
> >>
> >> Prior to that I was getting:
> >> baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort.old/tests/charm++/megatest> make pgm
> >> ../../../bin/charmc  -cxx=icc -o pgm  megatest.o groupring.o nodering.o varsizetest.o varraystest.o groupcast.o groupmulti.o groupsectiontest.o multisectiontest.o nodecast.o synctest.o fib.o arrayring.o tempotest.o packtest.o queens.o migration.o marshall.o priomsg.o priotest.o rotest.o statistics.o templates.o inherit.o reduction.o bitvector.o immediatering.o callback.o inlineem.o  -language charm++ -lguide -cxx=icc
> >> megatest.o:(.gnu.linkonce.d._ZTI4main[.gnu.linkonce.d._ZTI4main]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
> >> megatest.o:(.gnu.linkonce.d.DW.ref.__gxx_personality_v0+0x0): undefined reference to `__gxx_personality_v0'
> >> groupring.o: In function `groupring_group::~groupring_group()':
> >> groupring.C:(.gnu.linkonce.t._ZN15groupring_groupD1Ev[.gnu.linkonce.t._ZN15groupring_groupD1Ev]+0x4a): undefined reference to `operator delete[](void*)'
> >> groupring.C:(.gnu.linkonce.t._ZN15groupring_groupD1Ev[.gnu.linkonce.t._ZN15groupring_groupD1Ev]+0xf3): undefined reference to `operator delete[](void*)'
> >> ... (More similar undefined references)
> >>
> >> Also, while building charm++ I got the following error:
> >>
> >> ../bin/charmc  -optimize  -nobs -I. -o ../lib/libmemory-charmdebug.o -DCMK_MEMORY_BUILD_CHARMDEBUG memory.c || touch ../lib/libmemory-charmdebug.o
> >> memory-charmdebug.c(388): warning #181: argument is incompatible with corresponding format string conversion
> >>    printf("scanning stack from %p (%d) to %p (%d)\n",begin_stack,begin_stack,end_stack,end_stack);
> >>                                                                  ^
> >>
> >> memory-charmdebug.c(388): warning #181: argument is incompatible with corresponding format string conversion
> >>    printf("scanning stack from %p (%d) to %p (%d)\n",begin_stack,begin_stack,end_stack,end_stack);
> >>                                                                                        ^
> >>
> >> memory-charmdebug.c(401): warning #181: argument is incompatible with corresponding format string conversion
> >>    printf("scanning data from %p (%d) to %p (%d)\n",info->begin_data,info->begin_data,info->end_data,info->end_data);
> >>                                                                      ^
> >>
> >> memory-charmdebug.c(401): warning #181: argument is incompatible with corresponding format string conversion
> >>    printf("scanning data from %p (%d) to %p (%d)\n",info->begin_data,info->begin_data,info->end_data,info->end_data);
> >>                                                                                                      ^
> >>
> >> memory-charmdebug.c(416): warning #181: argument is incompatible with corresponding format string conversion
> >>    printf("scanning bss from %p (%d) to %p (%d)\n",info->begin_bss,info->begin_bss,info->end_bss,info->end_bss);
> >>                                                                    ^
> >>
> >> memory-charmdebug.c(416): warning #181: argument is incompatible with corresponding format string conversion
> >>    printf("scanning bss from %p (%d) to %p (%d)\n",info->begin_bss,info->begin_bss,info->end_bss,info->end_bss);
> >>                                                                                                  ^
> >>
> >> memory-charmdebug.c(1321): warning #181: argument is incompatible with corresponding format string conversion
> >>    sprintf(buf,"slot size %d\n",sizeof(Slot));
> >>                                 ^
> >>
> >> memory-charmdebug.c(1542): error: expected an identifier
> >>  void setMemoryTypeChare(void *ptr) {
> >>                                     ^
> >>
> >> memory-charmdebug.c(1542): error: expected a ";"
> >>  void setMemoryTypeChare(void *ptr) {
> >>                                     ^
> >>
> >> memory-isomalloc.h(17): warning #12: parsing restarts here after previous syntax error
> >>  void *malloc_nomigrate(size_t size);
> >>                                     ^
> >>
> >> compilation aborted for memory.c (code 2)
> >> Fatal Error by charmc in directory /discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort/tmp
> >>   Command mpicc -m64 -fPIC -I../bin/../include -D__CHARMC__=1 -I. -DCMK_MEMORY_BUILD_CHARMDEBUG -O -c memory.c -o ../lib/libmemory-charmdebug.o returned error code 2
> >> charmc exiting...
> >>
> >> However charm++ does build successfully despite these errors.
> >>
> >> Cheers,
> >>
> >> - Brent
> >>
> >>
> >> On 9/27/10 5:36 PM, "Phil Miller" <mille121 AT illinois.edu> wrote:
> >>
> >> On Mon, Sep 27, 2010 at 17:11, Swartz, Brent A. (GSFC-610.3)[ADVANCED
> >> MANAGEMENT TECHNOLOGY] <brent.a.swartz AT nasa.gov> wrote:
> >>> In order to install the latest NAMD on our cluster here I am attempting to
> >>> build charm for our Intel Xeon EM64T processor (Nehamlem, Harpertown,
> >>> Woodcrest, Dempsey) cluster running SUSE Linux here at NASA GSFC. I want to
> >>> use the Intel compilers (icc, ifort) and Intel MPI to communicate between
> >>> the nodes.  So I have the following modules loaded on our system:
> >>>
> >>> module load comp/intel-11.1.072 mpi/impi-3.2.2.006
> >>>
> >>> baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2
> >>> .7b4_Source/charm-6.2.2> uname -a
> >>> Linux discover08 2.6.16.60-0.42.5-smp #1 SMP Mon Aug 24 09:41:41 UTC 2009
> >>> x86_64 x86_64 x86_64 GNU/Linux
> >>>
> >>> Then I attempt to build charm:
> >>>
> >>> env MPICXX=mpicxx ./build charm++ mpi-linux icc ifort --no-build-shared
> >>> --with-production >& build.out.3
> >>>
> >>> If you like I can email you the entire build.out.3 file, but the main
> >>> problems appear to be:
> >>>
> >>> [snip irrelevant parts]
> >>>
> >>> Running configure on i386-gcc
> >>> ( cd QuickThreads ; /usr/bin/gmake qt  OPTS='-optimize  -nobs' ) || ( echo
> >>> "Did you try to build net-linux on a 64-bit system? Try net-linux-x86_64" &&
> >>> false )
> >>> gmake[1]: Entering directory
> >>> `/gpfsm/dnb33/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/m
> >>> pi-linux-ifort-icc/tmp/QuickThreads'
> >>> rm -f qtmd.h
> >>> ln -s md/i386.h qtmd.h
> >>> ../../bin/charmc -seq -I. -o qt.o qt.c
> >>> as -o qtmds.o md/i386.s
> >>> md/i386.s: Assembler messages:
> >>> md/i386.s:69: Error: suffix or operands invalid for `push'
> >>> md/i386.s:70: Error: suffix or operands invalid for `push'
> >>> md/i386.s:71: Error: suffix or operands invalid for `push'
> >>> md/i386.s:72: Error: suffix or operands invalid for `push'
> >>> md/i386.s:77: Error: suffix or operands invalid for `push'
> >>> md/i386.s:78: Error: suffix or operands invalid for `push'
> >>> md/i386.s:79: Error: suffix or operands invalid for `push'
> >>> md/i386.s:81: Error: suffix or operands invalid for `call'
> >>> md/i386.s:84: Error: suffix or operands invalid for `pop'
> >>> md/i386.s:85: Error: suffix or operands invalid for `pop'
> >>> md/i386.s:86: Error: suffix or operands invalid for `pop'
> >>> md/i386.s:87: Error: suffix or operands invalid for `pop'
> >>> md/i386.s:98: Error: suffix or operands invalid for `push'
> >>> md/i386.s:99: Error: suffix or operands invalid for `call'
> >>> md/i386.s:100: Error: suffix or operands invalid for `pop'
> >>> md/i386.s:102: Error: suffix or operands invalid for `call'
> >>> md/i386.s:104: Error: suffix or operands invalid for `push'
> >>> md/i386.s:105: Error: suffix or operands invalid for `push'
> >>> md/i386.s:106: Error: suffix or operands invalid for `call'
> >>> gmake[1]: *** [qt] Error 1
> >>> gmake[1]: Leaving directory
> >>> `/gpfsm/dnb33/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/m
> >>> pi-linux-ifort-icc/tmp/QuickThreads'
> >>> Did you try to build net-linux on a 64-bit system? Try net-linux-x86_64
> >>> gmake: *** [QUICK_THREADS] Error 1
> >>> -------------------------------------------------
> >>> Charm++ NOT BUILT. Either cd into mpi-linux-ifort-icc/tmp and try
> >>> to resolve the problems yourself, visit
> >>> http://charm.cs.uiuc.edu/
> >>> for more information. Otherwise, email the developers at charm AT cs.uiuc.edu
> >>>
> >>> I suspect I am missing something in the charm build... Any Suggestions?
> >>>
> >>> I also attempted to build with:
> >>> env MPICXX=mpicxx ./build charm++ mpi-linux-x86_64 ifort --no-build-shared
> >>> --with-production >& build.out.2
> >>>
> >>> But that apparently is for Opterons.
> >>
> >> As Chao Mei suggested, mpi-linux-x86_64 is the right build target for
> >> your EM64T (64-bit x86) systems, which use the same ISA as Opterons.
> >> Did that build fail? If so, what was the resulting error?
> >>
> >> Phil
> >>
> >>
> >>
> >> --
> >> _______________________________________________________
> >> Brent Swartz
> >> Scientific Application Computational Scientist
> >> Software Integration & Visualization Office - Code 610.3
> >> NASA Goddard Space Flight Center
> >>
> >> (715)726-1405  Brent.A.Swartz AT nasa.gov
> >> _______________________________________________________
> >>
> >
> >
> >
> > --
> > _______________________________________________________
> > Brent Swartz
> > Scientific Application Computational Scientist
> > Software Integration & Visualization Office - Code 610.3
> > NASA Goddard Space Flight Center
> >
> > (715)726-1405  Brent.A.Swartz AT nasa.gov
> > _______________________________________________________
> >


-- 
_______________________________________________________
Brent Swartz
Scientific Application Computational Scientist
Software Integration & Visualization Office - Code 610.3
NASA Goddard Space Flight Center

(715)726-1405  Brent.A.Swartz AT nasa.gov
_______________________________________________________