Charm++ parallel programming system

["Swartz, Brent A. (GSFC-610.3)[ADVANCED MANAGEMENT TECHNOLOGY]" <brent.a.swartz AT nasa.gov>]

Title: Re: [charm] Charm 6.2.2 build

Hi Phil et al,

I successfully rebuilt using:

baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2> env MPICXX=mpicxx ./build charm++ mpi-linux-x86_64 ifort --no-build-shared --with-production >& build.out.4

But when I compile the megatest pgm I get:

baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort/tests/charm++/megatest> make pgm >& make.out.1

../../../bin/charmc  -cxx=icc -o pgm  megatest.o groupring.o nodering.o varsizetest.o varraystest.o groupcast.o groupmulti.o groupsectiontest.o multisectiontest.o nodecast.o synctest.o fib.o arrayring.o tempotest.o packtest.o queens.o migration.o marshall.o priomsg.o priotest.o rotest.o statistics.o templates.o inherit.o reduction.o bitvector.o immediatering.o callback.o inlineem.o  -language charm++ -lguide -cxx=icc
megatest.o:(.gnu.linkonce.d._ZTI4main[.gnu.linkonce.d._ZTI4main]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
megatest.o:(.gnu.linkonce.d.DW.ref.__gxx_personality_v0+0x0): undefined reference to `__gxx_personality_v0'
groupring.o: In function `groupring_group::~groupring_group()':
groupring.C:(.gnu.linkonce.t._ZN15groupring_groupD1Ev[.gnu.linkonce.t._ZN15groupring_groupD1Ev]+0x4a): undefined reference to `operator delete[](void*)'
groupring.C:(.gnu.linkonce.t._ZN15groupring_groupD1Ev[.gnu.linkonce.t._ZN15groupring_groupD1Ev]+0xf3): undefined reference to `operator delete[](void*)'
groupring.C:(.gnu.linkonce.t._ZN15groupring_groupD1Ev[.gnu.linkonce.t._ZN15groupring_groupD1Ev]+0x18e): undefined reference to `operator delete[](void*)'

Then I edited the Makefile to remove all –cxx=icc references. Then I get:

../../../bin/charmc   -o inlineem.o inlineem.C
../../../bin/charmc   -o pgm  megatest.o groupring.o nodering.o varsizetest.o varraystest.o groupcast.o groupmulti.o groupsectiontest.o multisectiontest.o nodecast.o synctest.o fib.o arrayring.o tempotest.o packtest.o queens.o migration.o marshall.o priomsg.o priotest.o rotest.o statistics.o templates.o inherit.o reduction.o bitvector.o immediatering.o callback.o inlineem.o  -language charm++ -lguide
../../../bin/../lib/libthreads-default.o: In function `CthRegister':
threads.c:(.text+0xdb): undefined reference to `_intel_fast_memset'
../../../bin/../lib/libthreads-default.o: In function `CthRegistered':
threads.c:(.text+0x16b): undefined reference to `_intel_fast_memset'
../../../bin/../lib/libthreads-default.o: In function `CthSwitchThread':
threads.c:(.text+0x26a): undefined reference to `_intel_fast_memset'
../../../bin/../lib/libthreads-default.o: In function `CthBaseResume':
threads.c:(.text+0x4da): undefined reference to `_intel_fast_memset'
...
../../../bin/../lib/libconv-core.a(debug-conv.o): In function `CpdDebugCallAllocationTree':
debug-conv.c:(.text+0x459): undefined reference to `__intel_sse2_strlen'
../../../bin/../lib/libconv-core.a(debug-conv.o): In function `CpdDebugCallMemStat':
debug-conv.c:(.text+0x788): undefined reference to `__intel_sse2_strlen'
...

This is the problem I saw before. I thought these routines were in the –lguide compiler lib, as  I can see then in the nm output:

baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort/tests/charm++/megatest> nm /usr/local/intel/Compiler/11.1/072/lib/intel64/libguide.a > guide.nm

fastmemset.o:
                 U __kmp_external___intel_cpu_indicator
                 U __kmp_external___intel_cpu_indicator_init
0000000000000000 T __kmp_external__intel_fast_memset
0000000000000036 T __kmp_external__intel_fast_memset.A
000000000000002e T __kmp_external__intel_fast_memset.J
                 U __kmp_external___intel_new_memset
                 U memset

Although these symbols are not exactly _intel_fast_memset ... They look more like pthreads references.
I get the same undefined references with and without the –lguide.
So I guess the question here is what libs do I need to include to satisfy these undefined references?
Or should I build charm with different options (icc)?

Thanks,

- Brent

On 9/28/10 11:18 AM, "Jim Phillips" <jim AT ks.uiuc.edu> wrote:

> Adding -cxx=icc is probably what broke it.  The mpi builds need to use
> mpicc and mpicxx to get all of the libraries correctly.  Your mpicc and
> mpicxx scripts need to use icc as the base compiler.
>
> The NAMD build uses icc directly and calls charmc to link, but everything
> in the charm tree just uses charmc (and in this case mpicc/mpicxx).
>
> -Jim
>
>
> On Tue, 28 Sep 2010, Swartz, Brent A. (GSFC-610.3)[ADVANCED MANAGEMENT
> TECHNOLOGY] wrote:
>
> > Hi Phil,
> >
> > Yes I originally tried the mpi-linux-x86_64 build target, with and
> > without icc and ifort, and charm++ built, but when I attempt to compile
> > the megatest sample code (I added the -cxx=icc to the Makefile) I get:
> >
> > baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-mpicxx/tests/charm++/megatest>
> > make pgm
> >
> > ...
> > ../../../bin/charmc  -cxx=icc -c inlineem.ci
> > ../../../bin/charmc  -cxx=icc -o inlineem.o inlineem.C
> > icpc: command line warning #10156: ignoring option '-c'; no argument required
> > ../../../bin/charmc  -cxx=icc -o pgm  megatest.o groupring.o nodering.o varsizetest.o varraystest.o groupcast.o groupmulti.o groupsectiontest.o multisectiontest.o nodecast.o synctest.o fib.o arrayring.o tempotest.o packtest.o queens.o migration.o marshall.o priomsg.o priotest.o rotest.o statistics.o templates.o inherit.o reduction.o bitvector.o immediatering.o callback.o inlineem.o  -language charm++ -lguide -cxx=icc
> > Fatal Error by charmc in directory /discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/tests/charm++/megatest
> >   charmc : No such thread mode default
> > charmc exiting...
> > make: *** [pgm] Error 1
> >
> > Prior to that I was getting:
> > baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort.old/tests/charm++/megatest> make pgm
> > ../../../bin/charmc  -cxx=icc -o pgm  megatest.o groupring.o nodering.o varsizetest.o varraystest.o groupcast.o groupmulti.o groupsectiontest.o multisectiontest.o nodecast.o synctest.o fib.o arrayring.o tempotest.o packtest.o queens.o migration.o marshall.o priomsg.o priotest.o rotest.o statistics.o templates.o inherit.o reduction.o bitvector.o immediatering.o callback.o inlineem.o  -language charm++ -lguide -cxx=icc
> > megatest.o:(.gnu.linkonce.d._ZTI4main[.gnu.linkonce.d._ZTI4main]+0x0): undefined reference to `vtable for __cxxabiv1::__si_class_type_info'
> > megatest.o:(.gnu.linkonce.d.DW.ref.__gxx_personality_v0+0x0): undefined reference to `__gxx_personality_v0'
> > groupring.o: In function `groupring_group::~groupring_group()':
> > groupring.C:(.gnu.linkonce.t._ZN15groupring_groupD1Ev[.gnu.linkonce.t._ZN15groupring_groupD1Ev]+0x4a): undefined reference to `operator delete[](void*)'
> > groupring.C:(.gnu.linkonce.t._ZN15groupring_groupD1Ev[.gnu.linkonce.t._ZN15groupring_groupD1Ev]+0xf3): undefined reference to `operator delete[](void*)'
> > ... (More similar undefined references)
> >
> > Also, while building charm++ I got the following error:
> >
> > ../bin/charmc  -optimize  -nobs -I. -o ../lib/libmemory-charmdebug.o -DCMK_MEMORY_BUILD_CHARMDEBUG memory.c || touch ../lib/libmemory-charmdebug.o
> > memory-charmdebug.c(388): warning #181: argument is incompatible with corresponding format string conversion
> >    printf("scanning stack from %p (%d) to %p (%d)\n",begin_stack,begin_stack,end_stack,end_stack);
> >                                                                  ^
> >
> > memory-charmdebug.c(388): warning #181: argument is incompatible with corresponding format string conversion
> >    printf("scanning stack from %p (%d) to %p (%d)\n",begin_stack,begin_stack,end_stack,end_stack);
> >                                                                                        ^
> >
> > memory-charmdebug.c(401): warning #181: argument is incompatible with corresponding format string conversion
> >    printf("scanning data from %p (%d) to %p (%d)\n",info->begin_data,info->begin_data,info->end_data,info->end_data);
> >                                                                      ^
> >
> > memory-charmdebug.c(401): warning #181: argument is incompatible with corresponding format string conversion
> >    printf("scanning data from %p (%d) to %p (%d)\n",info->begin_data,info->begin_data,info->end_data,info->end_data);
> >                                                                                                      ^
> >
> > memory-charmdebug.c(416): warning #181: argument is incompatible with corresponding format string conversion
> >    printf("scanning bss from %p (%d) to %p (%d)\n",info->begin_bss,info->begin_bss,info->end_bss,info->end_bss);
> >                                                                    ^
> >
> > memory-charmdebug.c(416): warning #181: argument is incompatible with corresponding format string conversion
> >    printf("scanning bss from %p (%d) to %p (%d)\n",info->begin_bss,info->begin_bss,info->end_bss,info->end_bss);
> >                                                                                                  ^
> >
> > memory-charmdebug.c(1321): warning #181: argument is incompatible with corresponding format string conversion
> >    sprintf(buf,"slot size %d\n",sizeof(Slot));
> >                                 ^
> >
> > memory-charmdebug.c(1542): error: expected an identifier
> >  void setMemoryTypeChare(void *ptr) {
> >                                     ^
> >
> > memory-charmdebug.c(1542): error: expected a ";"
> >  void setMemoryTypeChare(void *ptr) {
> >                                     ^
> >
> > memory-isomalloc.h(17): warning #12: parsing restarts here after previous syntax error
> >  void *malloc_nomigrate(size_t size);
> >                                     ^
> >
> > compilation aborted for memory.c (code 2)
> > Fatal Error by charmc in directory /discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/mpi-linux-x86_64-ifort/tmp
> >   Command mpicc -m64 -fPIC -I../bin/../include -D__CHARMC__=1 -I. -DCMK_MEMORY_BUILD_CHARMDEBUG -O -c memory.c -o ../lib/libmemory-charmdebug.o returned error code 2
> > charmc exiting...
> >
> > However charm++ does build successfully despite these errors.
> >
> > Cheers,
> >
> > - Brent
> >
> >
> > On 9/27/10 5:36 PM, "Phil Miller" <mille121 AT illinois.edu> wrote:
> >
> > On Mon, Sep 27, 2010 at 17:11, Swartz, Brent A. (GSFC-610.3)[ADVANCED
> > MANAGEMENT TECHNOLOGY] <brent.a.swartz AT nasa.gov> wrote:
> >> In order to install the latest NAMD on our cluster here I am attempting to
> >> build charm for our Intel Xeon EM64T processor (Nehamlem, Harpertown,
> >> Woodcrest, Dempsey) cluster running SUSE Linux here at NASA GSFC. I want to
> >> use the Intel compilers (icc, ifort) and Intel MPI to communicate between
> >> the nodes.  So I have the following modules loaded on our system:
> >>
> >> module load comp/intel-11.1.072 mpi/impi-3.2.2.006
> >>
> >> baswartz@discover08:/discover/nobackup/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2
> >> .7b4_Source/charm-6.2.2> uname -a
> >> Linux discover08 2.6.16.60-0.42.5-smp #1 SMP Mon Aug 24 09:41:41 UTC 2009
> >> x86_64 x86_64 x86_64 GNU/Linux
> >>
> >> Then I attempt to build charm:
> >>
> >> env MPICXX=mpicxx ./build charm++ mpi-linux icc ifort --no-build-shared
> >> --with-production >& build.out.3
> >>
> >> If you like I can email you the entire build.out.3 file, but the main
> >> problems appear to be:
> >>
> >> [snip irrelevant parts]
> >>
> >> Running configure on i386-gcc
> >> ( cd QuickThreads ; /usr/bin/gmake qt  OPTS='-optimize  -nobs' ) || ( echo
> >> "Did you try to build net-linux on a 64-bit system? Try net-linux-x86_64" &&
> >> false )
> >> gmake[1]: Entering directory
> >> `/gpfsm/dnb33/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/m
> >> pi-linux-ifort-icc/tmp/QuickThreads'
> >> rm -f qtmd.h
> >> ln -s md/i386.h qtmd.h
> >> ../../bin/charmc -seq -I. -o qt.o qt.c
> >> as -o qtmds.o md/i386.s
> >> md/i386.s: Assembler messages:
> >> md/i386.s:69: Error: suffix or operands invalid for `push'
> >> md/i386.s:70: Error: suffix or operands invalid for `push'
> >> md/i386.s:71: Error: suffix or operands invalid for `push'
> >> md/i386.s:72: Error: suffix or operands invalid for `push'
> >> md/i386.s:77: Error: suffix or operands invalid for `push'
> >> md/i386.s:78: Error: suffix or operands invalid for `push'
> >> md/i386.s:79: Error: suffix or operands invalid for `push'
> >> md/i386.s:81: Error: suffix or operands invalid for `call'
> >> md/i386.s:84: Error: suffix or operands invalid for `pop'
> >> md/i386.s:85: Error: suffix or operands invalid for `pop'
> >> md/i386.s:86: Error: suffix or operands invalid for `pop'
> >> md/i386.s:87: Error: suffix or operands invalid for `pop'
> >> md/i386.s:98: Error: suffix or operands invalid for `push'
> >> md/i386.s:99: Error: suffix or operands invalid for `call'
> >> md/i386.s:100: Error: suffix or operands invalid for `pop'
> >> md/i386.s:102: Error: suffix or operands invalid for `call'
> >> md/i386.s:104: Error: suffix or operands invalid for `push'
> >> md/i386.s:105: Error: suffix or operands invalid for `push'
> >> md/i386.s:106: Error: suffix or operands invalid for `call'
> >> gmake[1]: *** [qt] Error 1
> >> gmake[1]: Leaving directory
> >> `/gpfsm/dnb33/baswartz/bench/NAS_BM_PGK/NAMD/NAMD_2.7b4_Source/charm-6.2.2/m
> >> pi-linux-ifort-icc/tmp/QuickThreads'
> >> Did you try to build net-linux on a 64-bit system? Try net-linux-x86_64
> >> gmake: *** [QUICK_THREADS] Error 1
> >> -------------------------------------------------
> >> Charm++ NOT BUILT. Either cd into mpi-linux-ifort-icc/tmp and try
> >> to resolve the problems yourself, visit
> >> http://charm.cs.uiuc.edu/
> >> for more information. Otherwise, email the developers at charm AT cs.uiuc.edu
> >>
> >> I suspect I am missing something in the charm build... Any Suggestions?
> >>
> >> I also attempted to build with:
> >> env MPICXX=mpicxx ./build charm++ mpi-linux-x86_64 ifort --no-build-shared
> >> --with-production >& build.out.2
> >>
> >> But that apparently is for Opterons.
> >
> > As Chao Mei suggested, mpi-linux-x86_64 is the right build target for
> > your EM64T (64-bit x86) systems, which use the same ISA as Opterons.
> > Did that build fail? If so, what was the resulting error?
> >
> > Phil
> >
> >
> >
> > --
> > _______________________________________________________
> > Brent Swartz
> > Scientific Application Computational Scientist
> > Software Integration & Visualization Office - Code 610.3
> > NASA Goddard Space Flight Center
> >
> > (715)726-1405  Brent.A.Swartz AT nasa.gov
> > _______________________________________________________
> >


-- 
_______________________________________________________
Brent Swartz
Scientific Application Computational Scientist
Software Integration & Visualization Office - Code 610.3
NASA Goddard Space Flight Center

(715)726-1405  Brent.A.Swartz AT nasa.gov
_______________________________________________________