Charm++ parallel programming system

["Matthew B. Roark" <Roarkma AT wabash.edu>]

I am running Rocks 5.1 and want to install NAMD.  I have downloaded the 
source files for v2.7 (which uses Charm-6.2.2).  I want to build
several versions to test efficiency and scalability with both a gigabit 
ethernet connection and infiniband.  
I am able to install (and make a megatest) with the following build commands:

CHARM_FLAGS="--no-build-shared --with-production"
./build charm++ multicore-linux64 gcc -j4 ${CHARM_FLAGS}
./build charm++ net-linux-x86_64 gcc -j4 ${CHARM_FLAGS}
./build charm++ net-linux-x86_64 gcc ibverbs -j4 ${CHARM_FLAGS}

I am not able to build Charm-6.2.2 with OpenMPI v1.2.6 that came with the 
OFED package for Rocks 5.1.  I am using a script to automate the setup, 
and here are the relevant parts:
MPI_INCLUDE="/usr/mpi/gcc/openmpi-1.2.6/include"
MPI_LIB="/usr/mpi/gcc/openmpi-1.2.6/lib64"

CHARMSRC="${NAMDSRC}/charm-6.2.2"
cp -r ${CHARMSRC}/src/arch/mpi-linux-x86_64 
${CHARMSRC}/src/arch/mpi-linux-x86_64-openmpi
sed -e 's/MPICXX_DEF=mpgcc/MPICXX_DEF=mpicxx/' -i 
${CHARMSRC}/src/arch/mpi-linux-x86_64-openmpi/conv-mach.sh

env MPICXX=mpicxx ./build charm++ mpi-linux-x86_64-openmpi mpicxx -j4 
${CHARM_FLAGS} --incdir ${MPI_INCLUDE} --libdir ${MPI_LIB} -O

I save the output of the build command in a log file and it reads:

Selected Compiler: mpicxx
Selected Options: 
Copying src/scripts/Makefile to mpi-linux-x86_64-openmpi-mpicxx/tmp
Soft-linking over bin
Soft-linking over lib
Soft-linking over lib_so
Soft-linking over include
Soft-linking over tmp
Generating mpi-linux-x86_64-openmpi-mpicxx/tmp/conv-mach-pre.sh
Performing '/usr/bin/gmake -j 4 basics OPTS=-optimize -O QUIET=' in 
mpi-linux-x86_64-openmpi-mpicxx/tmp
/usr/bin/gmake charmxi
gmake[1]: Entering directory 
`/export/home/charmm/NAMD_2.7_Source/charm-6.2.2/mpi-linux-x86_64-openmpi-mpicxx/tmp'
gmake[1]: Nothing to be done for `charmxi'.
gmake[1]: Leaving directory 
`/export/home/charmm/NAMD_2.7_Source/charm-6.2.2/mpi-linux-x86_64-openmpi-mpicxx/tmp'
/usr/bin/gmake headerlinks
gmake[1]: Entering directory 
`/export/home/charmm/NAMD_2.7_Source/charm-6.2.2/mpi-linux-x86_64-openmpi-mpicxx/tmp'
../bin/charmc -intrinsic -optimize  -O -nobs ckcallback.ci
Passing -O to charmc no longer uses charm's platform-specific
optimization defaults. Use -optimize for that instead.
The -O flag is being passed to the underlying compiler.
../bin/charmxi: error while loading shared libraries: libmpi_cxx.so.0: cannot 
open shared object file: No such file or directory
Fatal Error by charmc in directory 
/export/home/charmm/NAMD_2.7_Source/charm-6.2.2/mpi-linux-x86_64-openmpi-mpicxx/tmp
   Command ../bin/charmxi -intrinsic ckcallback.ci returned error code 127
charmc exiting...
gmake[1]: *** [CkCallback.decl.h] Error 1
gmake[1]: Leaving directory 
`/export/home/charmm/NAMD_2.7_Source/charm-6.2.2/mpi-linux-x86_64-openmpi-mpicxx/tmp'
gmake: *** [headers] Error 2
-------------------------------------------------
Charm++ NOT BUILT. Either cd into mpi-linux-x86_64-openmpi-mpicxx/tmp and try
to resolve the problems yourself, visit
http://charm.cs.uiuc.edu/
for more information. Otherwise, email the developers at 
charm AT cs.uiuc.edu


I do not understand why it is not able to find this shared library.  It does 
exist in the specified library directory:
[charmm@rocks2
 NAMD_2.7_Source]$ ls -l /usr/mpi/gcc/openmpi-1.2.6/lib64/
total 11752
-rwxr-xr-x 1 root root    1034 Jun  5  2008 libmca_common_sm.la
lrwxrwxrwx 1 root root      25 Apr 29  2009 libmca_common_sm.so -> 
libmca_common_sm.so.0.0.0
lrwxrwxrwx 1 root root      25 Apr 29  2009 libmca_common_sm.so.0 -> 
libmca_common_sm.so.0.0.0
-rwxr-xr-x 1 root root   27894 Jun  5  2008 libmca_common_sm.so.0.0.0
-rwxr-xr-x 1 root root     994 Jun  5  2008 libmpi_cxx.la
lrwxrwxrwx 1 root root      19 Apr 29  2009 libmpi_cxx.so -> 
libmpi_cxx.so.0.0.0
lrwxrwxrwx 1 root root      19 Apr 29  2009 libmpi_cxx.so.0 -> 
libmpi_cxx.so.0.0.0
-rwxr-xr-x 1 root root 1446431 Jun  5  2008 libmpi_cxx.so.0.0.0
-rwxr-xr-x 1 root root     998 Jun  5  2008 libmpi_f77.la
lrwxrwxrwx 1 root root      19 Apr 29  2009 libmpi_f77.so -> 
libmpi_f77.so.0.0.0
lrwxrwxrwx 1 root root      19 Apr 29  2009 libmpi_f77.so.0 -> 
libmpi_f77.so.0.0.0
-rwxr-xr-x 1 root root 1756773 Jun  5  2008 libmpi_f77.so.0.0.0
-rwxr-xr-x 1 root root     989 Jun  5  2008 libmpi_f90.la
lrwxrwxrwx 1 root root      19 Apr 29  2009 libmpi_f90.so -> 
libmpi_f90.so.0.0.0
lrwxrwxrwx 1 root root      19 Apr 29  2009 libmpi_f90.so.0 -> 
libmpi_f90.so.0.0.0
-rwxr-xr-x 1 root root   35008 Jun  5  2008 libmpi_f90.so.0.0.0
-rwxr-xr-x 1 root root    1075 Jun  5  2008 libmpi.la
lrwxrwxrwx 1 root root      15 Apr 29  2009 libmpi.so -> libmpi.so.0.0.0
lrwxrwxrwx 1 root root      15 Apr 29  2009 libmpi.so.0 -> libmpi.so.0.0.0
-rwxr-xr-x 1 root root 5298971 Jun  5  2008 libmpi.so.0.0.0
-rwxr-xr-x 1 root root    1009 Jun  5  2008 libopen-pal.la
lrwxrwxrwx 1 root root      20 Apr 29  2009 libopen-pal.so -> 
libopen-pal.so.0.0.0
lrwxrwxrwx 1 root root      20 Apr 29  2009 libopen-pal.so.0 -> 
libopen-pal.so.0.0.0
-rwxr-xr-x 1 root root 1002148 Jun  5  2008 libopen-pal.so.0.0.0
-rwxr-xr-x 1 root root    1057 Jun  5  2008 libopen-rte.la
lrwxrwxrwx 1 root root      20 Apr 29  2009 libopen-rte.so -> 
libopen-rte.so.0.0.0
lrwxrwxrwx 1 root root      20 Apr 29  2009 libopen-rte.so.0 -> 
libopen-rte.so.0.0.0
-rwxr-xr-x 1 root root 2176406 Jun  5  2008 libopen-rte.so.0.0.0
-rwxr-xr-x 1 root root  210202 Jun  5  2008 mpi.mod
drwxr-xr-x 2 root root    4096 Apr 29  2009 openmpi