Charm++ parallel programming system

["Choi, Jaemin" <jchoi157 AT illinois.edu>]

Can someone who've built NAMD before answer this question?

Thanks,

Jaemin Choi

Ph.D. Student, Research Assistant

Parallel Programming Laboratory

University of Illinois Urbana-Champaign

---

From: sutirtha kamal sen [sks.silentnight AT gmail.com]
Sent: Saturday, August 05, 2017 2:41 AM
To: Choi, Jaemin
Subject: Re: [charm] Problem with charm++ building

oh! sorry, I must have missed them.

I followed the following line of instructions:

tar xf NAMD_2.11_Source.tar.gz
cd NAMD_2.11_Source
tar xf charm-6.7.0.tar
wget http://www.ks.uiuc.edu/Research/namd/libraries/fftw-linux-x86_64.tar.gz
wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl8.5.9-linux-x86_64.tar.gz
wget http://www.ks.uiuc.edu/Research/namd/libraries/tcl8.5.9-linux-x86_64-threaded.tar.gz
tar xzf fftw-linux-x86_64.tar.gz
mv linux-x86_64 fftw
tar xzf tcl*-linux-x86_64.tar.gz
tar xzf tcl*-linux-x86_64-threaded.tar.gz
mv tcl*-linux-x86_64 tcl
mv tcl*-linux-x86_64-threaded tcl-threaded
cd charm-6.7.0
./build charm++  verbs-linux-x86_64 gcc --with-production

In first try, they were showing some indiband/verbs.h missing; then when I corrected that, a string of other errors in various machine.c and related files

On Fri, Aug 4, 2017 at 10:10 PM, Choi, Jaemin <jchoi157 AT illinois.edu> wrote:

Hi Sutirtha,

Could you tell us what the build command was?
Also, the attachments seem to be missing.

Thanks,



Jaemin Choi
Ph.D. Student, Research Assistant
Parallel Programming Laboratory
University of Illinois Urbana-Champaign

________________________________________
From: sutirtha kamal sen [sks.silentnight AT gmail.com]
Sent: Friday, August 04, 2017 1:45 AM
To: charm AT cs.illinois.edu
Subject: [charm] Problem with charm++ building

Hello
I am having problems with building of charm++. Apparently, a file named "infiniband/verbs.h" is missing in the package due to which the build is not executed completely. Even if I make the program ignore that include statement, I'm getting a string of errors due to which the build is terminated. I am getting the errors, as given in the files attached error_1 and error_2 respectively. Please check the problems and look into it.I will be obliged.

Thanking You,
Sutirtha Kamal Sen

convcore.c: In function ‘CmiGetArgLongDesc’:
convcore.c:501:19: warning: format ‘%lld’ expects argument of type 
‘long long int *’, but argument 3 has type ‘CmiInt8 * {aka long int 
*}’ [-Wformat=]
    if (sscanf(opt,"%lld",optDest)<1) {
                   ^
../bin/charmc  -optimize -production   -I.   -c -o conv-conds.o conv-conds.c
../bin/charmc  -optimize -production   -I.   -c -o queueing.o queueing.C
../bin/charmc  -optimize -production   -I.   -c -o msgmgr.o msgmgr.c
../bin/charmc  -optimize -production   -I.   -c -o cpm.o cpm.c
../bin/charmc  -optimize -production   -I.   -c -o cpthreads.o cpthreads.c
../bin/charmc  -optimize -production   -I.   -c -o futures.o futures.c
../bin/charmc  -optimize -production   -I.   -c -o cldb.o cldb.c
../bin/charmc  -optimize -production   -I.   -c -o random.o random.c
../bin/charmc  -optimize -production   -I.   -c -o debug-conv.o debug-conv.c
../bin/charmc  -optimize -production   -I.   -c -o debug-conv++.o 
debug-conv++.C
../bin/charmc  -optimize -production   -I.   -c -o conv-ccs.o conv-ccs.c
../bin/charmc  -optimize -production   -I.   -c -o ccs-builtins.o 
ccs-builtins.C
../bin/charmc  -optimize -production   -I.   -c -o middle-ccs.o middle-ccs.C
../bin/charmc  -optimize -production   -I.   -c -o traceCore.o traceCore.C
../bin/charmc  -optimize -production   -I.   -c -o traceCoreCommon.o 
traceCoreCommon.C
../bin/charmc  -optimize -production   -I.   -c -o converseProjections.o 
converseProjections.C
../bin/charmc  -optimize -production   -I.   -c -o machineProjections.o 
machineProjections.C
../bin/charmc  -optimize -production   -I.   -c -o quiescence.o quiescence.c
../bin/charmc  -optimize -production   -I.   -c -o isomalloc.o isomalloc.c
../bin/charmc  -optimize -production   -I.   -c -o mem-arena.o mem-arena.c
../bin/charmc  -optimize -production   -I.   -c -o conv-counter.o 
conv-counter.c
../bin/charmc  -optimize -production   -I.   -c -o memory-darwin-clang.o 
memory-darwin-clang.C
../bin/charmc  -optimize -production   -I.   -c -o global-nop.o global-nop.c
../bin/charmc  -optimize -production   -I.   -c -o cmipool.o cmipool.c
../bin/charmc  -optimize -production   -I.   -c -o cpuaffinity.o cpuaffinity.c
../bin/charmc  -optimize -production   -I.   -c -o cputopology.o cputopology.C
../bin/charmc  -optimize -production   -I.   -c -o cmitls.o cmitls.c
../bin/charmc  -optimize -production   -I.   -c -o memoryaffinity.o 
memoryaffinity.c
SRCBASE=../../src ./commitid.sh
Copying VERSION.new = 
v6.8.0-beta1-314-g4decdbf-namd-charm-6.8.0-build-2017-Jul-18-4809 over 
VERSION =
echo "const char * const CmiCommitID = \"`cat VERSION`\";" > commitid.c
../bin/charmc  -optimize -production   -I.   -c -o commitid.o commitid.c
../bin/charmc  -optimize -production   -I.   -c -o sdag.o sdag.C
../bin/charmc  -optimize -production   -I.   -c -o conv-interoperate.o 
conv-interoperate.C
../bin/charmc  -optimize -production    -o ../lib/libconv-core.a convcore.o 
conv-conds.o queueing.o msgmgr.o cpm.o cpthreads.o futures.o cldb.o random.o 
debug-conv.o debug-conv++.o conv-ccs.o ccs-builtins.o middle-ccs.o 
traceCore.o traceCoreCommon.o converseProjections.o machineProjections.o 
quiescence.o isomalloc.o mem-arena.o conv-counter.o memory-darwin-clang.o 
global-nop.o cmipool.o cpuaffinity.o cputopology.o cmitls.o memoryaffinity.o 
commitid.o sdag.o conv-interoperate.o 
ar: creating ../lib/libconv-core.a
../bin/charmc  -optimize -production   -I. -o machine.o  -DFOR_CPLUS=1 
machine.c
machine.c: In function ‘machine_exit’:
machine.c:327:3: warning: implicit declaration of function 
‘EmergencyExit’ [-Wimplicit-function-declaration]
   EmergencyExit();
   ^
machine.c: At top level:
machine.c:865:1: warning: data definition has no type or storage class
 _Cmi_myrank=0;
 ^
machine.c:865:1: warning: type defaults to ‘int’ in declaration of 
‘_Cmi_myrank’ [-Wimplicit-int]
In file included from machine-dgram.c:640:0,
                 from machine.c:1397:
machine-ibverbs.c:39:6: error: variable ‘mtu’ has initializer but 
incomplete type
 enum ibv_mtu mtu = IBV_MTU_2048;
      ^
machine-ibverbs.c:39:20: error: ‘IBV_MTU_2048’ undeclared here (not in a 
function)
 enum ibv_mtu mtu = IBV_MTU_2048;
                    ^
machine-ibverbs.c:148:16: error: field ‘key’ has incomplete type
  struct ibv_mr key;
                ^
machine-ibverbs.c:194:17: error: array type has incomplete element type 
‘struct ibv_sge’
  struct ibv_sge elemList[2];
                 ^
machine-ibverbs.c:195:21: error: field ‘wr’ has incomplete type
  struct ibv_send_wr wr;
                     ^
In file included from machine-ibverbs.c:328:0,
                 from machine-dgram.c:640,
                 from machine.c:1397:
machine-onesided.h: In function ‘LrtsSetRdmaOpInfo’:
machine-onesided.h:102:8: warning: implicit declaration of function 
‘ibv_reg_mr’ [-Wimplicit-function-declaration]
   mr = ibv_reg_mr(context->pd, ptr, size, IBV_ACCESS_REMOTE_READ |
        ^
machine-onesided.h:102:43: error: ‘IBV_ACCESS_REMOTE_READ’ undeclared 
(first use in this function)
   mr = ibv_reg_mr(context->pd, ptr, size, IBV_ACCESS_REMOTE_READ |
                                           ^
machine-onesided.h:102:43: note: each undeclared identifier is reported only 
once for each function it appears in
machine-onesided.h:103:7: error: ‘IBV_ACCESS_LOCAL_WRITE’ undeclared 
(first use in this function)
       IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
       ^
machine-onesided.h:103:32: error: ‘IBV_ACCESS_REMOTE_WRITE’ undeclared 
(first use in this function)
       IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
                                ^
machine-onesided.h:108:19: error: dereferencing pointer to incomplete type 
‘struct ibv_mr’
   rdmaOp->key = mr->rkey;
                   ^
In file included from machine-dgram.c:640:0,
                 from machine.c:1397:
machine-ibverbs.c: In function ‘newPacket’:
machine-ibverbs.c:369:19: error: ‘IBV_WR_SEND’ undeclared (first use in 
this function)
  pkt->wr.opcode = IBV_WR_SEND;
                   ^
machine-ibverbs.c:370:23: error: ‘IBV_SEND_SIGNALED’ undeclared (first 
use in this function)
  pkt->wr.send_flags = IBV_SEND_SIGNALED;
                       ^
machine-ibverbs.c: In function ‘infi_unregAndFreeMeta’:
machine-ibverbs.c:397:21: warning: implicit declaration of function 
‘ibv_dereg_mr’ [-Wimplicit-function-declaration]
       int unregstat=ibv_dereg_mr(((infiCmiChunkMetaData*)md)->key);
                     ^
machine-ibverbs.c: At top level:
machine-ibverbs.c:412:87: warning: ‘struct ibv_device’ declared inside 
parameter list
 void createLocalQps(struct ibv_device *dev,int ibPort, int myNode,int 
numNodes,struct infiAddr *localAddr);
                                                                              
         ^
machine-ibverbs.c:412:87: warning: its scope is only this definition or 
declaration, which is probably not what you want
machine-ibverbs.c: In function ‘checkQp’:
machine-ibverbs.c:415:21: error: storage size of ‘attr’ isn’t known
  struct ibv_qp_attr attr;
                     ^
machine-ibverbs.c:416:26: error: storage size of ‘init_attr’ isn’t known
  struct ibv_qp_init_attr init_attr;
                          ^
machine-ibverbs.c:418:2: warning: implicit declaration of function 
‘ibv_query_qp’ [-Wimplicit-function-declaration]
  ibv_query_qp(qp, &attr, IBV_QP_STATE | IBV_QP_CUR_STATE|IBV_QP_CAP  
,&init_attr);
  ^
machine-ibverbs.c:418:26: error: ‘IBV_QP_STATE’ undeclared (first use in 
this function)
  ibv_query_qp(qp, &attr, IBV_QP_STATE | IBV_QP_CUR_STATE|IBV_QP_CAP  
,&init_attr);
                          ^
machine-ibverbs.c:418:41: error: ‘IBV_QP_CUR_STATE’ undeclared (first use 
in this function)
  ibv_query_qp(qp, &attr, IBV_QP_STATE | IBV_QP_CUR_STATE|IBV_QP_CAP  
,&init_attr);
                                         ^
machine-ibverbs.c:418:58: error: ‘IBV_QP_CAP’ undeclared (first use in 
this function)
  ibv_query_qp(qp, &attr, IBV_QP_STATE | IBV_QP_CUR_STATE|IBV_QP_CAP  
,&init_attr);
                                                          ^
machine-ibverbs.c:419:26: error: ‘IBV_QPS_RTS’ undeclared (first use in 
this function)
  if(attr.cur_qp_state != IBV_QPS_RTS){
                          ^
machine-ibverbs.c: In function ‘CmiMachineInit’:
machine-ibverbs.c:464:13: warning: implicit declaration of function 
‘ibv_get_device_list’ [-Wimplicit-function-declaration]
  devList =  ibv_get_device_list(&num_devices);
             ^
machine-ibverbs.c:464:10: warning: assignment makes pointer from integer 
without a cast [-Wint-conversion]
  devList =  ibv_get_device_list(&num_devices);
          ^
machine-ibverbs.c:490:21: warning: implicit declaration of function 
‘ibv_open_device’ [-Wimplicit-function-declaration]
  context->context = ibv_open_device(dev);
                     ^
machine-ibverbs.c:490:19: warning: assignment makes pointer from integer 
without a cast [-Wint-conversion]
  context->context = ibv_open_device(dev);
                   ^
machine-ibverbs.c:496:32: error: storage size of ‘attr’ isn’t known
           struct ibv_port_attr attr;
                                ^
machine-ibverbs.c:497:15: warning: implicit declaration of function 
‘ibv_query_port’ [-Wimplicit-function-declaration]
           if (ibv_query_port(context->context, ibPort, &attr) != 0) continue;
               ^
In file included from /usr/include/string.h:630:0,
                 from converse.h:59,
                 from machine.c:202:
machine-ibverbs.c:513:36: warning: implicit declaration of function 
‘ibv_get_device_name’ [-Wimplicit-function-declaration]
      if(strcmp(usr_ibv_device_name,ibv_get_device_name(dev))==0)
                                    ^
machine-ibverbs.c:513:36: warning: passing argument 1 of 
‘__builtin_strlen’ makes pointer from integer without a cast 
[-Wint-conversion]
      if(strcmp(usr_ibv_device_name,ibv_get_device_name(dev))==0)
                                    ^
machine-ibverbs.c:513:36: note: expected ‘const char *’ but argument is 
of type ‘int’
machine-ibverbs.c:513:36: warning: passing argument 2 of 
‘__builtin_strcmp’ makes pointer from integer without a cast 
[-Wint-conversion]
      if(strcmp(usr_ibv_device_name,ibv_get_device_name(dev))==0)
                                    ^
machine-ibverbs.c:513:36: note: expected ‘const char *’ but argument is 
of type ‘int’
machine-ibverbs.c:513:36: warning: passing argument 2 of 
‘__builtin_strcmp’ makes pointer from integer without a cast 
[-Wint-conversion]
      if(strcmp(usr_ibv_device_name,ibv_get_device_name(dev))==0)
                                    ^
machine-ibverbs.c:513:36: note: expected ‘const char *’ but argument is 
of type ‘int’
machine-ibverbs.c:513:36: warning: passing argument 1 of 
‘__builtin_strlen’ makes pointer from integer without a cast 
[-Wint-conversion]
      if(strcmp(usr_ibv_device_name,ibv_get_device_name(dev))==0)
                                    ^
machine-ibverbs.c:513:36: note: expected ‘const char *’ but argument is 
of type ‘int’
machine-ibverbs.c:513:36: warning: passing argument 2 of 
‘__builtin_strcmp’ makes pointer from integer without a cast 
[-Wint-conversion]
      if(strcmp(usr_ibv_device_name,ibv_get_device_name(dev))==0)
                                    ^
machine-ibverbs.c:513:36: note: expected ‘const char *’ but argument is 
of type ‘int’
machine-ibverbs.c:513:36: warning: passing argument 2 of 
‘__builtin_strcmp’ makes pointer from integer without a cast 
[-Wint-conversion]
      if(strcmp(usr_ibv_device_name,ibv_get_device_name(dev))==0)
                                    ^
machine-ibverbs.c:513:36: note: expected ‘const char *’ but argument is 
of type ‘int’
In file included from machine-dgram.c:640:0,
                 from machine.c:1397:
machine-ibverbs.c:536:16: warning: implicit declaration of function 
‘ibv_alloc_pd’ [-Wimplicit-function-declaration]
  context->pd = ibv_alloc_pd(context->context);
                ^
machine-ibverbs.c:536:14: warning: assignment makes pointer from integer 
without a cast [-Wint-conversion]
  context->pd = ibv_alloc_pd(context->context);
              ^
machine-ibverbs.c:579:18: warning: passing argument 1 of ‘createLocalQps’ 
from incompatible pointer type [-Wincompatible-pointer-types]
   createLocalQps(dev,ibPort,Lrts_myNode,Lrts_numNodes,context->localAddr);
                  ^
machine-ibverbs.c:412:6: note: expected ‘struct ibv_device *’ but 
argument is of type ‘struct ibv_device *’
 void createLocalQps(struct ibv_device *dev,int ibPort, int myNode,int 
numNodes,struct infiAddr *localAddr);
      ^
machine-ibverbs.c: At top level:
machine-ibverbs.c:679:87: warning: ‘struct ibv_device’ declared inside 
parameter list
 void createLocalQps(struct ibv_device *dev,int ibPort, int myNode,int 
numNodes,struct infiAddr *localAddr){
                                                                              
         ^
machine-ibverbs.c:679:6: error: conflicting types for ‘createLocalQps’
 void createLocalQps(struct ibv_device *dev,int ibPort, int myNode,int 
numNodes,struct infiAddr *localAddr){
      ^
machine-ibverbs.c:412:6: note: previous declaration of ‘createLocalQps’ 
was here
 void createLocalQps(struct ibv_device *dev,int ibPort, int myNode,int 
numNodes,struct infiAddr *localAddr);
      ^
machine-ibverbs.c: In function ‘createLocalQps’:
machine-ibverbs.c:690:20: warning: implicit declaration of function 
‘ibv_create_cq’ [-Wimplicit-function-declaration]
  context->sendCq = 
ibv_create_cq(context->context,context->sendCqSize,NULL,NULL,0);
                    ^
machine-ibverbs.c:690:18: warning: assignment makes pointer from integer 
without a cast [-Wint-conversion]
  context->sendCq = 
ibv_create_cq(context->context,context->sendCqSize,NULL,NULL,0);
                  ^
machine-ibverbs.c:697:18: warning: assignment makes pointer from integer 
without a cast [-Wint-conversion]
  context->recvCq = 
ibv_create_cq(context->context,context->recvCqSize,NULL,NULL,0);
                  ^
machine-ibverbs.c:709:10: error: variable ‘srqAttr’ has initializer but 
incomplete type
   struct ibv_srq_init_attr srqAttr = {
          ^
machine-ibverbs.c:710:4: error: unknown field ‘attr’ specified in 
initializer
    .attr = {
    ^
machine-ibverbs.c:710:12: error: extra brace group at end of initializer
    .attr = {
            ^
machine-ibverbs.c:710:12: note: (near initialization for ‘srqAttr’)
machine-ibverbs.c:710:12: warning: excess elements in struct initializer
machine-ibverbs.c:710:12: note: (near initialization for ‘srqAttr’)
machine-ibverbs.c:709:28: error: storage size of ‘srqAttr’ isn’t known
   struct ibv_srq_init_attr srqAttr = {
                            ^
machine-ibverbs.c:715:18: warning: implicit declaration of function 
‘ibv_create_srq’ [-Wimplicit-function-declaration]
   context->srq = ibv_create_srq(context->pd,&srqAttr);
                  ^
machine-ibverbs.c:718:10: error: variable ‘initAttr’ has initializer but 
incomplete type
   struct ibv_qp_init_attr initAttr = {
          ^
machine-ibverbs.c:719:4: error: unknown field ‘qp_type’ specified in 
initializer
    .qp_type = IBV_QPT_RC,
    ^
machine-ibverbs.c:719:15: error: ‘IBV_QPT_RC’ undeclared (first use in 
this function)
    .qp_type = IBV_QPT_RC,
               ^
machine-ibverbs.c:719:15: warning: excess elements in struct initializer
machine-ibverbs.c:719:15: note: (near initialization for ‘initAttr’)
machine-ibverbs.c:720:4: error: unknown field ‘send_cq’ specified in 
initializer
    .send_cq = context->sendCq,
    ^
machine-ibverbs.c:720:15: warning: excess elements in struct initializer
    .send_cq = context->sendCq,
               ^
machine-ibverbs.c:720:15: note: (near initialization for ‘initAttr’)
machine-ibverbs.c:721:4: error: unknown field ‘recv_cq’ specified in 
initializer
    .recv_cq = context->recvCq,
    ^
machine-ibverbs.c:721:15: warning: excess elements in struct initializer
    .recv_cq = context->recvCq,
               ^
machine-ibverbs.c:721:15: note: (near initialization for ‘initAttr’)
machine-ibverbs.c:722:4: error: unknown field ‘srq’ specified in 
initializer
    .srq   = context->srq,
    ^
machine-ibverbs.c:722:13: warning: excess elements in struct initializer
    .srq   = context->srq,
             ^
machine-ibverbs.c:722:13: note: (near initialization for ‘initAttr’)
machine-ibverbs.c:723:4: error: unknown field ‘sq_sig_all’ specified in 
initializer
    .sq_sig_all = 0,
    ^
machine-ibverbs.c:723:18: warning: excess elements in struct initializer
    .sq_sig_all = 0,
                  ^
machine-ibverbs.c:723:18: note: (near initialization for ‘initAttr’)
machine-ibverbs.c:724:4: error: unknown field ‘qp_context’ specified in 
initializer
    .qp_context = NULL,
    ^
machine-ibverbs.c:725:4: error: unknown field ‘cap’ specified in 
initializer
    .cap     = {
    ^
machine-ibverbs.c:725:15: error: extra brace group at end of initializer
    .cap     = {
               ^
machine-ibverbs.c:725:15: note: (near initialization for ‘initAttr’)
machine-ibverbs.c:725:15: warning: excess elements in struct initializer
machine-ibverbs.c:725:15: note: (near initialization for ‘initAttr’)
machine-ibverbs.c:718:27: error: storage size of ‘initAttr’ isn’t known
   struct ibv_qp_init_attr initAttr = {
                           ^
machine-ibverbs.c:730:22: error: storage size of ‘attr’ isn’t known
   struct ibv_qp_attr attr;
                      ^
machine-ibverbs.c:732:26: error: ‘IBV_QPS_INIT’ undeclared (first use in 
this function)
   attr.qp_state        = IBV_QPS_INIT;
                          ^
machine-ibverbs.c:735:26: error: ‘IBV_ACCESS_REMOTE_READ’ undeclared 
(first use in this function)
   attr.qp_access_flags = IBV_ACCESS_REMOTE_READ | IBV_ACCESS_REMOTE_WRITE;
                          ^
machine-ibverbs.c:735:51: error: ‘IBV_ACCESS_REMOTE_WRITE’ undeclared 
(first use in this function)
   attr.qp_access_flags = IBV_ACCESS_REMOTE_READ | IBV_ACCESS_REMOTE_WRITE;
                                                   ^
machine-ibverbs.c:746:22: warning: implicit declaration of function 
‘ibv_create_qp’ [-Wimplicit-function-declaration]
     context->qp[n] = ibv_create_qp(context->pd,&initAttr);
                      ^
machine-ibverbs.c:751:5: warning: implicit declaration of function 
‘ibv_modify_qp’ [-Wimplicit-function-declaration]
     ibv_modify_qp(context->qp[n], &attr,
     ^
machine-ibverbs.c:752:8: error: ‘IBV_QP_STATE’ undeclared (first use in 
this function)
        IBV_QP_STATE              |
        ^
machine-ibverbs.c:753:8: error: ‘IBV_QP_PKEY_INDEX’ undeclared (first use 
in this function)
        IBV_QP_PKEY_INDEX         |
        ^
machine-ibverbs.c:754:8: error: ‘IBV_QP_PORT’ undeclared (first use in 
this function)
        IBV_QP_PORT               |
        ^
machine-ibverbs.c:755:8: error: ‘IBV_QP_ACCESS_FLAGS’ undeclared (first 
use in this function)
        IBV_QP_ACCESS_FLAGS);  
        ^
machine-ibverbs.c:757:38: error: dereferencing pointer to incomplete type 
‘struct ibv_qp’
     localAddr[n].qpn = context->qp[n]->qp_num;
                                      ^
machine-ibverbs.c: In function ‘getLocalLid’:
machine-ibverbs.c:783:23: error: storage size of ‘attr’ isn’t known
  struct ibv_port_attr attr;
                       ^
machine-ibverbs.c: In function ‘initInfiOtherNodeData’:
machine-ibverbs.c:813:9: error: variable ‘attr’ has initializer but 
incomplete type
  struct ibv_qp_attr attr = {
         ^
machine-ibverbs.c:814:3: error: unknown field ‘qp_state’ specified in 
initializer
   .qp_state  = IBV_QPS_RTR,
   ^
machine-ibverbs.c:814:16: error: ‘IBV_QPS_RTR’ undeclared (first use in 
this function)
   .qp_state  = IBV_QPS_RTR,
                ^
machine-ibverbs.c:814:16: warning: excess elements in struct initializer
machine-ibverbs.c:814:16: note: (near initialization for ‘attr’)
machine-ibverbs.c:815:3: error: unknown field ‘path_mtu’ specified in 
initializer
   .path_mtu  = mtu,
   ^
machine-ibverbs.c:815:16: warning: excess elements in struct initializer
   .path_mtu  = mtu,
                ^
machine-ibverbs.c:815:16: note: (near initialization for ‘attr’)
machine-ibverbs.c:816:3: error: unknown field ‘dest_qp_num’ specified in 
initializer
   .dest_qp_num  = addr[1],
   ^
machine-ibverbs.c:816:19: warning: excess elements in struct initializer
   .dest_qp_num  = addr[1],
                   ^
machine-ibverbs.c:816:19: note: (near initialization for ‘attr’)
machine-ibverbs.c:817:3: error: unknown field ‘rq_psn’ specified in 
initializer
   .rq_psn   = addr[2],
   ^
machine-ibverbs.c:817:15: warning: excess elements in struct initializer
   .rq_psn   = addr[2],
               ^
machine-ibverbs.c:817:15: note: (near initialization for ‘attr’)
machine-ibverbs.c:818:3: error: unknown field ‘max_dest_rd_atomic’ 
specified in initializer
   .max_dest_rd_atomic = 1,
   ^
machine-ibverbs.c:818:25: warning: excess elements in struct initializer
   .max_dest_rd_atomic = 1,
                         ^
machine-ibverbs.c:818:25: note: (near initialization for ‘attr’)
machine-ibverbs.c:819:3: error: unknown field ‘min_rnr_timer’ specified 
in initializer
   .min_rnr_timer  = 31,
   ^
machine-ibverbs.c:819:21: warning: excess elements in struct initializer
   .min_rnr_timer  = 31,
                     ^
machine-ibverbs.c:819:21: note: (near initialization for ‘attr’)
machine-ibverbs.c:820:3: error: unknown field ‘ah_attr’ specified in 
initializer
   .ah_attr  = {
   ^
machine-ibverbs.c:820:15: error: extra brace group at end of initializer
   .ah_attr  = {
               ^
machine-ibverbs.c:820:15: note: (near initialization for ‘attr’)
machine-ibverbs.c:820:15: warning: excess elements in struct initializer
machine-ibverbs.c:820:15: note: (near initialization for ‘attr’)
machine-ibverbs.c:813:21: error: storage size of ‘attr’ isn’t known
  struct ibv_qp_attr attr = {
                     ^
machine-ibverbs.c:833:4: error: ‘IBV_QP_STATE’ undeclared (first use in 
this function)
    IBV_QP_STATE              |
    ^
machine-ibverbs.c:834:4: error: ‘IBV_QP_AV’ undeclared (first use in this 
function)
    IBV_QP_AV                 |
    ^
machine-ibverbs.c:835:4: error: ‘IBV_QP_PATH_MTU’ undeclared (first use 
in this function)
    IBV_QP_PATH_MTU           |
    ^
machine-ibverbs.c:836:4: error: ‘IBV_QP_DEST_QPN’ undeclared (first use 
in this function)
    IBV_QP_DEST_QPN           |
    ^
machine-ibverbs.c:837:4: error: ‘IBV_QP_RQ_PSN’ undeclared (first use in 
this function)
    IBV_QP_RQ_PSN             |
    ^
machine-ibverbs.c:838:4: error: ‘IBV_QP_MAX_DEST_RD_ATOMIC’ undeclared 
(first use in this function)
    IBV_QP_MAX_DEST_RD_ATOMIC |
    ^
machine-ibverbs.c:839:4: error: ‘IBV_QP_MIN_RNR_TIMER’ undeclared (first 
use in this function)
    IBV_QP_MIN_RNR_TIMER)) {
    ^
machine-ibverbs.c:846:23: error: ‘IBV_QPS_RTS’ undeclared (first use in 
this function)
  attr.qp_state      = IBV_QPS_RTS;
                       ^
machine-ibverbs.c:860:4: error: ‘IBV_QP_TIMEOUT’ undeclared (first use in 
this function)
    IBV_QP_TIMEOUT            |
    ^
machine-ibverbs.c:861:4: error: ‘IBV_QP_RETRY_CNT’ undeclared (first use 
in this function)
    IBV_QP_RETRY_CNT          |
    ^
machine-ibverbs.c:862:4: error: ‘IBV_QP_RNR_RETRY’ undeclared (first use 
in this function)
    IBV_QP_RNR_RETRY          |
    ^
machine-ibverbs.c:863:4: error: ‘IBV_QP_SQ_PSN’ undeclared (first use in 
this function)
    IBV_QP_SQ_PSN             |
    ^
machine-ibverbs.c:864:4: error: ‘IBV_QP_MAX_QP_RD_ATOMIC’ undeclared 
(first use in this function)
    IBV_QP_MAX_QP_RD_ATOMIC)) {
    ^
machine-ibverbs.c:877:17: error: ‘IBV_MTU_4096’ undeclared (first use in 
this function)
           mtu = IBV_MTU_4096;
                 ^
machine-ibverbs.c:877:11: error: ‘mtu’ has an incomplete type
           mtu = IBV_MTU_4096;
           ^
machine-ibverbs.c: In function ‘allocateInfiBufferPool’:
machine-ibverbs.c:952:49: error: ‘IBV_ACCESS_LOCAL_WRITE’ undeclared 
(first use in this function)
  bigKey = ibv_reg_mr(context->pd,bigBuf,bigSize,IBV_ACCESS_LOCAL_WRITE);
                                                 ^
machine-ibverbs.c: In function ‘postInitialRecvs’:
machine-ibverbs.c:984:51: error: invalid application of ‘sizeof’ to 
incomplete type ‘struct ibv_recv_wr’
  struct ibv_recv_wr *workRequests = malloc(sizeof(struct 
ibv_recv_wr)*numRecvs);
                                                   ^
machine-ibverbs.c:985:45: error: invalid application of ‘sizeof’ to 
incomplete type ‘struct ibv_sge’
  struct ibv_sge *sgElements = malloc(sizeof(struct ibv_sge)*numRecvs);
                                             ^
machine-ibverbs.c:993:3: error: invalid use of undefined type ‘struct 
ibv_sge’
   sgElements[j].addr = (uint64_t) 
recvBufferPool->buffers[startBufferIdx+j].buf;
   ^
machine-ibverbs.c:993:13: error: dereferencing pointer to incomplete type 
‘struct ibv_sge’
   sgElements[j].addr = (uint64_t) 
recvBufferPool->buffers[startBufferIdx+j].buf;
             ^
machine-ibverbs.c:994:3: error: invalid use of undefined type ‘struct 
ibv_sge’
   sgElements[j].length = sizePerBuffer;
   ^
machine-ibverbs.c:995:3: error: invalid use of undefined type ‘struct 
ibv_sge’
   sgElements[j].lkey = recvBufferPool->buffers[startBufferIdx+j].key->lkey;
   ^
machine-ibverbs.c:997:3: error: invalid use of undefined type ‘struct 
ibv_recv_wr’
   workRequests[j].wr_id = 
(uint64_t)&(recvBufferPool->buffers[startBufferIdx+j]);
   ^
machine-ibverbs.c:997:15: error: dereferencing pointer to incomplete type 
‘struct ibv_recv_wr’
   workRequests[j].wr_id = 
(uint64_t)&(recvBufferPool->buffers[startBufferIdx+j]);
               ^
machine-ibverbs.c:998:3: error: invalid use of undefined type ‘struct 
ibv_recv_wr’
   workRequests[j].sg_list = &sgElements[j];
   ^
machine-ibverbs.c:998:3: error: invalid use of undefined type ‘struct 
ibv_sge’
machine-ibverbs.c:999:3: error: invalid use of undefined type ‘struct 
ibv_recv_wr’
   workRequests[j].num_sge = 1;
   ^
machine-ibverbs.c:1001:4: error: invalid use of undefined type ‘struct 
ibv_recv_wr’
    workRequests[j].next = &workRequests[j+1];
    ^
machine-ibverbs.c:1001:4: error: invalid use of undefined type ‘struct 
ibv_recv_wr’
machine-ibverbs.c:1001:4: error: invalid use of undefined type ‘struct 
ibv_recv_wr’
machine-ibverbs.c:1005:2: error: invalid use of undefined type ‘struct 
ibv_recv_wr’
  workRequests[numRecvs-1].next = NULL;
  ^
machine-ibverbs.c:1005:2: error: invalid use of undefined type ‘struct 
ibv_recv_wr’
machine-ibverbs.c:1007:5: warning: implicit declaration of function 
‘ibv_post_srq_recv’ [-Wimplicit-function-declaration]
  if(ibv_post_srq_recv(context->srq,workRequests,&bad_wr)){
     ^
In file included from machine-dgram.c:640:0,
                 from machine.c:1397:
machine-ibverbs.c: In function ‘EnqueuePacket’:
machine-ibverbs.c:1094:14: warning: implicit declaration of function 
‘ibv_post_send’ [-Wimplicit-function-declaration]
  if(retval = ibv_post_send(node->infiData->qp,&(packet->wr),&bad_wr)){
              ^
machine-ibverbs.c: At top level:
machine-ibverbs.c:1250:41: warning: ‘struct ibv_wc’ declared inside 
parameter list
 static inline void processRecvWC(struct ibv_wc *recvWC,const int toBuffer);
                                         ^
machine-ibverbs.c:1251:41: warning: ‘struct ibv_wc’ declared inside 
parameter list
 static inline void processSendWC(struct ibv_wc *sendWC);
                                         ^
machine-ibverbs.c: In function ‘processAsyncEvents’:
machine-ibverbs.c:1256:25: error: storage size of ‘event’ isn’t known
  struct ibv_async_event event;
                         ^
In file included from /usr/include/x86_64-linux-gnu/sys/types.h:219:0,
                 from /usr/include/stdlib.h:314,
                 from converse.h:58,
                 from machine.c:202:
machine-ibverbs.c:1263:25: error: dereferencing pointer to incomplete type 
‘struct ibv_context’
  FD_SET(context->context->async_fd,&context->asyncFds);
                         ^
In file included from machine-dgram.c:640:0,
                 from machine.c:1397:
machine-ibverbs.c:1275:6: warning: implicit declaration of function 
‘ibv_get_async_event’ [-Wimplicit-function-declaration]
  if (ibv_get_async_event(context->context, &event)){
      ^
machine-ibverbs.c:1280:2: warning: implicit declaration of function 
‘ibv_ack_async_event’ [-Wimplicit-function-declaration]
  ibv_ack_async_event(&event);
  ^
machine-ibverbs.c: In function ‘pollRecvCq’:
machine-ibverbs.c:1392:16: error: array type has incomplete element type 
‘struct ibv_wc’
  struct ibv_wc wc[WC_LIST_SIZE];
                ^
machine-ibverbs.c:1395:7: warning: implicit declaration of function 
‘ibv_poll_cq’ [-Wimplicit-function-declaration]
  ne = ibv_poll_cq(context->recvCq,WC_LIST_SIZE,&wc[0]);
       ^
machine-ibverbs.c:1403:22: error: ‘IBV_WC_SUCCESS’ undeclared (first use 
in this function)
   if(wc[i].status != IBV_WC_SUCCESS){
                      ^
machine-ibverbs.c:1407:9: error: ‘IBV_WC_RECV’ undeclared (first use in 
this function)
    case IBV_WC_RECV:
         ^
machine-ibverbs.c: At top level:
machine-ibverbs.c:1421:42: warning: ‘struct ibv_wc’ declared inside 
parameter list
 static inline  void processRdmaWC(struct ibv_wc *rdmaWC,const int toBuffer);
                                          ^
machine-ibverbs.c: In function ‘pollSendCq’:
machine-ibverbs.c:1426:16: error: array type has incomplete element type 
‘struct ibv_wc’
  struct ibv_wc wc[WC_LIST_SIZE];
                ^
machine-ibverbs.c:1433:22: error: ‘IBV_WC_SUCCESS’ undeclared (first use 
in this function)
   if(wc[i].status != IBV_WC_SUCCESS){
                      ^
machine-ibverbs.c:1441:9: error: ‘IBV_WC_SEND’ undeclared (first use in 
this function)
    case IBV_WC_SEND:{
         ^
machine-ibverbs.c:1447:9: error: ‘IBV_WC_RDMA_READ’ undeclared (first use 
in this function)
    case IBV_WC_RDMA_READ:
         ^
machine-ibverbs.c:1453:9: error: ‘IBV_WC_RDMA_WRITE’ undeclared (first 
use in this function)
    case IBV_WC_RDMA_WRITE:
         ^
machine-ibverbs.c: At top level:
machine-ibverbs.c:1662:41: warning: ‘struct ibv_wc’ declared inside 
parameter list
 static inline void processRecvWC(struct ibv_wc *recvWC,const int toBuffer){
                                         ^
machine-ibverbs.c:1662:20: error: conflicting types for ‘processRecvWC’
 static inline void processRecvWC(struct ibv_wc *recvWC,const int toBuffer){
                    ^
machine-ibverbs.c:1250:20: note: previous declaration of ‘processRecvWC’ 
was here
 static inline void processRecvWC(struct ibv_wc *recvWC,const int toBuffer);
                    ^
machine-ibverbs.c: In function ‘processRecvWC’:
machine-ibverbs.c:1663:58: error: dereferencing pointer to incomplete type 
‘struct ibv_wc’
  struct infiBuffer *buffer = (struct infiBuffer *) recvWC->wr_id; 
                                                          ^
machine-ibverbs.c:1737:10: error: variable ‘list’ has initializer but 
incomplete type
   struct ibv_sge list = {
          ^
machine-ibverbs.c:1738:4: error: unknown field ‘addr’ specified in 
initializer
    .addr  = (uintptr_t) buffer->buf,
    ^
machine-ibverbs.c:1738:13: warning: excess elements in struct initializer
    .addr  = (uintptr_t) buffer->buf,
             ^
machine-ibverbs.c:1738:13: note: (near initialization for ‘list’)
machine-ibverbs.c:1739:4: error: unknown field ‘length’ specified in 
initializer
    .length = buffer->size,
    ^
machine-ibverbs.c:1739:14: warning: excess elements in struct initializer
    .length = buffer->size,
              ^
machine-ibverbs.c:1739:14: note: (near initialization for ‘list’)
machine-ibverbs.c:1740:4: error: unknown field ‘lkey’ specified in 
initializer
    .lkey  = buffer->key->lkey
    ^
machine-ibverbs.c:1740:13: warning: excess elements in struct initializer
    .lkey  = buffer->key->lkey
             ^
machine-ibverbs.c:1740:13: note: (near initialization for ‘list’)
machine-ibverbs.c:1737:18: error: storage size of ‘list’ isn’t known
   struct ibv_sge list = {
                  ^
machine-ibverbs.c:1743:10: error: variable ‘wr’ has initializer but 
incomplete type
   struct ibv_recv_wr wr = {
          ^
machine-ibverbs.c:1744:4: error: unknown field ‘wr_id’ specified in 
initializer
    .wr_id = (uint64_t)buffer,
    ^
machine-ibverbs.c:1744:13: warning: excess elements in struct initializer
    .wr_id = (uint64_t)buffer,
             ^
machine-ibverbs.c:1744:13: note: (near initialization for ‘wr’)
machine-ibverbs.c:1745:4: error: unknown field ‘sg_list’ specified in 
initializer
    .sg_list = &list,
    ^
machine-ibverbs.c:1745:15: warning: excess elements in struct initializer
    .sg_list = &list,
               ^
machine-ibverbs.c:1745:15: note: (near initialization for ‘wr’)
machine-ibverbs.c:1746:4: error: unknown field ‘num_sge’ specified in 
initializer
    .num_sge = 1,
    ^
machine-ibverbs.c:1746:15: warning: excess elements in struct initializer
    .num_sge = 1,
               ^
machine-ibverbs.c:1746:15: note: (near initialization for ‘wr’)
machine-ibverbs.c:1747:4: error: unknown field ‘next’ specified in 
initializer
    .next = NULL
    ^
machine-ibverbs.c:1743:22: error: storage size of ‘wr’ isn’t known
   struct ibv_recv_wr wr = {
                      ^
machine-ibverbs.c: At top level:
machine-ibverbs.c:1761:42: warning: ‘struct ibv_wc’ declared inside 
parameter list
 static inline  void processSendWC(struct ibv_wc *sendWC){
                                          ^
machine-ibverbs.c:1761:21: error: conflicting types for ‘processSendWC’
 static inline  void processSendWC(struct ibv_wc *sendWC){
                     ^
machine-ibverbs.c:1251:20: note: previous declaration of ‘processSendWC’ 
was here
 static inline void processSendWC(struct ibv_wc *sendWC);
                    ^
machine-ibverbs.c: In function ‘processSendWC’:
machine-ibverbs.c:1763:41: error: dereferencing pointer to incomplete type 
‘struct ibv_wc’
  infiPacket packet = (infiPacket )sendWC->wr_id;
                                         ^
machine-ibverbs.c: In function ‘processRdmaRequest’:
machine-ibverbs.c:1832:10: error: variable ‘list’ has initializer but 
incomplete type
   struct ibv_sge list = {
          ^
machine-ibverbs.c:1833:4: error: unknown field ‘addr’ specified in 
initializer
    .addr = (uintptr_t )buffer->buf,
    ^
machine-ibverbs.c:1833:12: warning: excess elements in struct initializer
    .addr = (uintptr_t )buffer->buf,
            ^
machine-ibverbs.c:1833:12: note: (near initialization for ‘list’)
machine-ibverbs.c:1834:4: error: unknown field ‘length’ specified in 
initializer
    .length = buffer->size,
    ^
machine-ibverbs.c:1834:14: warning: excess elements in struct initializer
    .length = buffer->size,
              ^
machine-ibverbs.c:1834:14: note: (near initialization for ‘list’)
machine-ibverbs.c:1835:4: error: unknown field ‘lkey’ specified in 
initializer
    .lkey  = buffer->key->lkey
    ^
machine-ibverbs.c:1835:13: warning: excess elements in struct initializer
    .lkey  = buffer->key->lkey
             ^
machine-ibverbs.c:1835:13: note: (near initialization for ‘list’)
machine-ibverbs.c:1832:18: error: storage size of ‘list’ isn’t known
   struct ibv_sge list = {
                  ^
machine-ibverbs.c:1839:10: error: variable ‘wr’ has initializer but 
incomplete type
   struct ibv_send_wr wr = {
          ^
machine-ibverbs.c:1840:4: error: unknown field ‘wr_id’ specified in 
initializer
    .wr_id = (uint64_t )rdmaPacket,
    ^
machine-ibverbs.c:1840:13: warning: excess elements in struct initializer
    .wr_id = (uint64_t )rdmaPacket,
             ^
machine-ibverbs.c:1840:13: note: (near initialization for ‘wr’)
machine-ibverbs.c:1841:4: error: unknown field ‘sg_list’ specified in 
initializer
    .sg_list = &list,
    ^
machine-ibverbs.c:1841:15: warning: excess elements in struct initializer
    .sg_list = &list,
               ^
machine-ibverbs.c:1841:15: note: (near initialization for ‘wr’)
machine-ibverbs.c:1842:4: error: unknown field ‘num_sge’ specified in 
initializer
    .num_sge = 1,
    ^
machine-ibverbs.c:1842:15: warning: excess elements in struct initializer
    .num_sge = 1,
               ^
machine-ibverbs.c:1842:15: note: (near initialization for ‘wr’)
machine-ibverbs.c:1843:4: error: unknown field ‘opcode’ specified in 
initializer
    .opcode = IBV_WR_RDMA_READ,
    ^
machine-ibverbs.c:1843:14: error: ‘IBV_WR_RDMA_READ’ undeclared (first 
use in this function)
    .opcode = IBV_WR_RDMA_READ,
              ^
machine-ibverbs.c:1843:14: warning: excess elements in struct initializer
machine-ibverbs.c:1843:14: note: (near initialization for ‘wr’)
machine-ibverbs.c:1844:4: error: unknown field ‘send_flags’ specified in 
initializer
    .send_flags = IBV_SEND_SIGNALED,
    ^
machine-ibverbs.c:1844:18: error: ‘IBV_SEND_SIGNALED’ undeclared (first 
use in this function)
    .send_flags = IBV_SEND_SIGNALED,
                  ^
machine-ibverbs.c:1844:18: warning: excess elements in struct initializer
machine-ibverbs.c:1844:18: note: (near initialization for ‘wr’)
machine-ibverbs.c:1845:4: error: unknown field ‘wr’ specified in 
initializer
    .wr.rdma = {
    ^
machine-ibverbs.c:1845:15: error: extra brace group at end of initializer
    .wr.rdma = {
               ^
machine-ibverbs.c:1845:15: note: (near initialization for ‘wr’)
machine-ibverbs.c:1845:15: warning: excess elements in struct initializer
machine-ibverbs.c:1845:15: note: (near initialization for ‘wr’)
machine-ibverbs.c:1839:22: error: storage size of ‘wr’ isn’t known
   struct ibv_send_wr wr = {
                      ^
machine-ibverbs.c: At top level:
machine-ibverbs.c:1861:42: warning: ‘struct ibv_wc’ declared inside 
parameter list
 static inline  void processRdmaWC(struct ibv_wc *rdmaWC,const int toBuffer){
                                          ^
machine-ibverbs.c:1861:21: error: conflicting types for ‘processRdmaWC’
 static inline  void processRdmaWC(struct ibv_wc *rdmaWC,const int toBuffer){
                     ^
machine-ibverbs.c:1421:21: note: previous declaration of ‘processRdmaWC’ 
was here
 static inline  void processRdmaWC(struct ibv_wc *rdmaWC,const int toBuffer);
                     ^
machine-ibverbs.c: In function ‘processRdmaWC’:
machine-ibverbs.c:1867:70: error: dereferencing pointer to incomplete type 
‘struct ibv_wc’
  struct infiRdmaPacket *rdmaPacket = (struct infiRdmaPacket *) rdmaWC->wr_id;
                                                                      ^
machine-ibverbs.c: In function ‘increaseTokens’:
machine-ibverbs.c:2141:5: warning: implicit declaration of function 
‘ibv_resize_cq’ [-Wimplicit-function-declaration]
  if(ibv_resize_cq(context->sendCq,currentCqSize+increase)){
     ^
machine-ibverbs.c: In function ‘registerMultiSendMesg’:
machine-ibverbs.c:2240:79: error: ‘IBV_ACCESS_REMOTE_READ’ undeclared 
(first use in this function)
  metaData->key = 
ibv_reg_mr(context->pd,res,(size+sizeof(infiCmiChunkHeader)),IBV_ACCESS_REMOTE_READ
 | IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
                                                                              
 ^
machine-ibverbs.c:2240:104: error: ‘IBV_ACCESS_LOCAL_WRITE’ undeclared 
(first use in this function)
  metaData->key = 
ibv_reg_mr(context->pd,res,(size+sizeof(infiCmiChunkHeader)),IBV_ACCESS_REMOTE_READ
 | IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
                                                                              
                          ^
machine-ibverbs.c:2240:129: error: ‘IBV_ACCESS_REMOTE_WRITE’ undeclared 
(first use in this function)
  metaData->key = 
ibv_reg_mr(context->pd,res,(size+sizeof(infiCmiChunkHeader)),IBV_ACCESS_REMOTE_READ
 | IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
                                                                              
                                                   ^
machine-ibverbs.c: In function ‘getInfiCmiChunk’:
machine-ibverbs.c:2469:32: warning: passing argument 1 of 
‘posix_memalign’ from incompatible pointer type 
[-Wincompatible-pointer-types]
                 posix_memalign(&res, ALIGN_BYTES, 
(allocSize+sizeof(infiCmiChunkHeader))*count);
                                ^
In file included from converse.h:58:0,
                 from machine.c:202:
/usr/include/stdlib.h:503:12: note: expected ‘void **’ but argument is of 
type ‘char **’
 extern int posix_memalign (void **__memptr, size_t __alignment, size_t 
__size)
            ^
In file included from machine-dgram.c:640:0,
                 from machine.c:1397:
machine-ibverbs.c:2472:95: error: ‘IBV_ACCESS_REMOTE_READ’ undeclared 
(first use in this function)
                 key = 
ibv_reg_mr(context->pd,res,(allocSize+sizeof(infiCmiChunkHeader))*count,IBV_ACCESS_REMOTE_READ
 | IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
                                                                              
                 ^
machine-ibverbs.c:2472:120: error: ‘IBV_ACCESS_LOCAL_WRITE’ undeclared 
(first use in this function)
                 key = 
ibv_reg_mr(context->pd,res,(allocSize+sizeof(infiCmiChunkHeader))*count,IBV_ACCESS_REMOTE_READ
 | IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
                                                                              
                                          ^
machine-ibverbs.c:2472:145: error: ‘IBV_ACCESS_REMOTE_WRITE’ undeclared 
(first use in this function)
                 key = 
ibv_reg_mr(context->pd,res,(allocSize+sizeof(infiCmiChunkHeader))*count,IBV_ACCESS_REMOTE_READ
 | IBV_ACCESS_LOCAL_WRITE | IBV_ACCESS_REMOTE_WRITE);
                                                                              
                                                                   ^
machine-ibverbs.c: In function ‘infi_CmiAlloc’:
machine-ibverbs.c:2538:26: warning: passing argument 1 of 
‘posix_memalign’ from incompatible pointer type 
[-Wincompatible-pointer-types]
           posix_memalign(&res, ALIGN_BYTES, size + sizeof(void*));
                          ^
In file included from converse.h:58:0,
                 from machine.c:202:
/usr/include/stdlib.h:503:12: note: expected ‘void **’ but argument is of 
type ‘char **’
 extern int posix_memalign (void **__memptr, size_t __alignment, size_t 
__size)
            ^
In file included from machine-dgram.c:640:0,
                 from machine.c:1397:
machine-ibverbs.c: At top level:
machine-ibverbs.c:2775:16: error: field ‘senderKey’ has incomplete type
  struct ibv_mr senderKey;
                ^
In file included from machine-dgram.c:640:0,
                 from machine.c:1397:
machine-ibverbs.c: In function ‘CmiDirect_createHandle’:
machine-ibverbs.c:2921:73: error: ‘IBV_ACCESS_REMOTE_READ’ undeclared 
(first use in this function)
  table->handles[idx].key = ibv_reg_mr(context->pd, recvBuf, 
recvBufSize,IBV_ACCESS_REMOTE_READ | IBV_ACCESS_LOCAL_WRITE | 
IBV_ACCESS_REMOTE_WRITE);
                                                                         ^
machine-ibverbs.c:2921:98: error: ‘IBV_ACCESS_LOCAL_WRITE’ undeclared 
(first use in this function)
  table->handles[idx].key = ibv_reg_mr(context->pd, recvBuf, 
recvBufSize,IBV_ACCESS_REMOTE_READ | IBV_ACCESS_LOCAL_WRITE | 
IBV_ACCESS_REMOTE_WRITE);
                                                                              
                    ^
machine-ibverbs.c:2921:123: error: ‘IBV_ACCESS_REMOTE_WRITE’ undeclared 
(first use in this function)
  table->handles[idx].key = ibv_reg_mr(context->pd, recvBuf, 
recvBufSize,IBV_ACCESS_REMOTE_READ | IBV_ACCESS_LOCAL_WRITE | 
IBV_ACCESS_REMOTE_WRITE);
                                                                              
                                             ^
machine-ibverbs.c:2936:61: error: invalid application of ‘sizeof’ to 
incomplete type ‘struct ibv_mr’
  memcpy(userHandle.recverKey,table->handles[idx].key,sizeof(struct ibv_mr));
                                                             ^
machine-ibverbs.c: In function ‘CmiDirect_assocLocalBuffer’:
machine-ibverbs.c:2995:74: error: ‘IBV_ACCESS_REMOTE_READ’ undeclared 
(first use in this function)
  table->handles[idx].key =  ibv_reg_mr(context->pd, sendBuf, 
sendBufSize,IBV_ACCESS_REMOTE_READ | IBV_ACCESS_LOCAL_WRITE | 
IBV_ACCESS_REMOTE_WRITE);
                                                                          ^
machine-ibverbs.c:2995:99: error: ‘IBV_ACCESS_LOCAL_WRITE’ undeclared 
(first use in this function)
  table->handles[idx].key =  ibv_reg_mr(context->pd, sendBuf, 
sendBufSize,IBV_ACCESS_REMOTE_READ | IBV_ACCESS_LOCAL_WRITE | 
IBV_ACCESS_REMOTE_WRITE);
                                                                              
                     ^
machine-ibverbs.c:2995:124: error: ‘IBV_ACCESS_REMOTE_WRITE’ undeclared 
(first use in this function)
  table->handles[idx].key =  ibv_reg_mr(context->pd, sendBuf, 
sendBufSize,IBV_ACCESS_REMOTE_READ | IBV_ACCESS_LOCAL_WRITE | 
IBV_ACCESS_REMOTE_WRITE);
                                                                              
                                              ^
machine-ibverbs.c: In function ‘CmiDirect_put’:
machine-ibverbs.c:3089:11: error: variable ‘list’ has initializer but 
incomplete type
    struct ibv_sge list = {
           ^
machine-ibverbs.c:3090:5: error: unknown field ‘addr’ specified in 
initializer
     .addr = (uintptr_t )table->handles[idx].buf,
     ^
machine-ibverbs.c:3090:13: warning: excess elements in struct initializer
     .addr = (uintptr_t )table->handles[idx].buf,
             ^
machine-ibverbs.c:3090:13: note: (near initialization for ‘list’)
machine-ibverbs.c:3091:5: error: unknown field ‘length’ specified in 
initializer
     .length = table->handles[idx].size,
     ^
machine-ibverbs.c:3091:15: warning: excess elements in struct initializer
     .length = table->handles[idx].size,
               ^
machine-ibverbs.c:3091:15: note: (near initialization for ‘list’)
machine-ibverbs.c:3092:5: error: unknown field ‘lkey’ specified in 
initializer
     .lkey  = table->handles[idx].key->lkey
     ^
machine-ibverbs.c:3092:14: warning: excess elements in struct initializer
     .lkey  = table->handles[idx].key->lkey
              ^
machine-ibverbs.c:3092:14: note: (near initialization for ‘list’)
machine-ibverbs.c:3089:19: error: storage size of ‘list’ isn’t known
    struct ibv_sge list = {
                   ^
machine-ibverbs.c:3098:11: error: variable ‘wr’ has initializer but 
incomplete type
    struct ibv_send_wr wr = {
           ^
machine-ibverbs.c:3099:5: error: unknown field ‘wr_id’ specified in 
initializer
     .wr_id = (uint64_t)table->handles[idx].rdmaPacket,
     ^
machine-ibverbs.c:3099:14: warning: excess elements in struct initializer
     .wr_id = (uint64_t)table->handles[idx].rdmaPacket,
              ^
machine-ibverbs.c:3099:14: note: (near initialization for ‘wr’)
machine-ibverbs.c:3100:5: error: unknown field ‘sg_list’ specified in 
initializer
     .sg_list = &list,
     ^
machine-ibverbs.c:3100:16: warning: excess elements in struct initializer
     .sg_list = &list,
                ^
machine-ibverbs.c:3100:16: note: (near initialization for ‘wr’)
machine-ibverbs.c:3101:5: error: unknown field ‘num_sge’ specified in 
initializer
     .num_sge = 1,
     ^
machine-ibverbs.c:3101:16: warning: excess elements in struct initializer
     .num_sge = 1,
                ^
machine-ibverbs.c:3101:16: note: (near initialization for ‘wr’)
machine-ibverbs.c:3102:5: error: unknown field ‘opcode’ specified in 
initializer
     .opcode = IBV_WR_RDMA_WRITE,
     ^
machine-ibverbs.c:3102:15: error: ‘IBV_WR_RDMA_WRITE’ undeclared (first 
use in this function)
     .opcode = IBV_WR_RDMA_WRITE,
               ^
machine-ibverbs.c:3102:15: warning: excess elements in struct initializer
machine-ibverbs.c:3102:15: note: (near initialization for ‘wr’)
machine-ibverbs.c:3103:5: error: unknown field ‘send_flags’ specified in 
initializer
     .send_flags = IBV_SEND_SIGNALED,
     ^
machine-ibverbs.c:3103:19: error: ‘IBV_SEND_SIGNALED’ undeclared (first 
use in this function)
     .send_flags = IBV_SEND_SIGNALED,
                   ^
machine-ibverbs.c:3103:19: warning: excess elements in struct initializer
machine-ibverbs.c:3103:19: note: (near initialization for ‘wr’)
machine-ibverbs.c:3105:5: error: unknown field ‘wr’ specified in 
initializer
     .wr.rdma = {
     ^
machine-ibverbs.c:3105:16: error: extra brace group at end of initializer
     .wr.rdma = {
                ^
machine-ibverbs.c:3105:16: note: (near initialization for ‘wr’)
machine-ibverbs.c:3105:16: warning: excess elements in struct initializer
machine-ibverbs.c:3105:16: note: (near initialization for ‘wr’)
machine-ibverbs.c:3098:23: error: storage size of ‘wr’ isn’t known
    struct ibv_send_wr wr = {
                       ^
machine-ibverbs.c: In function ‘CmiMachineCleanup’:
machine-ibverbs.c:3225:9: warning: implicit declaration of function 
‘ibv_dealloc_pd’ [-Wimplicit-function-declaration]
         ibv_dealloc_pd(context->pd);
         ^
machine-ibverbs.c:3226:9: warning: implicit declaration of function 
‘ibv_close_device’ [-Wimplicit-function-declaration]
         ibv_close_device(context->context);
         ^
machine-ibverbs.c:3227:17: warning: assignment makes pointer from integer 
without a cast [-Wint-conversion]
         devList = ibv_get_device_list(&num_devices);
                 ^
machine-ibverbs.c:3228:9: warning: implicit declaration of function 
‘ibv_free_device_list’ [-Wimplicit-function-declaration]
         ibv_free_device_list(devList);
         ^
In file included from machine-ibverbs.c:3237:0,
                 from machine-dgram.c:640,
                 from machine.c:1397:
machine-onesided.c: In function ‘verbsOnesidedPostRdmaRead’:
machine-onesided.c:49:58: error: ‘IBV_ACCESS_LOCAL_WRITE’ undeclared 
(first use in this function)
   mr = ibv_reg_mr(context->pd, (void *)local_addr, size, 
IBV_ACCESS_LOCAL_WRITE |
                                                          ^
machine-onesided.c:50:7: error: ‘IBV_ACCESS_REMOTE_WRITE’ undeclared 
(first use in this function)
       IBV_ACCESS_REMOTE_WRITE | IBV_ACCESS_REMOTE_READ);
       ^
machine-onesided.c:50:33: error: ‘IBV_ACCESS_REMOTE_READ’ undeclared 
(first use in this function)
       IBV_ACCESS_REMOTE_WRITE | IBV_ACCESS_REMOTE_READ);
                                 ^
machine-onesided.c:56:10: error: variable ‘list’ has initializer but 
incomplete type
   struct ibv_sge list = {
          ^
machine-onesided.c:57:5: error: unknown field ‘addr’ specified in 
initializer
     .addr = (uintptr_t)local_addr,
     ^
machine-onesided.c:57:13: warning: excess elements in struct initializer
     .addr = (uintptr_t)local_addr,
             ^
machine-onesided.c:57:13: note: (near initialization for ‘list’)
machine-onesided.c:58:5: error: unknown field ‘length’ specified in 
initializer
     .length = size,
     ^
machine-onesided.c:58:15: warning: excess elements in struct initializer
     .length = size,
               ^
machine-onesided.c:58:15: note: (near initialization for ‘list’)
machine-onesided.c:59:5: error: unknown field ‘lkey’ specified in 
initializer
     .lkey = mr->lkey
     ^
machine-onesided.c:59:13: warning: excess elements in struct initializer
     .lkey = mr->lkey
             ^
machine-onesided.c:59:13: note: (near initialization for ‘list’)
machine-onesided.c:56:18: error: storage size of ‘list’ isn’t known
   struct ibv_sge list = {
                  ^
machine-onesided.c:63:10: error: variable ‘wr’ has initializer but 
incomplete type
   struct ibv_send_wr wr = {
          ^
machine-onesided.c:64:5: error: unknown field ‘wr_id’ specified in 
initializer
     .wr_id = (uint64_t)rdmaPacket,
     ^
machine-onesided.c:64:14: warning: excess elements in struct initializer
     .wr_id = (uint64_t)rdmaPacket,
              ^
machine-onesided.c:64:14: note: (near initialization for ‘wr’)
machine-onesided.c:65:5: error: unknown field ‘sg_list’ specified in 
initializer
     .sg_list = &list,
     ^
machine-onesided.c:65:16: warning: excess elements in struct initializer
     .sg_list = &list,
                ^
machine-onesided.c:65:16: note: (near initialization for ‘wr’)
machine-onesided.c:66:5: error: unknown field ‘num_sge’ specified in 
initializer
     .num_sge = 1,
     ^
machine-onesided.c:66:16: warning: excess elements in struct initializer
     .num_sge = 1,
                ^
machine-onesided.c:66:16: note: (near initialization for ‘wr’)
machine-onesided.c:67:5: error: unknown field ‘opcode’ specified in 
initializer
     .opcode = IBV_WR_RDMA_READ,
     ^
machine-onesided.c:67:15: error: ‘IBV_WR_RDMA_READ’ undeclared (first use 
in this function)
     .opcode = IBV_WR_RDMA_READ,
               ^
machine-onesided.c:67:15: warning: excess elements in struct initializer
machine-onesided.c:67:15: note: (near initialization for ‘wr’)
machine-onesided.c:68:5: error: unknown field ‘send_flags’ specified in 
initializer
     .send_flags = IBV_SEND_SIGNALED,
     ^
machine-onesided.c:68:19: error: ‘IBV_SEND_SIGNALED’ undeclared (first 
use in this function)
     .send_flags = IBV_SEND_SIGNALED,
                   ^
machine-onesided.c:68:19: warning: excess elements in struct initializer
machine-onesided.c:68:19: note: (near initialization for ‘wr’)
machine-onesided.c:69:5: error: unknown field ‘wr’ specified in 
initializer
     .wr.rdma = {
     ^
machine-onesided.c:69:16: error: extra brace group at end of initializer
     .wr.rdma = {
                ^
machine-onesided.c:69:16: note: (near initialization for ‘wr’)
machine-onesided.c:69:16: warning: excess elements in struct initializer
machine-onesided.c:69:16: note: (near initialization for ‘wr’)
machine-onesided.c:63:22: error: storage size of ‘wr’ isn’t known
   struct ibv_send_wr wr = {
                      ^
In file included from machine-dgram.c:640:0,
                 from machine.c:1397:
machine.c: At top level:
machine-ibverbs.c:1250:20: warning: ‘processRecvWC’ used but never defined
 static inline void processRecvWC(struct ibv_wc *recvWC,const int toBuffer);
                    ^
machine-ibverbs.c:1251:20: warning: ‘processSendWC’ used but never defined
 static inline void processSendWC(struct ibv_wc *sendWC);
                    ^
machine-ibverbs.c:1421:21: warning: ‘processRdmaWC’ used but never defined
 static inline  void processRdmaWC(struct ibv_wc *rdmaWC,const int toBuffer);
                     ^
Fatal Error by charmc in directory 
/home/sutirtha/NAMD_Git-2017-08-03_Source/charm-6.8.0-rc1/verbs-linux-x86_64-gcc/tmp
   Command gcc -I../bin/../include -D__CHARMC__=1 -I. -DFOR_CPLUS=1 -O2 
-U_FORTIFY_SOURCE -fno-stack-protector -c machine.c -o machine.o returned 
error code 1
charmc exiting...
Makefile:468: recipe for target '../lib/libconv-cplus-n.a' failed
make: *** [../lib/libconv-cplus-n.a] Error 1
-------------------------------------------------
Charm++ NOT BUILT. Either cd into verbs-linux-x86_64-gcc/tmp and try
to resolve the problems yourself, visit
http://charm.cs.illinois.edu/
for more information. Otherwise, email the developers at 
charm AT cs.illinois.edu

convcore.c: In function ‘CmiGetArgLongDesc’:
convcore.c:501:19: warning: format ‘%lld’ expects argument of type 
‘long long int *’, but argument 3 has type ‘CmiInt8 * {aka long int 
*}’ [-Wformat=]
    if (sscanf(opt,"%lld",optDest)<1) {
                   ^
../bin/charmc  -optimize -production   -I.   -c -o conv-conds.o conv-conds.c
../bin/charmc  -optimize -production   -I.   -c -o queueing.o queueing.C
../bin/charmc  -optimize -production   -I.   -c -o msgmgr.o msgmgr.c
../bin/charmc  -optimize -production   -I.   -c -o cpm.o cpm.c
../bin/charmc  -optimize -production   -I.   -c -o cpthreads.o cpthreads.c
../bin/charmc  -optimize -production   -I.   -c -o futures.o futures.c
../bin/charmc  -optimize -production   -I.   -c -o cldb.o cldb.c
../bin/charmc  -optimize -production   -I.   -c -o random.o random.c
../bin/charmc  -optimize -production   -I.   -c -o debug-conv.o debug-conv.c
../bin/charmc  -optimize -production   -I.   -c -o debug-conv++.o 
debug-conv++.C
../bin/charmc  -optimize -production   -I.   -c -o conv-ccs.o conv-ccs.c
../bin/charmc  -optimize -production   -I.   -c -o ccs-builtins.o 
ccs-builtins.C
../bin/charmc  -optimize -production   -I.   -c -o middle-ccs.o middle-ccs.C
../bin/charmc  -optimize -production   -I.   -c -o traceCore.o traceCore.C
../bin/charmc  -optimize -production   -I.   -c -o traceCoreCommon.o 
traceCoreCommon.C
../bin/charmc  -optimize -production   -I.   -c -o converseProjections.o 
converseProjections.C
../bin/charmc  -optimize -production   -I.   -c -o machineProjections.o 
machineProjections.C
../bin/charmc  -optimize -production   -I.   -c -o quiescence.o quiescence.c
../bin/charmc  -optimize -production   -I.   -c -o isomalloc.o isomalloc.c
../bin/charmc  -optimize -production   -I.   -c -o mem-arena.o mem-arena.c
../bin/charmc  -optimize -production   -I.   -c -o conv-counter.o 
conv-counter.c
../bin/charmc  -optimize -production   -I.   -c -o memory-darwin-clang.o 
memory-darwin-clang.C
../bin/charmc  -optimize -production   -I.   -c -o global-nop.o global-nop.c
../bin/charmc  -optimize -production   -I.   -c -o cmipool.o cmipool.c
../bin/charmc  -optimize -production   -I.   -c -o cpuaffinity.o cpuaffinity.c
../bin/charmc  -optimize -production   -I.   -c -o cputopology.o cputopology.C
../bin/charmc  -optimize -production   -I.   -c -o cmitls.o cmitls.c
../bin/charmc  -optimize -production   -I.   -c -o memoryaffinity.o 
memoryaffinity.c
SRCBASE=../../src ./commitid.sh
Copying VERSION.new = 
v6.8.0-beta1-314-g4decdbf-namd-charm-6.8.0-build-2017-Jul-18-4809 over 
VERSION =
echo "const char * const CmiCommitID = \"`cat VERSION`\";" > commitid.c
../bin/charmc  -optimize -production   -I.   -c -o commitid.o commitid.c
../bin/charmc  -optimize -production   -I.   -c -o sdag.o sdag.C
../bin/charmc  -optimize -production   -I.   -c -o conv-interoperate.o 
conv-interoperate.C
../bin/charmc  -optimize -production    -o ../lib/libconv-core.a convcore.o 
conv-conds.o queueing.o msgmgr.o cpm.o cpthreads.o futures.o cldb.o random.o 
debug-conv.o debug-conv++.o conv-ccs.o ccs-builtins.o middle-ccs.o 
traceCore.o traceCoreCommon.o converseProjections.o machineProjections.o 
quiescence.o isomalloc.o mem-arena.o conv-counter.o memory-darwin-clang.o 
global-nop.o cmipool.o cpuaffinity.o cputopology.o cmitls.o memoryaffinity.o 
commitid.o sdag.o conv-interoperate.o 
ar: creating ../lib/libconv-core.a
../bin/charmc  -optimize -production   -I. -o machine.o  -DFOR_CPLUS=1 
machine.c
machine.c: In function ‘machine_exit’:
machine.c:327:3: warning: implicit declaration of function 
‘EmergencyExit’ [-Wimplicit-function-declaration]
   EmergencyExit();
   ^
machine.c: At top level:
machine.c:865:1: warning: data definition has no type or storage class
 _Cmi_myrank=0;
 ^
machine.c:865:1: warning: type defaults to ‘int’ in declaration of 
‘_Cmi_myrank’ [-Wimplicit-int]
In file included from machine.c:1397:0:
machine-dgram.c:20:30: fatal error: infiniband/verbs.h: No such file or 
directory
compilation terminated.
Fatal Error by charmc in directory 
/home/sutirtha/NAMD_Git-2017-08-03_Source/charm-6.8.0-rc1/verbs-linux-x86_64-gcc/tmp
   Command gcc -I../bin/../include -D__CHARMC__=1 -I. -DFOR_CPLUS=1 -O2 
-U_FORTIFY_SOURCE -fno-stack-protector -c machine.c -o machine.o returned 
error code 1
charmc exiting...
Makefile:468: recipe for target '../lib/libconv-cplus-n.a' failed
make: *** [../lib/libconv-cplus-n.a] Error 1
-------------------------------------------------
Charm++ NOT BUILT. Either cd into verbs-linux-x86_64-gcc/tmp and try
to resolve the problems yourself, visit
http://charm.cs.illinois.edu/
for more information. Otherwise, email the developers at 
charm AT cs.illinois.edu