Charm++ parallel programming system

[Nikhil Jain <nikhil.jain AT acm.org>]

Hi Laercio,

Thanks for bringing this to our attention. May I know which versions of GNU’s gcc and ld are you working with? 

I think this is a new issue caused due to change in linker’s behavior. I compared the binary output generated by gcc 4.6/ld 2.22 with that of gcc 4.8/ld 2.24 for creating a dynamic executable. In the former, global variables such as myRank in your test case are listed in the dynamic relocation record, while in the latter they aren’t. swapglobal depends on modifying these records to point to the right set of copy of global variables.

I have created a redmine issue for the bug: https://charm.cs.illinois.edu/redmine/issues/965. Hope we will be able to find a resolution soon.

—Nikhil

--

Nikhil Jain, nikhil.jain AT acm.org, http://charm.cs.uiuc.edu/people/nikhil
Doctoral Candidate @ CS, UIUC

On January 29, 2016 at 12:09:45, Laércio Lima Pilla (laercio.pilla AT ufsc.br) wrote:

Dear all,

I have been trying to use -swapglobals for some simple tests in different Linux machines (both virtual and physical ones) to no avail.

For instance, compiling and linking with -swapglobals with the code below still results in the same address for myRank for all VPs, even thought I get the message "Charm++> -swapglobals enabled".

Any known bug/limitation or idea of what is going wrong?

#include <stdio.h>
#include <stdlib.h>
#include <mpi.h>
#include "charm.h"
#include "tcharmc.h"

int myRank;

int main(int argc,char **argv)
{
MPI_Init(&argc,&argv);
double inval,outval;
int rank,size;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
MPI_Comm_size(MPI_COMM_WORLD, &size);

myRank=rank;
printf("[%d] before MyRank=%d (at %p)\n",rank,myRank,&myRank);

for (int i=0;i<4;i++) {
MPI_Barrier(MPI_COMM_WORLD);
printf("[%d] after MyRank=%d (at %p) pe=%d\n",rank,myRank,&myRank,CmiMyPe());
if (myRank!=rank) {
printf("[%d] Global variable not preserved! (%d)\n",rank,myRank);
abort();
}
}

MPI_Finalize();
return 0;
}

Example of execution with problems:

./charmrun +p3 ++nodelist nodelist global +vp4

Charmrun> scalable start enabled.

Charmrun> started all node programs in 1.171 seconds.

Charm++> Running in non-SMP mode: numPes 3

Converse/Charm++ Commit ID: v6.7.0-35-g52cc1e9

Warning> Randomization of stack pointer is turned on in kernel, thread migration may not work! Run 'echo 0 > /proc/sys/kernel/randomize_va_space' as root to disable it, or try run with '+isomalloc_sync'.

Charm++> scheduler running in netpoll mode.

CharmLB> Load balancer assumes all CPUs are same.

WARNING: Running swapglobals without blacklist, globals from libraries might be getting un-necessarily swapped

Charm++> -swapglobals enabled

Charm++> Running on 1 unique compute nodes (3-way SMP).

Charm++> cpu topology info is gathered in 0.012 seconds.

[0] before MyRank=0 (at 0x92ed94)

[3] before MyRank=3 (at 0x92ed94)

[2] before MyRank=2 (at 0x92ed94)

[1] before MyRank=1 (at 0x92ed94)

[3] after MyRank=3 (at 0x92ed94) pe=2

[2] after MyRank=2 (at 0x92ed94) pe=1

[0] after MyRank=1 (at 0x92ed94) pe=0

[0] Global variable not preserved! (1)

 

Best regards,

-- 
Laércio Lima Pilla, PhD.
Associate Professor (Professor Adjunto)
UFSC - CTC - INE, Brazil
Email: laercio.pilla AT ufsc.br or laercio.lima.pilla AT gmail.com  
Tel: +55 (48) 9152 8120, +55 (48) 3721 7564
Website: www.inf.ufsc.br/~pilla/