Charm++ parallel programming system

[Evghenii Gaburov <e-gaburov AT northwestern.edu>]

Hi all,

In addition of the previous code I mail with all-to-all, I also tried the 
idea of return tickets from remote msg.

The code is below, it runs, but due to my lack of deep understanding of 
Charm++ msg processing,
I am not exactly sure whether the code logic guarantees the execution order,
namely do_work_complete() will only be called after all remote recv() have 
been processed.

The overhead of this code is in millisecond range for 4096 chares and 128 
procs (16 SMP nodes)

Could you please help me to confirm or disprove this in the code below? 

Thanks!

Cheers,
 Evghenii


void myClass::do_work()
{
  assert(nSend_counter == 0);

  /* do some work first */
  for (int i = 0; i < nsend; i++)
    if (remoteIndex[i] != thisIndex)
    {
      nSend_counter++;                            /* send msg counter, 
private to this chare */
      myClass[remoteIndex[i]].recv(results[i]);   /* send data to remote 
chares */ 
    }

  /* if this chare did not send any msg, proceed to the barrier immediately */
  if (nSend_counter == 0)
    contibute(CkCallback(do_complete_work(), thisProxy));
}

void myClass::recvTicket()   
{
  nSend_counter--;
  assert(nSend_counter >= 0);
  if (nSend_counter == 0)
    contribute(CkCallback(do_complete_work(), thisProxy));
}

void myClass::recv(myResults remote_result) 
{ 
  store_remote_result; 
  assert(remote_result.recvFromChare != thisIndex);
  myClass[remote_result.recvFromChare].recvTicket();
}

void myClass::do_work_complete()            
{
 assert(nSend_counter == 0); 
 process arrived remote_results; 
}
"

On Sep 25, 2011, at 10:03 AM, Gengbin Zheng wrote:

> priority does not help here. It only enforce an order of execution for
> all messages *currently* pending in the message queue (that is,
> already received messages).
> If messages arrived one after each other after each execution of
> messages, priority does not work for this case.
> 
> Currently we don;t have a way to set priority for reduction messages.
> 
> In your code, you either need something like Quiescence detection, or
> write explicit synchronization point to split the two phases ("recv"
> phase, and reduction phase).
> 
> Gengbin
> 
> On Sun, Sep 25, 2011 at 9:45 AM, Akhil langer 
> <akhilanger AT gmail.com>
>  wrote:
>> Lets say a total of 100 messages are sent by Chare A and Chare B to Chare 
>> C.
>> Consider a given instant at which out of these 100 messages about 10
>> messages have arrived at the node having chare C. These messages could be
>> any 10 out of those 100 sent messages (e.g. msg 1, msg 5, msg 10 from Chare
>> A and msg 2, msg5, msg3, etc. from Chare B). As you can see that messages
>> received by Chare C need not be in any order.
>> 
>> Now the Run time system has to pick from these 10 messages 1-by-1 and
>> execute the corresponding entry method on Chare C. This order of message
>> processing can be prioritized by assigning priority to the messages at the
>> sender side. The priority scheme can be FIFO, LIFO or any other custom
>> priority queue. This prioritization has been explained in Section 3.5
>> 
>> Does that clarify the difference?
>> 
>> --Akhil
>> 
>> On Sun, Sep 25, 2011 at 12:11 AM, Evghenii Gaburov
>> <e-gaburov AT northwestern.edu>
>>  wrote:
>>> 
>>>> After the message arrives at a PE, delivery to the user program can be
>>>> ordered using one of the priority schemes. Please note the difference
>>>> between http://charm.cs.illinois.edu/manuals/html/charm++/3_5.html
>>>> and
>>>> 
>>>> http://charm.cs.illinois.edu/manuals/html/charm++/3_8.html#SECTION00038300000000000000
>>>> (Messages are not guaranteed to be delivered in order).
>>> From documentation it is not clear how can I set priority for contribute.
>>> 
>>> How can I set priority of contribute(do_work_complete), such that it will
>>> be called *after* all recv have been processed?
>>> I suppose the priority is set for CkCallback, but I cannot find how to do
>>> this.
>>> 
>>> Do you perhaps know how to do this?
>>> 
>>> This type of communication is an important part of my applications, and I
>>> cannot yet
>>> wrap my head around on how to deal with this in Charm++.
>>> 
>>> Thanks!
>>> 
>>> Cheers,
>>>  Evghenii
>>> 
>>>> 
>>>> 
>>>> In your program, can the contribute call be moved to the remote chares
>>>> recv method instead??
>>>> 
>>>> 
>>>> On Sat, Sep 24, 2011 at 8:59 PM, Evghenii Gaburov
>>>> <e-gaburov AT northwestern.edu>
>>>>  wrote:
>>>> Hi All,
>>>> 
>>>> I have some misunderstanding upon the order in which the messages
>>>> arrive. I read, that messages by default obey FIFO order
>>>> 
>>>> So, does the following code
>>>> 
>>>> "
>>>> void myClass::do_work()
>>>> {
>>>>  /* do some work first */
>>>>  for (int i = 0; i < nsend; i++)
>>>>   myClass[remoteIndex[i]].recv(results[i]);                       /*
>>>> send data to remote chares */
>>>>  contribute(0, 0, CkReduction::concat, CkCallback(do_work_complete(),
>>>> thisProxy));   /* barrier */
>>>> }
>>>> 
>>>> void myClass::recv(myResults remote_result) {  store_remote_result; }
>>>> void myClass::do_work_complete()            { process arrived
>>>> remote_results; }
>>>> "
>>>> 
>>>> guarantees that myClass::recv(..) methods will be executed first
>>>> (because they were called first),
>>>> and only afterwards a reduction part will call
>>>> myClass::do_work_complete() (because it is called second).
>>>> This order is required to make sure that when do_work_complete is only
>>>> invoked when *all* remote data has arrived.
>>>> 
>>>> Thanks!
>>>> 
>>>> Cheers,
>>>>  Evghenii
>>>> --
>>>> Evghenii Gaburov, 
>>>> e-gaburov AT northwestern.edu
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> 
>>>> _______________________________________________
>>>> charm mailing list
>>>> charm AT cs.uiuc.edu
>>>> http://lists.cs.uiuc.edu/mailman/listinfo/charm
>>>> 
>>> 
>>> --
>>> Evghenii Gaburov, 
>>> e-gaburov AT northwestern.edu
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>> 
>> 
>> _______________________________________________
>> charm mailing list
>> charm AT cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/charm
>> 
>> _______________________________________________
>> ppl mailing list
>> ppl AT cs.uiuc.edu
>> http://lists.cs.uiuc.edu/mailman/listinfo/ppl
>> 
>> 

--
Evghenii Gaburov, 
e-gaburov AT northwestern.edu