Charm++ parallel programming system

[Jozsef Bakosi <jbakosi AT gmail.com>]

Ah, I guess this means that two reductions (regardless of type and target) cannot happen on the same chare array arbitrarily. I think the sequencing you describe can be hard to guarantee in general. Is this a frequent problem so shadow arrays are frequent? (I think I can guarantee the correct sequencing in the piece of code I'm currently working on, but I should check out shadow arrays then.) Is there a telltale sign, e.g., a specific runtime error, that I should learn to recognize if a shadow array is needed? How can I test that if I have the correct sequencing?

On Wed, Jul 20, 2016 at 9:52 AM, Phil Miller <mille121 AT illinois.edu> wrote:

The multiple concurrent uses of the reducer is not a problem. The problem that Tom and Sanjay bring up is multiple concurrent reductions on the same array. What reducer they use and what entry method they target is irrelevant.

The only thing that matters is whether or not the contribute calls are strictly sequenced by the rest of your code's logic. If your code guarantees that contribute() call site A always happens before contribute() call site B, in every object, you're fine. If some objects may execute B and then A, you have a problem. Either the code needs to enforce that sequence, or the reductions need to happen in separate contexts, such as another chare array instance (e.g. a shadow array).

On Wed, Jul 20, 2016 at 10:45 AM, Jozsef Bakosi <jbakosi AT gmail.com> wrote:

I'm not sure if I understand correctly, let me be more specific starting from the example in the user manual 16.2 but modified a bit to be closer to how I'm using it.

  /*initnode*/ void registerSumTwoShorts(void) {

    sumTwoShortsType=CkReduction::addReducer(sumTwoShorts);

  }

  /* reductiontarget */ void fn1( CkReductionMsg* msg ) { /* 1st final aggregated result here */ }

  /* reductiontarget */ void fn2( CkReductionMsg* msg ) { /* 2nd final aggregated result here */ }

  // In some member function, contribute data to a customized reduction:

  short data[2]=...;

  std::pair< int, std::unique_ptr<char[]> > stream = serialize( data );  // returns serialized data as length and char stream

  CkCallback cb( fn1(nullptr), thisProxy );

  contribute( stream.first, stream.second.get(), sumTwoShortsType, cb );

  // In some OTHER member function, contribute data to the ANOTHER customized reduction but of the same type:

  short otherdata[2]=...;

  std::pair< int, std::unique_ptr<char[]> > stream = serialize( otherdata );  // returns serialized otherdata as length and char stream

  CkCallback cb( fn2(nullptr), thisProxy );

  contribute( stream.first, stream.second.get(), sumTwoShortsType, cb );  // 'sumTwoShortsType' okay or need 'sumTwoShortsType2'?

So my question is: Is it okay to use a single sumTwoShortsType for both reductions which may overlap, i.e., happening at the same time?

I'm thinking that since I have two separate reduction targets, fn1() and fn2(), for the two reductions, I should be fine. However, the two reductions do happen on the same chare array/group and collect different data (of the same custom type). I have a feeling that this should be okay, but even if not, I could just register a second custom reducer, e.g., sumTwoShortsType2, and use it in the second contribute() call, which would make the two reductions have nothing to do with each other. Correct?

On Tue, Jul 19, 2016 at 3:30 PM, Kale, Laxmikant V <kale AT illinois.edu> wrote:

So, Jozsef, in case you are using those reductions in a way that the sequencing cannot be guaranteed to be the same (i.e. some chare array elements will contribute to reduction A before B,  while others contribute B before A), you should learn how to use the “shadow arrays” (called bound arrays in the manual : http://charm.cs.illinois.edu/manuals/html/charm++/13.html ) to achieve this.

Your phrase “at the same time” makes me think you may need it.  A bound array is bound to another chare array of the same shape, so that the corresponding elements (i.e. those with the same index) are guaranteed to be on the same processor. One of your reductions will contribute via the bound array.

 

Let us know if you have trouble with this.

 

Laxmikant (Sanjay) Kale         http://charm.cs.uiuc.edu

Professor, Computer Science     kale AT illinois.edu

201 N. Goodwin Avenue           Ph:  (217) 244-0094

Urbana, IL  61801-2302 

 

 

From: Phil Miller <mille121 AT illinois.edu>
Reply-To: Phil Miller <mille121 AT illinois.edu>
Date: Tuesday, July 19, 2016 at 2:57 PM
To: Tom Quinn <trq AT astro.washington.edu>
Cc: Jozsef Bakosi <jbakosi AT gmail.com>, "charm AT cs.uiuc.edu" <charm AT cs.uiuc.edu>
Subject: Re: [charm] Run multiple custom reducers at (potentially) the same time

 

On Tue, Jul 19, 2016 at 2:48 PM, Tom Quinn <trq AT astro.washington.edu> wrote:

Are you sure this works if the reducers are running on the same chare array?  In ChaNGa we had to set up "shadow arrays" to have two reductions running concurrently.

 

There is a potential concern if elements may end up making contribution calls concurrently, and in potentially different orders. Every element in an array, group, or section must contribute to the same sequence of reductions, just like MPI programs must call the same sequence of collections on a given communicator. The result is otherwise supposed to lead to assertion failures in the runtime, but may have undefined behavior.

 

On Tue, 19 Jul 2016, Phil Miller wrote:

On Tue, Jul 19, 2016 at 9:08 AM, Jozsef Bakosi <jbakosi AT gmail.com> wrote:
      I have been successfully using custom reducers, described in
      16.2 of the user manual. Now I'm wondering if it is safe to use
      more than one reducer, utilizing the same reducer type,
      registered in the initnode routine, at the same time, i.e.,
      potentially doing two (or more) reductions running at the same
      time but whose end-points, of course, point to different
      reduction entry method member functions. Or do I have to create
      multiple reducers as the manual example would require a
      sumTwoShortsType2?


The registration of custom reducers is just to associate a common global
index to t he function pointer that may differ across processes. So go right
ahead, and it should work fine.

The only case I can think of where multiple concurrent uses of a reducer
could be a problem is if the reducer implementation contained `static`
variables, which would seem quite strange.