Charm++ parallel programming system

[Phil Miller <mille121 AT illinois.edu>]

On Tue, Jul 2, 2013 at 9:26 PM, Kale, Laxmikant V 
<kale AT illinois.edu>
 wrote:
> I don't think anything ensures they are received on processors in the same
> order, but the array manager buffers them to ensure in order  delivery.
> (just stating what I think happens.. Not based on the knowledge of current
> codebase).

That in-order delivery to the array elements is based on their arrival
order at each PE. If they are processed in different orders at
different PEs, then they will be delivered to array elements in those
different orders. From the examination I've done so far, I believe
that it is possible for two broadcasts to arrive at and be processed
by different PEs in different orders. I'll try to construct a test
exhibiting this tomorrow.

As Eric pointed out to me in conversation today, nothing in our
documentation actually assures users of ordered delivery of broadcasts
for groups or arrays. The only guarantee we provide is that each
broadcast will be delivered exactly once by each member of the
recipient set. In the case of array elements migrating in the middle
of racing array broadcasts, different PE-level broadcast delivery
order could lead to incorrect non-/multiple-delivery of those
broadcasts to migrated elements.