Charm++ parallel programming system

[Jozsef Bakosi <jbakosi AT gmail.com>]

Hi folks,

After reading the papers at http://charm.cs.illinois.edu/research/msa and the thesis at https://charm.cs.illinois.edu/newPapers/05-05/paper.pdf, though may not exactly have been designed for my use case, it looks like what I need could also be done using multiphase shared arrays (MSA).

In particular, the thesis above says: "MSA also allows us to set a cap on the memory used per processor. Thus it can be used to read in a huge amount of data on one processor but store it simply and efficiently across other processors." This seems like exactly what I need. Even if this is not very efficient, since I only need to do this once during setup, I think this should work and would be better than saving many small files and re-reading them by chares. In fact, I would already be happy if creating many chares, initialized by chunks of the large data in alldata, would survive without blowing out of communication buffers with, say 100K+ CPUs.

Does anyone think MSA is NOT okay for this purpose?

Jozsef