Charm++ parallel programming system

[Ronak Buch <rabuch2 AT illinois.edu>]

Hi Jozsef,

There was some work on adding read capabilities to CkIO a while back, it's on the gerrit/rohan/ckioread branch of charm.  The reading facility added there should work, but nobody has been using it in a production applications, so it's out of date (and it hasn't been rebased on a long time).  Now that there's some interest in using parallel file reading, I'll update the branch and do some performance tests and get back to you.

Otherwise, a node group level file I/O scheme should work well in the meantime.  Your sketch on how things should be distributed makes sense to me.

Thanks,

Ronak

On Fri, Feb 10, 2017 at 9:37 PM, Jozsef Bakosi <jbakosi AT gmail.com> wrote:

Hi folks,

Currently, I have file/PE-style read of a mesh that is in a single file and about a few tens of GB containing a few hundred million cells and their node coordinates. This is okay but does not scale very well beyond a few thousand cores.

I have read the paper on ckio from 2011 and looked at the source and it is definitely something I would like to explore for writing checkpoints in the future, but I wonder what my options are for input. ckio does not seem to have a capability at this point for reading.

Another option that seems appealing is to use Charm++'s node groups and/or the "Physical Node API" for simply replacing my file/PE read with a node/PE read and then after reading my input file in that fashion in a second step I would distribute the data to group and/or chare array elements.

What do you suggest? At this point I am interested in expl oring both quick/dirty as well as good/longer-term solutions.

Thanks,

Jozsef