Charm++ parallel programming system

[Qian Sun <sqianfly AT gmail.com>]

Hi Ronak,

Thank you so much for your reply!

The I/O behavior I am looking for targeting temporal characteristics of both write and read. 

As far as I know, most of current scientific simulations/applications are using SPMD parallelism. Thus their write behavior is typically similar -- all of the processes will write a portion of data concurrently at the end of each iteration. But for Charm++ ported application, such as EpiSimdemics you mentioned, for a single iterations, is it possible that some of the tasks might complete first and generate a subset of output files earlier than others? Theoretically, I thought it is a reasonable behavior, because those parallel tasks are independent with each other, as well as their completion time. But I am not sure if there is a case in the real Charm++ ported applications. 

Similarly, for read, an task-based application might not need to wait for all of the input files become available. For example, 10 tasks are scheduled to run at the beginning, but only 4 of them get their input files available. Then these 4 tasks can start to run regardless of the left 6 tasks. Is there any Charm++ ported application performing in this way? 

Thanks a lot.

Regards,

Qian

On Thu, Mar 3, 2016 at 7:15 PM, Ronak Buch <rabuch2 AT illinois.edu> wrote:

Hi Qian,

There are many Charm++ applications that read or write "large" amounts of data, including OpenAtom, ChaNGa, NAMD, and EpiSimdemics among others.  Often, these are dependent on the size of the simulation they are trying to run.  Depending on the size and frequency of their simulation, they can often be on the scale of tens of gigabytes.

Regarding I/O behaviors, can you be a bit more specific about what exactly you're looking for?

It is certainly possible that the same applications will generate output data at different times (for example, as far as we can recall, EpiSimdemics does this).  There is also Charm++ library for parallel I/O, CkIO, that can facilitate large scale asynchronous I/O patterns.

Additionally, a few members of the PPL group had a published a paper regarding asynchronous I/O a few years ago that you may find relevant: http://charm.cs.illinois.edu/newPapers/11-32/paper.pdf

Thanks,

Ronak

On Thu, Mar 3, 2016 at 11:16 AM, Qian Sun <sqianfly AT gmail.com> wrote:

Hi,

I am looking for task-based scientific applications. I noticed that there are some applications have been ported to Charm++, such as NAMD, as well as some MiniApps, such as AMR. 

Actually, I am more interested in the I/O behavior of those Charm++ ported applications. Is there any Charm++ ported application generates a large amount of data as final output, or takes a large amount of data as input?

If so, which applications have this characteristics? and what is their I/O behaviors? I was wondering is it possible that tasks in the same applications will generate output data at different times? (Just as a typical DAG-based workflow, some tasks may finish and output a portion of data first, others may require longer time)

Any suggestions or comments would be appreciated! Thank you so much!

Regards,

Qian