Charm++ parallel programming system

[Jozsef Bakosi <jbakosi AT gmail.com>]

Hi Phil,

Thanks for your answers, some questions/comments below inline.

On Mon, Apr 28, 2014 at 4:14 PM, Phil Miller <mille121 AT illinois.edu> wrote:

Hi Jozsef,

There's some documentation on how to interoperate MPI and Charm++ code in our manual:
http://charm.cs.illinois.edu/manuals/html/charm++/25.html

I have seen this, but I thought the functionality that section describes is for using Charm++ libraries in MPI programs. What I'm after is the opposite: using a Charm++ program with MPI libraries, i.e., a predominantly Charm++ application using, e.g., an MPI-based library for, e.g., I/O.

There was also a talk by Nikhil Jain at last year's Charm++ Workshop on exactly this topic:
http://charm.cs.illinois.edu/workshops/charmWorkshop2013/program.php

Those slides are interesting and do shed some light on the Charm++/MPI interoperation - using two main approaches: AMPI for existing MPI apps and (gradually) converting MPI apps to Charm++ apps. I'm in the (lucky) situation that I'm designing a code from scratch and I only want to use MPI if I have to: I would like to use Charm++ predominantly and use some existing well-established (MPI) libraries for e.g., the solution of linear systems and I/O.

 

I wonder if there are examples (advantages, disadvantages, etc) of using third-party libraries that use MPI within a Charm++ application.

There's a paper currently in submission that covers this sort of mixing in both directions (MPI libraries in predominantly Charm++ application, Charm++ libraries in predominantly MPI application)

In the Charm++ repository, there are some code examples in examples/charm++/mpi-coexist and examples/charm++/mpi-charmcollide

 

I have looked at those two examples - thanks for drawing my attention to them. Don't they both demonstrate Charm++/MPI interoparability from the viewpoint of a predominantly MPI application using Charm++ functionality as a library? (I may not be familiar with Charm++ enough, so I will revisit these examples.)

  

At least these various combinations of software have been tried experimentally:

- Charm++ application with MPI IO library

- Charm++ application with MPI FFTW

Cool! These seem like exactly what I'm looking for. BTW, I managed to build and run Hypre with AMPI (with Charm++ built on the mpi layer mpi-linux-x86_64), so it seems like this will work fine. I also timed one of the Hypre examples -- using algebraic multigrid as a preconditioner solving a simple cartesian-mesh-discretized Poisson equation -- with pure-MPI and Charm++/AMPI on top of MPI and the performance of the two were very close: the AMPI version was actually even faster by about 0.2% (which is probably only statistical variability). (Note that I did no virtualization: I simply run using the same number of CPU cores with MPI and Charm++. This was interesting for me to see if there is any kind of performance hit putting another layer (Charm++ and AMPI) between an MPI application and the MPI runtime.)

  

The paper I referenced above (which I'll leave it to the authors to decide whether/how to share) says that yes, it's possible and worthwhile.

 

I would be interested in those papers. Please let me know when they are available for public consumption.

 

Exactly how the various pieces are compiled and executed depends on the platform an application is going to run on. In general, the design is such that MPI libraries are natively usable, without any use of AMPI.

That, I believe, answers my question. I will try to use MPI libraries then that way: simply compile them as they are usually compiled and link them to a Charm++ application.

Thanks for your time,

Jozsef