Charm++ parallel programming system

["Kunzman, David M" <david.m.kunzman AT intel.com>]

I don’t think either Gengbin or I disagreed with your criticism.

 

I fear that you didn’t read what I wrote the way I intended it to be read.  You seem to be reading it as me suggesting “don’t program that way,” which is not my intent.

 

Item (2) is simply about how Charm++ is used most often.  In production codes, which in large part drives the development of Charm++, people don’t use readonly variables the way you are using them because of the limitation I mentioned.  This is to say, I’m not sure there has been a need to support what you are trying to do in the past, so it has just gone overlooked.  I don’t see why it couldn’t or shouldn’t be supported.

 

Item (1) is an ongoing debate between object oriented and non-object oriented programmers that has been going on since object oriented languages have existed.  It’s a matter of personal styles, preferences, and principles.  The fact that Charm++ includes global readonly variables in the first place is, I think, a recognition that some people have exactly the preference you’ve expressed and Charm++ isn’t trying to stop you from doing what you want...  It’s just that, today, the charmxi grammer doesn’t include what you have in mind.  And again, that is a consequence of practice, not principle.  As I mentioned when we talked in person, the Charm++ group mainly focuses on the runtime (with some language and programming model research, such as Structured Dagger and Charj), so the charmxi grammar isn’t as large/flexible/robust/etc. as the C/C++ grammer or the grammar of other languages.  As Gengbin put it, it’s simple because that is not where the research group spends its time.  It supports what it has needed to support to date.

 

Thanks,

Dave K

 

 

 

From: Van Der Wijngaart, Rob F
Sent: Monday, September 29, 2014 1:43 PM
To: Kunzman, David M; Hammond, Jeff R; Phil Miller
Cc: Nikhil Jain; Mattson, Timothy G; charm AT cs.illinois.edu
Subject: RE: Global variables in Charm++

 

Let me add that my criticisms are meant to be constructive. My initial experiments and readings of Charm++ literature tell me there is a lot to like about the runtime. But if it wants to be accepted by the computing community at large, it should support general programming styles. I am running into a lot of issues with my programming style, and I will be happy to defend that style in front of any forum. So please take these criticisms to heart.

 

Rob

 

From: Van Der Wijngaart, Rob F
Sent: Monday, September 29, 2014 1:33 PM
To: Kunzman, David M; Hammond, Jeff R; Phil Miller
Cc: Nikhil Jain; Mattson, Timothy G; charm AT cs.illinois.edu
Subject: RE: Global variables in Charm++

 

Thanks, Dave.

 

Actually, as I wrote last week, I wanted to be able to define something a little more complex, which turned out to be totally unpalatable by charmc:

readonly double weight[2*RADIUS+1][2*RADIUS+1];

Why would that be a reasonable thing to do for a readonly variable, one may ask? The reason is that this is an array that specifies the weights of a stencil. In my workload I allow different stencil sizes and different weights. But once defined, weight is a global constant. I let the user specify the constant symbol RADIUS, and use that to compute weight’s dimensions. In the .C file I can do that to my heart’s content, but not in the .ci file. To me that is not reasonable. I think this points to a critical weakness of Charm++. The only things that get supported in the processing of .ci files are:

1.       Constructs that are foreseen by the Charm++ developers.

2.       Constructs that the Charm++ developers deem to have merit.

 

Rob

 

From: Kunzman, David M
Sent: Monday, September 29, 2014 1:20 PM
To: Van Der Wijngaart, Rob F; Hammond, Jeff R; Phil Miller
Cc: Nikhil Jain; Mattson, Timothy G; charm AT cs.illinois.edu
Subject: RE: Global variables in Charm++

 

In principle, I don’t see a reason why what you are trying to do (using an ‘_expression_’ in the array dimension) could not be supported in ci files.  I think this is just a case of, charmxi (the tool that parses and processes the ci files) not being a “full blown” compiler.  Taking a quick look at the grammar for charmxi, you’ll see…

 

Readonly        : READONLY Type QualName DimList

 

DimList         : /* Empty */

                | Dim DimList

 

Dim             : '[' ArrayDim ']'

 

ArrayDim        : NUMBER

                | QualName

 

QualName        : IDENT

                | QualName ':'':' IDENT

 

… So, within the square brackets, you either need a number or a qualified name.  Rob, in your case, I think the preprocessor is leaving an _expression_ after replacing RADIUS, which just isn’t currently defined in the charmxi grammar.

 

That said, to be honest with you, I don’t really recall people using readonly variables in the way that you are using them (data for the calculation itself), except in very simple test/example codes, for two reasons…

 

(1)  Essentially, you are programming in a style that tries to use “globals” in an object oriented programming model (nothing inherently wrong with this, but people tend to avoid globals in object oriented programming models such as C++, which is unlike MPI where people rely on globals more often or in C).  This is not to say that people don’t do this (they obviously do, because quite frankly, it gets the job done in many cases).

 

(2)  In Charm++, readonly also have the restriction that they must have their value set after the main chare constructors are finished executing, so they tend to be somewhat limited in usefulness for anything beyond the simplest of codes.  In production codes, application startup tends to occur after of the main chare constructors have completed, so using something like a group or a node group to calculate and/or transmit runtime constants (lookup tables, weight matrices, whatever) makes more sense from a Charm++ perspective.

 

Because of item 2 in particular, I don’t think people have really cared about this in the past before.

 

Thanks,

Dave K

 

 

 

From: Van Der Wijngaart, Rob F
Sent: Monday, September 29, 2014 12:49 PM
To: Hammond, Jeff R; Phil Miller
Cc: Nikhil Jain; Kunzman, David M; Mattson, Timothy G; charm AT cs.illinois.edu
Subject: RE: Global variables in Charm++

 

But then again, maybe “double weight[4.5+4.5];” is OK. In some parallel universe this must be true.

 

Rob

 

From: Van Der Wijngaart, Rob F
Sent: Monday, September 29, 2014 12:47 PM
To: Hammond, Jeff R; Phil Miller
Cc: Nikhil Jain; Kunzman, David M; Mattson, Timothy G; charm AT cs.illinois.edu
Subject: RE: Global variables in Charm++

 

Eh, that would stink spectacularly. I refuse to accept that “double weight[3+3+3];” may result in a syntax error. I invite the Charm++ team to explain why this would be reasonable.

 

Rob

 

From: Hammond, Jeff R
Sent: Monday, September 29, 2014 12:38 PM
To: Van Der Wijngaart, Rob F; Phil Miller
Cc: Nikhil Jain; Kunzman, David M; Mattson, Timothy G; charm AT cs.illinois.edu
Subject: Re: Global variables in Charm++

 

Can you do something evil where you extern pointer to storage and then declare it in a generic C++ file that will be preprocessed properly?

 

Obviously, may provoke the question “what is that terrible smell?” but I’ve seen far worse workarounds associated with legacy Fortran :-)

 

Jeff

 

From: "<Van Der Wijngaart>", Rob F <rob.f.van.der.wijngaart AT intel.com>
Date: Monday, September 29, 2014 at 9:55 AM
To: Phil Miller <mille121 AT illinois.edu>
Cc: Nikhil Jain <nikhil.life AT gmail.com>, "Kunzman, David M" <david.m.kunzman AT intel.com>, Jeff R Hammond <jeff.r.hammond AT intel.com>, "Mattson, Timothy G" <timothy.g.mattson AT intel.com>, "charm AT cs.illinois.edu" <charm AT cs.illinois.edu>
Subject: RE: Global variables in Charm++

 

OK, it appears not to be the problem of running the preprocessor per se, but of having expressions in array dimensions. I replaced

  readonly double weight[(2*RADIUS+1)*(2*RADIUS+1)];

with

  readonly double weight[3+3+3];

and got another syntax error.

This looks like a pretty serious limitation to me. I am now required to do the array dimension computation for charmc and stuff that into the .ci file explicitly as a constant. This means running another script/preprocessor before I can run charmc.

 

Rob

 

From: Van Der Wijngaart, Rob F
Sent: Friday, September 26, 2014 3:49 PM
To: 'Phil Miller'
Cc: Nikhil Jain; Kunzman, David M; Hammond, Jeff R; Mattson, Timothy G; charm AT cs.illinois.edu
Subject: RE: Global variables in Charm++

 

Thanks Phil. I just added –E to my OPTS parameter, but then all hell breaks loose.

This is what charmc says is happening when it tries to compile my .ci file:

../../../bin/charmc -Ofast -DRADIUS=1 -DSTAR -E  jacobi2d.ci

And this is the line in my .ci file that it complains about:

  readonly double weight[(2*RADIUS+1)*(2*RADIUS+1)];

 

Output:

[rfvander@bar1 PRKstencil5]$ make

../../../bin/charmc -Ofast -DRADIUS=1 -DSTAR -E  jacobi2d.ci

STDIN:8: Charmxi syntax error> syntax error

Invalid construct

Fatal Error by charmc in directory /home/rfvander/charm/examples/charm++/PRKstencil5

   Command ../../../bin/charmxi -orig-file jacobi2d.ci returned error code 1

charmc exiting...

../../../bin/charmc -Ofast -DRADIUS=1 -DSTAR -c jacobi2d.C

jacobi2d.C(1): catastrophic error: cannot open source file "jacobi2d.decl.h"

  #include "jacobi2d.decl.h"

 

By the way, you may wonder about the subject line of this message. It started because I wanted a (small) 2D array as a global variable. I couldn’t get that to work (is that by design?), so made it into a 1D array. And then I ran into the issue I just reported.

 

Rob

 

From:unmobile AT gmail.com [mailto:unmobile AT gmail.com] On Behalf Of Phil Miller
Sent: Friday, September 26, 2014 3:37 PM
To: Van Der Wijngaart, Rob F
Cc: Nikhil Jain; Kunzman, David M; Hammond, Jeff R; Mattson, Timothy G; charm AT cs.illinois.edu
Subject: Re: Global variables in Charm++

 

For historical reasons [1], charmc defaults to *not* running the preprocessor on .ci files. Preprocessing .ci files can be enabled by passing charmc the -E flag when it's working on .ci files.

[1] Enabling pre-processing when it wasn't historically the default risks breaking existing application code by having macros that may have been used as identifiers in .ci files suddenly be subject to expansion/substitution.

 

On Fri, Sep 26, 2014 at 5:31 PM, Van Der Wijngaart, Rob F <rob.f.van.der.wijngaart AT intel.com> wrote:

Hello Nikhil,

 

I have a symbol that I define on the compile line (i.e. I stuff it in OPTS, as below):

OPTS=-DRADIUS=1

CHARMC=../../../bin/charmc $(OPTS)

When I now use RADIUS in my .C file, the C preprocessor does its work nicely and recognizes that I have defined RADIUS, and also what its value is. However, when I use RADIUS in the .ci file, charmc croaks. So is it true that no preprocessor is run on .ci files? Thanks!

 

Rob