Skip to Content.
Sympa Menu

charm - Re: [charm] Build with ninja

charm AT lists.cs.illinois.edu

Subject: Charm++ parallel programming system

List archive

Re: [charm] Build with ninja


Chronological Thread 
  • From: Jozsef Bakosi <jbakosi AT gmail.com>
  • To: Phil Miller <mille121 AT illinois.edu>
  • Cc: "charm AT cs.uiuc.edu" <charm AT cs.uiuc.edu>
  • Subject: Re: [charm] Build with ninja
  • Date: Wed, 16 Jul 2014 16:45:53 -0600
  • List-archive: <http://lists.cs.uiuc.edu/pipermail/charm/>
  • List-id: CHARM parallel programming system <charm.cs.uiuc.edu>

On Wed, Jul 16, 2014 at 3:46 PM, Phil Miller <mille121 AT illinois.edu> wrote:
Thanks, this is really helpful.

There is a slight nuisance with the solution below that changes the CMAKE_CXX_LINK_EXECUTABLE variable though: the linker will be exchanged for ALL executables the project generates. This is not an issue for me at this time, but I thought it was worth mentioning.

Another solution to exchange the linker in cmake that does allow per-file specification of the linker is based on hijacking the link line cmake generates with an external script and exchange the linker inside that script. That works too, but only for makefile-generators and not for ninja. In case someone wants to go that route, here is the relevant question on the cmake email list, as of now unanswered, that has more info how to make this version work and the hijack script: http://public.kitware.com/pipermail/cmake/2014-May/057536.html
 
What, if anything, did you use to address compiling the .ci file(s)?

Here is what I do in cmake to add the necessary dependencies and the custom command that generates the decl.h and def.h for a chare:

function(addCharmModule MODULE PARTOF)
  # Arguments:
  #   MODULE:    Name of the Charm++ module.
  #   PARTOF:    Name of the library or executable the Charm++ module (and the
  #              Charm++ host file) will link to. Note that the library or
  #              executable PARTOF requires a custom link command as that must
  #              be linked by the charmc wrapper.

  # Add custom command generating .decl.h and .def.h from .ci
  add_custom_command(OUTPUT ${MODULE}.decl.h ${MODULE}.def.h
                     DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/${MODULE}.ci
                     COMMAND ${CHARM_COMPILER}
                             ${CMAKE_CURRENT_SOURCE_DIR}/${MODULE}.ci)
  # Add custom target dependency for Charm++ module
  add_custom_target(${MODULE}CharmModule
                    DEPENDS ${MODULE}.decl.h ${MODULE}.def.h)
  # Add dependency of PARTOF on new Charm++ module
  add_dependencies(${PARTOF} ${MODULE}CharmModule)
endfunction(addCharmModule)

Basically, the above adds a custom cmake command that is used to tell cmake how to generate the decl.h and def.h from a .ci file. Then creates a custom target (the new Charm++ module) depending on the decl.h and def.h. Finally, it adds this newly added target as a dependency of a library or executable that is passed in in PARTOF. The above function is then used elsewhere as

addCharmModule( "<interface-file-name-without-the-ci-extension>" "<name-of-target-the-charm-module-will-belong-to>" )

J
 


On Mon, Jul 14, 2014 at 10:56 AM, Jozsef Bakosi <jbakosi AT gmail.com> wrote:
So here is how I'm able to use ninja with Charm++ now:

As you suggested, I use the naked compiler to generate the intermediate objects and I only use the charmc wrapper for the final link step for the executable. Here is the relevant cmake code that instructs cmake to exchange the linker for charmc only for the final executable:

# Link Charm++ executable with the charmc wrapper
STRING(REGEX REPLACE "<CMAKE_CXX_COMPILER>" ${CHARM_COMPILER} CMAKE_CXX_LINK_EXECUTABLE "${CMAKE_CXX_LINK_EXECUTABLE}")

The above line does a string substitution on the linker command line and replaces the template, "<CMAKE_CXX_COMPILER>", by the value of the cmake variable CHARM_COMPILER, which should be set to the charmc wrapper.

Jozsef

On Sun, May 4, 2014 at 4:05 PM, Phil Miller <mille121 AT illinois.edu> wrote:
So, looking up those flags in the g++ man page, that command line is not actually aiming to compile test.C at all - it's trying to generate dependencies for its compilation.

charmc's support for this usage is non-existent. Those flags require rather special behavior from anything wrapping the compiler and interpreting other flags, beyond just adding include paths and passing everything through to the backend compiler driver. It would probably take me a couple hours of reading dense manuals to even estimate the effort it would take to properly support these.

In the cases I know of where we do dependency generation for our code (the Charm++ runtime itself and the OpenAtom application), we call g++ directly.

You might want to look at the Makefile used in NAMD. They simply use g++ or whatever other native compiler for object compilation steps, and only invoke charmc for the final linking step. Since ninja's documentation describes using some external tool to generate ninja's input files, generating these rules appropriately should hopefully not be too challenging. At a first pass, it looks like the only 'special' argument you'll absolutely have to pass to the compiler is the path to the 'include' directory in your build of Charm++.

If you go this route, I'd be interested to see what you put together.

Phil




On Sat, May 3, 2014 at 8:47 PM, Jozsef Bakosi <jbakosi AT gmail.com> wrote:
Hi folks,

I'm trying to compile a Charm++ program using the ninja build system (instead of make) and I'm running into errors with the charmc script. Here is a shortened example compile line ninja generates:

$ charmc -MMD -MT test.C.o -MF test.C.o.d -o test.C.o -c test.C
Fatal Error by charmc in directory ...
   test.C.o: file not recognized: File truncated
charmc exiting...

Another one, failing due to a different argument:

$ charmc -MF test.C.o.d -o test.C.o -c test.C
Fatal Error by charmc in directory ...
   file with unrecognized extension test.C.o.d
charmc exiting...

If I take out both the "-MF test.C.o.d" and the "-MT test.C.o" options, it compiles fine. I'm not familiar with how and why ninja generates these -MT and -MF options but I assumed there is a good reason and I cannot just eliminate them.

Is there anyone else using ninja with charm?

I tried grepping for -MF and -MT in the charmc script and found no occurrences. Shouldn't the charmc wrapper script pass through any compiler option it does not recognize?

In any case, can this be fixed? That would be awesome as I prefer ninja to make.

Thank you in advance,
Jozsef

_______________________________________________
charm mailing list
charm AT cs.uiuc.edu
http://lists.cs.uiuc.edu/mailman/listinfo/charm








Archive powered by MHonArc 2.6.16.

Top of Page