Skip to Content.
Sympa Menu

charm - Re: [charm] Synchronizing flushing of collected traces to log files

charm AT lists.cs.illinois.edu

Subject: Charm++ parallel programming system

List archive

Re: [charm] Synchronizing flushing of collected traces to log files


Chronological Thread 
  • From: Alexander Frolov <alexndr.frolov AT gmail.com>
  • To: "Kale, Laxmikant V" <kale AT illinois.edu>
  • Cc: "charm AT cs.uiuc.edu" <charm AT cs.uiuc.edu>
  • Subject: Re: [charm] Synchronizing flushing of collected traces to log files
  • Date: Wed, 22 Jul 2015 16:11:32 +0300
  • List-archive: <http://lists.cs.uiuc.edu/pipermail/charm/>
  • List-id: CHARM parallel programming system <charm.cs.uiuc.edu>

Just another question. 

Is it possible to save customized colors for entry methods and use in subsequent runs of the projections tool?  

On Wed, Jul 22, 2015 at 4:05 PM, Alexander Frolov <alexndr.frolov AT gmail.com> wrote:
Thank you very much for your reply!

Relating to enlarging memory allocated to traces. I am using mpirun to run my application, that is +logsize parameter is unavailable for me (or I dont know how to set it via mpirun). 

Currentlly I am tuning directly 
#define DefaultLogBufSize      10000000    (in the ./src/ck-perf/trace-projections.C). 

Is this correct? or there are any other means to set maximum memory size allocated for traces under mpirun?

I will be obliged very much if somebody tell which call enforce flushing. Thank you!  

On Wed, Jul 22, 2015 at 3:45 PM, Kale, Laxmikant V <kale AT illinois.edu> wrote:
There is a fixed amount of memory allocated to storing traces (which can be controlled, within limits, by +logsize parameter. (to be used while running the application). 

Once this runs out, on any given processor (separately), that processor will flush the log to the log file before continuing. If these flushes (which take milliseconds) happen at different times on different processors, the performance data will be distorted (ompared with an execution that is not recording traces)  because of these interruptions. Thats what the warning is telling you.

The trace should not be corrupted by this (if it is, thats another problem). Just that it will show long gaps on individual processors during flushing. 

Synchronizing: use application logic to find a point at which to flush traces from all processors, via an explicit call. (will someone post the name of the call, and update the manual to list in the above-linked appendix E?).

Another option, for fine grained applications that generate a lot of events, is to only sample a portion of the execution in detail (turning traces on and off explicitly; see “Selective tracing” in appendix E of the manual), and optionally, to trace the entire execution in summary mode.

Hope this helps. (and not too obvious a summary for you).

-- 
--- 
Laxmikant (Sanjay) Kale         http://charm.cs.uiuc.edu
Professor, Computer Science     kale AT illinois.edu
201 N. Goodwin Avenue           Ph:  (217) 244-0094
Urbana, IL  61801-2302          

From: Alexander Frolov <alexndr.frolov AT gmail.com>
Date: Wednesday, July 22, 2015 at 6:11 AM
To: "charm AT cs.uiuc.edu" <charm AT cs.uiuc.edu>
Subject: [charm] Synchronizing flushing of collected traces to log files

Hi!

I am profiling my charm++ application with projections. It works well until I increase the problem size and receive the warnings like that:

Warning: Projections log flushed to disk 4 times on 2 cores: 0 1 .
Warning: The performance data is likely invalid, unless the flushes have been explicitly synchronized by your program. 

In this case the trace is corrupted. How can I synchronize it?

Best, 
   Alex






Archive powered by MHonArc 2.6.16.

Top of Page