Charm++ parallel programming system

[Roberto de Quadros Gomes <rqg.gomes AT gmail.com>]

Nikhil,

Did you find a solution for this?

I tried to see some changes in branch objsize_lbdatabase but it seems doesn't exist any more.

Maybe it was merged to mainline.
Phil, can you answer that please? Did you close the branch objsize_lbdatabase, or I'm lost?

Thanks

Roberto

2013/8/13 Nikhil Jain <nikhil.life AT gmail.com>

I checked in a fix for this. 

--Nikhil

On Mon, Aug 12, 2013 at 6:48 PM, Roberto de Quadros Gomes <rqg.gomes AT gmail.com> wrote:

After i got the branch objsize_lbdatabase and  I tried build charm, I received an erro because pup_c.h didn't include stddef.h for size_t.  I put there and it finished ok.
However, I received this result when I use branch:

Charmrun> started all node programs in 1.418 seconds.

Converse/Charm++ Commit ID: v6.5.0-beta1-856-g6d7e5f6
Charm++> scheduler running in netpoll mode.
Charm++> Running on 1 unique compute nodes.
Charm++> cpu topology info is gathered in 0.001 seconds.                                                                                      

[0] Greedy LB   created                                                                                                                          

------------- Processor 0 Exiting: Called CmiAbort ------------                                                                               
Reason: CthPupBase: Cannot pack running thread!                                                                                               
[0] Stack Traceback:                                                                                                                          
------------- Processor 1 Exiting: Called CmiAbort ------------                                                                               
Reason: CthPupBase: Cannot pack running thread!                                                                                               
[0:0] CmiAbort+0x75  [0x8261b7e]                                                                               
[0:1] CthPupBase+0x23  [0x819e352]                                                                               
[0:2] CthPup+0x37  [0x819ed1f]                                                                                     
[0:3] _ZN6TCharm9pupThreadERN3PUP2erE+0x64  [0x8155544]
[0:4] _ZN6TCharm3pupERN3PUP2erE+0x3aa  [0x815546c]
[0:5] _ZN12CkMigratable6AtSyncEi+0xb5  [0x81c49b3]
[0:6] _ZN6TCharm7migrateEv+0x19  [0x8155bdb]
[0:7] TCHARM_Migrate+0x59  [0x8156a3e]
[0:8] AMPI_Migrate+0xb  [0x816d1e5]
[0:9] AMPI_Main+0x9c  [0x81545c0]
[0:10] AMPI_Fallback_Main+0x2a  [0x8166f11]
[0:11] _ZN17MPI_threadstart_t5startEv+0x4a  [0x8189d00]
[0:12] AMPI_threadstart+0x2e  [0x8167233]
[0:13]   [0x8154c1c]
[0:14]   [0x819eb30]
[0:15] qt_error+0  [0x8287bf4]
[0:16] [0x9fb97d8]
Fatal error on PE 0> CthPupBase: Cannot pack running thread!
--------

The code I used in test is attached.

I've used "$ampicc teste.c -memory isomalloc -o bin/teste -module GreedyLB" to build

and "./charmrun ./bin/teste +p2 +vp3 +LBTestPESpeed +balancer GreedyLB" to run

The result is like this when i run using  6.5.0

[3] Hello, parallel world!
[1] Hello, parallel world!
[2] Hello, parallel world!

> 2013/8/9 Roberto de Quadros Gomes <rqg.gomes AT gmail.com>
>>
>> Awsome people,
>>
>> You were so fast! I am grateful a lot for this effort!
>> Now i will continue my development and, when I finish it, I'll share the results.
>>
>> Thanks 
>>
>>
>> 2013/8/9 Harshitha Menon <gplkrsh2 AT illinois.edu>
>>>
>>> I have implemented what you need in charm. It is in branch objsize_lbdatabase. It records the size automatically and the new field pupSize is there in obj data which can be used in the lb strategy. 
>>>
>>>
>>> On Fri, Aug 9, 2013 at 12:39 PM, Roberto de Quadros Gomes <rqg.gomes AT gmail.com> wrote:
>>>>
>>>> Hi Harshitha,
>>>>
>>>> If you have easy access to the code, can you send me the sample that you mentioned yesterday, please?
>>>> I tried to find in branch that you said but I didn't.
>>>>
>>>> Thanks
>>>>
>>>>
>>>>
>>>> 2013/8/9 Harshitha Menon <gplkrsh2 AT illinois.edu>
>>>>>
>>>>> How about adding a setter and getter (setObjSize and getObjSize) functions? You can look at how setObjTime method is implemented.
>>>>>
>>>>> Alternatively, a better way would be for you to use a feature implemented by Gengbin (in branch  gzheng/lb-userdata) to add arbitrary user-defined data in LB database. This is in a separate branch of charm. I have sample code which includes the object size to lb database and a lb strategy can use it.
>>>>>
>>>>> Harshitha
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Thu, Aug 8, 2013 at 7:07 PM, Roberto de Quadros Gomes <rqg.gomes AT gmail.com> wrote:
>>>>>>
>>>>>> Yes, I am trying to use the size of data will be transfer on migration process in lb strategy. I want to combine cpuload, communication and memory size.
>>>>>>
>>>>>> Let me explain what i am doing...
>>>>>>
>>>>>> Cpu load and communication is easy to get, but when i use the ObjGraph on strategy the  memory size in not a member of Vertex class . Then, I opened the code and added this filed. I did some modifications for collect and put memory info to LDStats. With this way I can put any value and I can read that inside of strategy.
>>>>>> I modified LBObj , LDObjData too. I just put "int mem_size" in each struct and modifying the pup functions.
>>>>>>
>>>>>> But I don't know how feed memory info with real value. I believe that there is some way to do this. I asked about AddObj method because I tried the reverse path (from LDStats to AMPI code) and there seems a good place to read this.
>>>>>>
>>>>>> I don't have sure if tcharm size, or its stack size, has the memory information of my variables used in MPI code but I'd like try use this.
>>>>>>
>>>>>>
>>>>>> And yes, the object size of part of code could help me.
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013/8/8 Harshitha Menon <gplkrsh2 AT illinois.edu>
>>>>>>>
>>>>>>> Are you trying to use the size of the object in some load balancing strategy?
>>>>>>>
>>>>>>> How about collecting the object size as a part of the application code and using that in the lb strategy? Will that work for you?
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Aug 8, 2013 at 5:59 PM, Roberto de Quadros Gomes <rqg.gomes AT gmail.com> wrote:
>>>>>>>>
>>>>>>>> Hi Harshitha,
>>>>>>>>
>>>>>>>> Your code block worked! Thank you for that.
>>>>>>>> But now I would like to share my real problem.
>>>>>>>>
>>>>>>>> I didn't understand what is the part of code represents the tcharm object created from AMPI code in LB.
>>>>>>>>
>>>>>>>> I was thinking that the method AddObj ( located in LBDBManager.C) would be the best place to collect memory size from this object, but i don't know how do this. I thought that "__userdata"  represented this tcharm object, but I failed when tried use (TCharm*)__userdata.
>>>>>>>>  
>>>>>>>> I belive I think wrong and I really appreciate some help.
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>> Roberto Gomes
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> 2013/8/8 Harshitha Menon <gplkrsh2 AT illinois.edu>
>>>>>>>>>
>>>>>>>>> To get the size of an object, you can use the following piece of code.
>>>>>>>>>
>>>>>>>>> int size;
>>>>>>>>> PUP::sizer ps;
>>>>>>>>> this->pup(ps);
>>>>>>>>> size=ps.size();
>>>>>>>>>
>>>>>>>>> Let me know if are having trouble using it in the LB strategy.
>>>>>>>>>
>>>>>>>>> Harshitha
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Thu, Aug 8, 2013 at 9:43 AM, Bohm, Eric J <ebohm AT illinois.edu> wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Begin forwarded message:
>>>>>>>>>>
>>>>>>>>>>> From: Roberto de Quadros Gomes <rqg.gomes AT gmail.com>
>>>>>>>>>>> Subject: [charm] Object size
>>>>>>>>>>> Date: August 5, 2013 9:12:09 PM CDT
>>>>>>>>>>> To: <charm AT cs.uiuc.edu>
>>>>>>>>>>>
>>>>>>>>>>> Is there some way to get amount of bytes will be transfer in advance when the an object will be migrated?
>>>>>>>>>>> I try collect this value to use on lb strategy but i am having difficulties.
>>>>>>>>>>>
>>>>>>>>>>> Looking the code in cklocation.C file, CkLocRec_local constructor seems the best location, but i dont know to do this.
>>>>>>>>>>>
>>>>>>>>>>> Some tip?
>>>>>>>>>>> Thanks
>>>>>>>>>>> Roberto Gomes
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> charm mailing list
>>>>>>>>>>> charm AT cs.uiuc.edu
>>>>>>>>>>> http://lists.cs.uiuc.edu/mailman/listinfo/charm
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

--
"Success may ditch you most of the times, but if you ditch it once : your job is done."
Nikhil Jain, nikhil.life AT gmail.com, +1-217-979-0918