charm AT lists.cs.illinois.edu

Subject: Charm++ parallel programming system

List archive

RE: [charm] charm and gRPC objects

From: Juan J Galvez <jjgalvez AT msn.com>
To: David Epstein <davideps AT umich.edu>, "charm AT cs.illinois.edu" <charm AT cs.illinois.edu>
Subject: RE: [charm] charm and gRPC objects
Date: Wed, 22 Jan 2020 21:28:35 +0000
Accept-language: en-US
Arc-authentication-results: i=1; mx.microsoft.com 1; spf=none; dmarc=none; dkim=none; arc=none
Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eEewczuibs8X/buwU6uUeznbm5IG3w5Oa87JhTjjTLc=; b=Ok2ivP9gNcbw/GbZeBXhNwe0GYXpaazhjzdj9vLBUBkYoSfj7lagellcf5Kw+5J3C9X5l5GBe9NLg52aEhf8z+ghMKd2sK1kSLdUnF50mqzfeJJvoIauAf+BBkfUi/44DIsduEtDnSYaJZ2f/7ZkhaDuhz2ZPECDABKvDuZy+05vRtCFRaUjU25nEB1Vs/1hoRQdqB6lZZ9seByiWENFDRbXZQljBH5RYpf3qU/Roy3FZLFRB1lNyBaV11G2QjOITj+MqZW7m6W2tWyjaQuVL/4TB7PBJyp6Yo0RH5AWW9EKne4btpYEVs7h+/jj4mJeglgMWFsyh3nq88ZQMfy3eg==
Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Afekq8yridizJGEv+Z/C2d7oZRSUF5akSrxmRFTRYpYX55izWM5AviKFtmfY6UT857iQEhdzVUmqK7r6MMG3K7zqlDMn2TwTTvIEPvgDaieI51C45fxHIVkVdzsBWCVla10AIKlsfCd1MSYK4a2c8KU2/TfGUB/Z/t+um8L4SjvUgIl35XvWfUzp3K5jITzpZQskjhK32cybmn6sH0IGt+GQQzGtx+TqixfMHIpBQmLRLLt4SlcwfO+sc9FN5WONeL9l+IUOqnk6dIlGzCwXVu+isGeRq+HJptDBTAYwnQg+BVC5/2Dg/jCb0G+LvIFcbjda9/mB9pbXpe1UEuFxsQ==
Authentication-results: illinois.edu; spf=softfail smtp.mailfrom=jjgalvez AT msn.com; dkim=pass header.d=msn.com header.s=selector1; dmarc=pass header.from=msn.com

Hi David,

Charm4py doesn’t use pickle when it can copy directly from a memory buffer for certain objects like NumPy arrays and byte arrays. If the gRPC object is laid out contiguously in memory and you have access to its byte array, you could try sending that (for example the bytes object) or wrapping it in a NumPy buffer using numpy.frombuffer or something similar.

Thanks,

-Juan

From: David Epstein
Sent: Tuesday, January 21, 2020 4:01 PM
To: charm AT cs.illinois.edu
Subject: [charm] charm and gRPC objects

Hi,

I'm looking for a Python parallel processing framework that permits creating multiple processes that receive or generate gRPC objects that cannot be pickled. I notice that charm uses pickle for serialization. My project involves creating autonomous actors that interact with a Grakn database. Grakn uses a gRPC object as its transaction handle. Is there a way around this pickle problem?

Thank you,

-david

[charm] charm and gRPC objects, David Epstein, 01/21/2020
- RE: [charm] charm and gRPC objects, Juan J Galvez, 01/22/2020