tvm.contrib.rpc

Lightweight TVM RPC module.

RPC enables connect to a remote server, upload and launch functions. This is useful to for cross-compile and remote testing, The compiler stack runs on local server, while we use RPC server to run on remote runtime which don’t have a compiler available.

The test program compiles the program on local server, upload and run remote RPC server, get the result back to verify correctness.

tvm.contrib.rpc.connect(url, port, key='', session_timeout=0)

Connect to RPC Server

Parameters:
  • url (str) – The url of the host
  • port (int) – The port to connect to
  • key (str, optional) – Additional key to match server
  • session_timeout (float, optional) – The duration of the session, allows server to kill the connection when duration is longer than this value. When duration is zero, it means the request must always be kept alive.
Returns:

sess – The connected session.

Return type:

RPCSession

tvm.contrib.rpc.connect_tracker(url, port)

Connect to a RPC tracker

Parameters:
  • url (str) – The url of the host
  • port (int) – The port to connect to
Returns:

sess – The connected tracker session.

Return type:

TrackerSession

class tvm.contrib.rpc.TrackerSession(addr)

Tracker client session.

Parameters:addr (tuple) – The address tuple
close()

Close the tracker connection.

request(key, priority=1, session_timeout=0, max_retry=5)

Request a new connection from the tracker.

Parameters:
  • key (str) – The type key of the device.
  • priority (int, optional) – The priority of the request.
  • session_timeout (float, optional) – The duration of the session, allows server to kill the connection when duration is longer than this value. When duration is zero, it means the request must always be kept alive.
  • max_retry (int, optional) – Maximum number of times to retry before give up.
request_and_run(key, func, priority=1, session_timeout=0, max_retry=2)

Request a resource from tracker and run the func.

This function safe-guard rare server node dropout during execution. In such case, a new resource will be requested and func will be ran again.

Parameters:
  • key (str) – The type key of the device.
  • func (function of session -> value) – A stateless function
  • priority (int, optional) – The priority of the request.
  • session_timeout (float, optional) – The duration of the session, allows server to kill the connection when duration is longer than this value. When duration is zero, it means the request must always be kept alive.
  • max_retry (int, optional) – Maximum number of times to retry the function before give up.
summary()

Get the summary dict of the tracker.

text_summary()

Get a text summary of the tracker.

class tvm.contrib.rpc.RPCSession(sess)

RPC Client session module

Do not directly create the obhect, call connect

cl(dev_id=0)

Construct OpenCL device.

context(dev_type, dev_id=0)

Construct a remote context.

Parameters:
  • dev_type (int or str) –
  • dev_id (int, optional) –
Returns:

ctx – The corresponding encoded remote context.

Return type:

TVMContext

cpu(dev_id=0)

Construct CPU device.

download(path)

Download file from remote temp folder.

Parameters:path (str) – The relative location to remote temp folder.
Returns:blob – The result blob from the file.
Return type:bytearray
ext_dev(dev_id=0)

Construct extension device.

get_function(name)

Get function from the session.

Parameters:name (str) – The name of the function
Returns:f – The result function.
Return type:Function
gpu(dev_id=0)

Construct GPU device.

load_module(path)

Load a remote module, the file need to be uploaded first.

Parameters:path (str) – The relative location to remote temp folder.
Returns:m – The remote module containing remote function.
Return type:Module
metal(dev_id=0)

Construct Metal device.

opengl(dev_id=0)

Construct OpenGL device.

upload(data, target=None)

Upload file to remote runtime temp folder

Parameters:
  • data (str or bytearray) – The file name or binary in local to upload.
  • target (str, optional) – The path in remote
class tvm.contrib.rpc.LocalSession

RPCSession interface backed by local environment.

This class can be used to implement functions that need to be ran both locally and remotely.

cl(dev_id=0)

Construct OpenCL device.

context(dev_type, dev_id=0)

Construct a remote context.

Parameters:
  • dev_type (int or str) –
  • dev_id (int, optional) –
Returns:

ctx – The corresponding encoded remote context.

Return type:

TVMContext

cpu(dev_id=0)

Construct CPU device.

download(path)

Download file from remote temp folder.

Parameters:path (str) – The relative location to remote temp folder.
Returns:blob – The result blob from the file.
Return type:bytearray
ext_dev(dev_id=0)

Construct extension device.

get_function(name)

Get function from the session.

Parameters:name (str) – The name of the function
Returns:f – The result function.
Return type:Function
gpu(dev_id=0)

Construct GPU device.

load_module(path)

Load a remote module, the file need to be uploaded first.

Parameters:path (str) – The relative location to remote temp folder.
Returns:m – The remote module containing remote function.
Return type:Module
metal(dev_id=0)

Construct Metal device.

opengl(dev_id=0)

Construct OpenGL device.

upload(data, target=None)

Upload file to remote runtime temp folder

Parameters:
  • data (str or bytearray) – The file name or binary in local to upload.
  • target (str, optional) – The path in remote
class tvm.contrib.rpc.Server(host, port=9091, port_end=9199, is_proxy=False, use_popen=False, tracker_addr=None, key='', load_library=None)

Start RPC server on a seperate process.

This is a simple python implementation based on multi-processing. It is also possible to implement a similar C based sever with TVM runtime which does not depend on the python.

Parameters:
  • host (str) – The host url of the server.
  • port (int) – The port to be bind to
  • port_end (int, optional) – The end port to search
  • is_proxy (bool, optional) – Whether the address specified is a proxy. If this is true, the host and port actually corresponds to the address of the proxy server.
  • use_popen (bool, optional) – Whether to use Popen to start a fresh new process instead of fork. This is recommended to switch on if we want to do local RPC demonstration for GPU devices to avoid fork safety issues.
  • key (str, optional) – The key used to identify the server in Proxy connection.
  • load_library (str, optional) – List of additional libraries to be loaded during execution.
terminate()

Terminate the server process