Contrib APIs

Contrib APIs of TVM python package.

Contrib API provides many useful not core features. Some of these are useful utilities to interact with thirdparty libraries and tools.


Utility to invoke nvcc compiler in the system

tvm.contrib.nvcc.compile_cuda(code, target=’ptx’, arch=None, options=None, path_target=None)

Compile cuda code with NVCC from env.

  • code (str) – The cuda code.
  • target (str) – The target format
  • arch (str) – The architecture
  • options (str) – The additional options
  • path_target (str, optional) – Output file.

cubin – The bytearray of the cubin

Return type:



Utility function to find cuda path

Returns:path – Path to cuda root.
Return type:str

Util to invoke c++ compilers in the system., objects, options=None, cc=’g++’)

Create shared library.

  • output (str) – The target shared library.
  • objects (list) – List of object files.
  • options (list) – The list of additional options string.
  • cc (str, optional) – The compile string.


Utility to invoke Xcode compiler toolchain


Codesign the shared libary

This is an required step for library to be loaded in the app.

Parameters:lib (The path to the library.) –
tvm.contrib.xcode.compile_metal(code, path_target=None, sdk=’macosx’)

Compile metal with CLI tool from env.

  • code (str) – The cuda code.
  • path_target (str, optional) – Output file.
  • sdk (str, optional) – The target platform SDK.

metallib – The bytearray of the metallib

Return type:


tvm.contrib.xcode.create_dylib(output, objects, arch, sdk=’macosx’)

Create dynamic library.

  • output (str) – The target shared library.
  • objects (list) – List of object files.
  • options (str) – The additional options.
  • arch (str) – Target major architectures
  • sdk (str) – The sdk to be used.
tvm.contrib.xcode.popen_test_rpc(host, port, key, destination, libs=None, options=None)

Launch rpc server via xcodebuild test through another process.

  • host (str) – The address of RPC proxy host.
  • port (int) – The port of RPC proxy host
  • key (str) – The key of the RPC server
  • destination (str) – Destination device of deployment, as in xcodebuild
  • libs (list of str) – List of files to be packed into app/Frameworks/tvm These can be dylibs that can be loaed remoted by RPC.
  • options (list of str) – Additional options to xcodebuild

proc – The test rpc server process. Don’t do wait() on proc, since it can terminate normally.

Return type:



Run xcrun and return the output.

Parameters:cmd (list of str) – The command sequence.
Returns:out – The output string.
Return type:str


RPC interface for easy testing.

RPC enables connect to a remote server, upload and launch functions. This is useful to for cross-compile and remote testing, The compiler stack runs on local server, while we use RPC server to run on remote runtime which don’t have a compiler available.

The test program compiles the program on local server, upload and run remote RPC server, get the result back to verify correctness.

class tvm.contrib.rpc.RPCSession(sess)

RPC Client session module

Do not directly create the obhect, call connect


Construct remote OpenCL device.

context(dev_type, dev_id=0)

Construct a remote context.

  • dev_type (int or str) –
  • dev_id (int, optional) –

ctx – The corresponding encoded remote context.

Return type:



Construct remote CPU device.


Download file from remote temp folder.

Parameters:path (str) – The relative location to remote temp folder.
Returns:blob – The result blob from the file.
Return type:bytearray

Construct remote extension device.


Get function from the session.

Parameters:name (str) – The name of the function
Returns:f – The result function.
Return type:Function

Construct remote GPU device.


Load a remote module, the file need to be uploaded first.

Parameters:path (str) – The relative location to remote temp folder.
Returns:m – The remote module containing remote function.
Return type:Module

Construct remote Metal device.


Construct remote OpenGL device.

upload(data, target=None)

Upload file to remote runtime temp folder

  • data (str or bytearray) – The file name or binary in local to upload.
  • target (str, optional) – The path in remote
class tvm.contrib.rpc.Server(host, port=9091, port_end=9199, is_proxy=False, use_popen=False, key=’‘)

Start RPC server on a seperate process.

This is a simple python implementation based on multi-processing. It is also possible to implement a similar C based sever with TVM runtime which does not depend on the python.

  • host (str) – The host url of the server.
  • port (int) – The port to be bind to
  • port_end (int, optional) – The end port to search
  • is_proxy (bool, optional) – Whether the address specified is a proxy. If this is true, the host and port actually corresponds to the address of the proxy server.
  • use_popen (bool, optional) – Whether to use Popen to start a fresh new process instead of fork. This is recommended to switch on if we want to do local RPC demonstration for GPU devices to avoid fork safety issues.
  • key (str, optional) – The key used to identify the server in Proxy connection.

Terminate the server process

tvm.contrib.rpc.connect(url, port, key=’‘)

Connect to RPC Server

  • url (str) – The url of the host
  • port (int) – The port to connect to
  • key (str, optional) – Additional key to match server

sess – The connected session.

Return type:



Minimum graph runtime that executes graph containing TVM PackedFunc.

class tvm.contrib.graph_runtime.GraphModule(module, ctx)

Wrapper runtime module.

This is a thin wrapper of the underlying TVM module. you can also directly call set_input, run, and get_output of underlying module functions

  • module (Module) – The interal tvm module that holds the actual graph functions.
  • ctx (TVMContext) – The context this module is under

Module – The interal tvm module that holds the actual graph functions.


TVMContext – The context this module is under

get_output(index, out)

Get index-th output to out

  • index (int) – The input index
  • out (NDArray) – The output array container

Load parameters from serialized byte array of parameter dict.

Parameters:params_bytes (bytearray) – The serialized parameter dict.

Run forward execution of the graph

Parameters:input_dict (dict of str to NDArray) – List of input values to be feed to
set_input(key=None, value=None, **params)

Set inputs to the module via kwargs

  • key (int or str) – The input key
  • value (the input value.) – The input key
  • params (dict of str to NDArray) – Additonal arguments
tvm.contrib.graph_runtime.create(graph_json_str, libmod, ctx)

Create a runtime executor module given a graph and module.

  • graph_json_str (str or graph class) – The graph to be deployed in json format output by nnvm graph. The graph can only contain one operator(tvm_op) that points to the name of PackedFunc in the libmod.
  • libmod (tvm.Module) – The module of the corresponding function
  • ctx (TVMContext) – The context to deploy the module, can be local or remote.

graph_module – Runtime graph module that can be used to execute the graph.

Return type:



Common system utilities

class tvm.contrib.util.TempDirectory

Helper object to manage temp directory during testing.

Automatically removes the directory when it went out of scope.


“List contents in the dir.

Returns:names – The content of directory
Return type:list

Relative path in temp dir

Parameters:name (str) – The name of the file.
Returns:path – The concatenated path.
Return type:str

Remote the tmp dir


Create temp dir which deletes the contents when exit.

Returns:temp – The temp directory object
Return type:TempDirectory


External function interface to BLAS libraries.

tvm.contrib.cblas.matmul(lhs, rhs, transa=False, transb=False)

Create an extern op that compute matrix mult of A and rhs with CrhsLAS

This function serves as an example on how to call external libraries.

  • lhs (Tensor) – The left matrix operand
  • rhs (Tensor) – The right matrix operand
  • transa (bool) – Whether transpose lhs
  • transb (bool) – Whether transpose rhs

C – The result tensor.

Return type: