How to Deploy TVM Modules¶
We provide an example on how to deploy TVM modules in apps/howto_deploy
To run the example, you can use the following command
cd apps/howto_deploy ./run_example.sh
Get TVM Runtime Library¶
The only thing we need is to link to a TVM runtime in your target platform.
TVM provides a minimum runtime, which costs around 300K to 600K depending on how much modules we use.
In most cases, we can use
libtvm_runtime.so that comes with the build.
If somehow you find it is hard to build
libtvm_runtime, checkout tvm_runtime_pack.cc.
It is an example all in one file that gives you TVM runtime.
You can compile this file using your build system and include this into your project.
You can also checkout apps for example applications build with TVM on iOS, Android and others.
Dynamic Library vs. System Module¶
- Store library as a shared library and dynamically load the library into your project.
- Bundle the compiled library into your project in system module mode.
Dynamic loading is more flexible and can load new modules on the fly. System module is a more
static approach. We can use system module in places where dynamic library loading is banned.
Deploy to Android¶
Build model for Android Target¶
NNVM compilation of model for android target could follow same approach like android_rpc.
An reference exampe can be found at chainer-nnvm-example
Above example will directly run the compiled model on RPC target. Below modification at rum_mobile.py will save the compilation output which is required on android target.
lib.export_library("deploy_lib.so", ndk.create_shared) with open("deploy_graph.json", "w") as fo: fo.write(graph.json()) with open("deploy_param.params", "wb") as fo: fo.write(nnvm.compiler.save_param_dict(params))
deploy_lib.so, deploy_graph.json, deploy_param.params will go to android target.