简体繁体 English

从经过训练的神经网络创建实时运行 model 的最佳实践

[英]Best practice to create a live running model from a trained neural network

原文 2021-01-25 19:51:22 9 1 python/ tensorflow/ machine-learning/ deep-learning/ neural-network

I'm planning to create a neural network for trading, I'm investigating on how to use the trained neural network in a language that is used in runtime by my system (ie C++ or C#).我计划创建一个用于交易的神经网络，我正在研究如何在我的系统运行时使用的语言（即 C++ 或 C#）中使用经过训练的神经网络。

Let's say I'm using tensorflow in python (or any other nn framework), I've successfully trained my neural network that has n hidden layers.假设我在 python（或任何其他 nn 框架）中使用 tensorflow，我已经成功训练了具有 n 个隐藏层的神经网络。 I have at that point the values for weights and bias.那时我有权重和偏差的值。 At this point I would like to use the nn model in production.此时我想在生产中使用 nn model。

In my understanding it is not best practice to run in production tensorflow to run the model, since it's practically too slow (python) and clunky to do so (correct me if I'm wrong).据我了解，在生产 tensorflow 中运行 model 并不是最佳实践，因为它实际上太慢（python）而且很笨重（如果我错了，请纠正我）。

So I'd like to recreate the nn model in C# or C++ (or any other language), do I need to recreate in code all the maths for each layer based on weight and bias for every neuron?所以我想在 C# 或 C++ （或任何其他语言）中重新创建 nn model ，我是否需要在代码中重新创建每个神经元数学的所有基于权重和偏差的每一层？

Questions:问题：

I've never done it, so this is me guessing the workflow/best practice, what is the best way to do it?我从来没有做过，所以这是我在猜测工作流程/最佳实践，最好的方法是什么？
What are some online articles/websites I can read about this topic?我可以阅读哪些有关此主题的在线文章/网站？
What is this topic called, so that I can look for it?这个题目叫什么，好让我去找？

1 个解决方案

Frameworks like tensorflow should also be used in production.像 tensorflow 这样的框架也应该在生产中使用。 Tensorflow is written in highly optimized C++ and/or Cuda and will thus easily outperform anything you will write yourself. Tensorflow 是用高度优化的 C++ 和/或 Cuda 编写的，因此很容易胜过您自己编写的任何内容。 There is also a large amount of inbuilt tools for monitoring, load balancing etc. that comes with tensorflow. tensorflow 还附带了大量用于监控、负载平衡等的内置工具。

If you run your system on your machine you can use any language as a wrapper.如果你在你的机器上运行你的系统，你可以使用任何语言作为包装器。 If you really think you need a level of optimization for small tasks such as preprocessing etc. that can not be done in Python, just use the C++ API for tensorflow. If you really think you need a level of optimization for small tasks such as preprocessing etc. that can not be done in Python, just use the C++ API for tensorflow. Just let tensorflow (or torch or whatever production lvl framework you are using) do the heavy lifting of handling the neural network.只需让 tensorflow（或 torch 或您正在使用的任何生产 lvl 框架）完成处理神经网络的繁重工作。 For 99.99% percent of tasks Python(using numpy, tf, dask etc.) will be just fine.对于 99.99% 的任务，Python（使用 numpy、tf、dask 等）就可以了。

For large scale applications you can host your model on aws/azure/google cloud etc. They offer optimized services for hosting tensorflow models (eg. as REST endpoints).对于大型应用程序，您可以在 aws/azure/google 云等上托管 model。它们为托管 tensorflow 模型（例如，作为 REST 端点）提供优化服务。 Lambda fucntions/Azure Functions can be a great tool to do the preprocessing step in a costefficient and scaleable way. Lambda 函数/Azure Functions 可以成为以经济高效且可扩展的方式执行预处理步骤的绝佳工具。