简体繁体 English

生产中的TensorFlow用于高流量应用中的实时预测 - 如何使用？

[英]TensorFlow in production for real time predictions in high traffic app - how to use?

原文 2016-02-15 16:53:26 6 1 python/ machine-learning/ tensorflow/ tensorflow-serving

What is the right way to use TensorFlow for real time predictions in a high traffic application. 在高流量应用中使用TensorFlow进行实时预测的正确方法是什么。

Ideally I would have a server/cluster running tensorflow listening on a port(s) where I can connect from app servers and get predictions similar to the way databases are used. 理想情况下，我将有一个服务器/集群运行tensorflow监听端口，我可以从应用服务器连接，并获得类似于数据库使用方式的预测。 Training should be done by cron jobs feeding the training data through the network to the same server/cluster. 培训应该由cron作业通过网络将培训数据提供给同一服务器/集群来完成。

How does one actually use tensorflow in production? 如何在生产中实际使用张量流？ Should I build a setup where the python is running as a server and use the python scripts to get predictions? 我应该构建一个python作为服务器运行的设置并使用python脚本来获取预测吗？ I'm still new to this but I feel that such script will need to open sessions etc.. which is not scalable. 我还是新手，但我觉得这样的脚本需要打开会话等......这是不可扩展的。 (I'm talking about 100s of predictions/sec). （我说的是每秒100次预测/秒）。

Any pointer to relevant information will be highly appreciated. 任何指向相关信息的指针都将受到高度赞赏。 I could not find any. 我找不到任何东西。

1 个解决方案

This morning, our colleagues released TensorFlow Serving on GitHub, which addresses some of the use cases that you mentioned. 今天早上，我们的同事在GitHub上发布了TensorFlow服务，它解决了你提到的一些用例。 It is a distributed wrapper for TensorFlow that is designed to support high-performance serving of multiple models. 它是TensorFlow的分布式包装器，旨在支持多个模型的高性能服务。 It supports both bulk processing and interactive requests from app servers. 它支持来自应用服务器的批量处理和交互式请求。

For more information, see the basic and advanced tutorials. 有关更多信息，请参阅基础和高级教程。