码迷，mamicode.com
首页 > 其他好文 > 详细

【tensorflow2.0】使用tensorflow-serving部署模型

时间：2020-04-13 13:48:49      阅读：64      评论：0      收藏：0      [点我收藏+]

标签：nbsp   地址   一个   target   服务   ram   art   ras   init   

TensorFlow训练好的模型以tensorflow原生方式保存成protobuf文件后可以用许多方式部署运行。

例如：通过 tensorflow-js 可以用javascrip脚本加载模型并在浏览器中运行模型。

通过 tensorflow-lite 可以在移动和嵌入式设备上加载并运行TensorFlow模型。

通过 tensorflow-serving 可以加载模型后提供网络接口API服务，通过任意编程语言发送网络请求都可以获取模型预测结果。

通过 tensorFlow for Java接口，可以在Java或者spark(scala)中调用tensorflow模型进行预测。

我们主要介绍tensorflow serving部署模型、使用spark(scala)调用tensorflow模型的方法

〇，tensorflow serving模型部署概述

使用 tensorflow serving 部署模型要完成以下步骤。

  • (1) 准备protobuf模型文件。

  • (2) 安装tensorflow serving。

  • (3) 启动tensorflow serving 服务。

  • (4) 向API服务发送请求，获取预测结果。

可通过以下colab链接测试效果《tf_serving》： https://colab.research.google.com/drive/1vS5LAYJTEn-H0GDb1irzIuyRB8E3eWc8

%tensorflow_version 2.x
import tensorflow as tf
print(tf.__version__)
from tensorflow.keras import *

一，准备protobuf模型文件

我们使用tf.keras 训练一个简单的线性回归模型，并保存成protobuf文件。

import tensorflow as tf
from tensorflow.keras import models,layers,optimizers
 
## 样本数量
n = 800
 
## 生成测试用数据集
X = tf.random.uniform([n,2],minval=-10,maxval=10) 
w0 = tf.constant([[2.0],[-1.0]])
b0 = tf.constant(3.0)
 
Y = X@w0 + b0 + tf.random.normal([n,1],
    mean = 0.0,stddev= 2.0) # @表示矩阵乘法,增加正态扰动
 
## 建立模型
tf.keras.backend.clear_session()
inputs = layers.Input(shape = (2,),name ="inputs") #设置输入名字为inputs
outputs = layers.Dense(1, name = "outputs")(inputs) #设置输出名字为outputs
linear = models.Model(inputs = inputs,outputs = outputs)
linear.summary()
 
## 使用fit方法进行训练
linear.compile(optimizer="rmsprop",loss="mse",metrics=["mae"])
linear.fit(X,Y,batch_size = 8,epochs = 100)  
 
tf.print("w = ",linear.layers[1].kernel)
tf.print("b = ",linear.layers[1].bias)
 
## 将模型保存成pb格式文件
export_path = "./data/linear_model/"
version = "1"       #后续可以通过版本号进行模型版本迭代与管理
linear.save(export_path+version, save_format="tf")
Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
inputs (InputLayer)          [(None, 2)]               0         
_________________________________________________________________
outputs (Dense)              (None, 1)                 3         
=================================================================
Total params: 3
Trainable params: 3
Non-trainable params: 0
_________________________________________________________________
Epoch 1/100
100/100 [==============================] - 0s 2ms/step - loss: 273.0472 - mae: 13.9096
Epoch 2/100
100/100 [==============================] - 0s 2ms/step - loss: 250.0846 - mae: 13.3155
Epoch 3/100
100/100 [==============================] - 0s 2ms/step - loss: 228.0106 - mae: 12.7211
Epoch 4/100
100/100 [==============================] - 0s 2ms/step - loss: 208.5060 - mae: 12.1514
Epoch 5/100
100/100 [==============================] - 0s 2ms/step - loss: 188.6825 - mae: 11.5647
Epoch 6/100
100/100 [==============================] - 0s 2ms/step - loss: 170.6377 - mae: 10.9862
Epoch 7/100
100/100 [==============================] - 0s 2ms/step - loss: 153.1913 - mae: 10.4133
Epoch 8/100
100/100 [==============================] - 0s 2ms/step - loss: 137.3440 - mae: 9.8525
Epoch 9/100
100/100 [==============================] - 0s 2ms/step - loss: 122.1956 - mae: 9.2907
Epoch 10/100
100/100 [==============================] - 0s 2ms/step - loss: 108.5923 - mae: 8.7439
Epoch 11/100
100/100 [==============================] - 0s 2ms/step - loss: 94.8144 - mae: 8.1773
Epoch 12/100
100/100 [==============================] - 0s 2ms/step - loss: 83.0037 - mae: 7.6339
Epoch 13/100
100/100 [==============================] - 0s 2ms/step - loss: 71.8595 - mae: 7.1003
Epoch 14/100
100/100 [==============================] - 0s 2ms/step - loss: 61.8016 - mae: 6.5690
Epoch 15/100
100/100 [==============================] - 0s 2ms/step - loss: 52.5519 - mae: 6.0456
Epoch 16/100
100/100 [==============================] - 0s 2ms/step - loss: 44.4070 - mae: 5.5431
Epoch 17/100
100/100 [==============================] - 0s 2ms/step - loss: 37.0890 - mae: 5.0457
Epoch 18/100
100/100 [==============================] - 0s 2ms/step - loss: 30.6758 - mae: 4.5701
Epoch 19/100
100/100 [==============================] - 0s 2ms/step - loss: 25.1626 - mae: 4.1214
Epoch 20/100
100/100 [==============================] - 0s 2ms/step - loss: 20.3433 - mae: 3.6880
Epoch 21/100
100/100 [==============================] - 0s 2ms/step - loss: 16.3199 - mae: 3.2814
Epoch 22/100
100/100 [==============================] - 0s 2ms/step - loss: 13.1249 - mae: 2.9330
Epoch 23/100
100/100 [==============================] - 0s 2ms/step - loss: 10.4714 - mae: 2.6117
Epoch 24/100
100/100 [==============================] - 0s 2ms/step - loss: 8.5397 - mae: 2.3433
Epoch 25/100
100/100 [==============================] - 0s 2ms/step - loss: 7.0484 - mae: 2.1351
Epoch 26/100
100/100 [==============================] - 0s 2ms/step - loss: 6.1226 - mae: 1.9872
Epoch 27/100
100/100 [==============================] - 0s 2ms/step - loss: 5.6070 - mae: 1.9047
Epoch 28/100
100/100 [==============================] - 0s 2ms/step - loss: 5.2954 - mae: 1.8510
Epoch 29/100
100/100 [==============================] - 0s 2ms/step - loss: 5.0835 - mae: 1.8137
Epoch 30/100
100/100 [==============================] - 0s 2ms/step - loss: 4.9148 - mae: 1.7841
Epoch 31/100
100/100 [==============================] - 0s 2ms/step - loss: 4.7715 - mae: 1.7581
Epoch 32/100
100/100 [==============================] - 0s 2ms/step - loss: 4.6395 - mae: 1.7303
Epoch 33/100
100/100 [==============================] - 0s 2ms/step - loss: 4.5205 - mae: 1.7106
Epoch 34/100
100/100 [==============================] - 0s 1ms/step - loss: 4.4232 - mae: 1.6903
Epoch 35/100
100/100 [==============================] - 0s 2ms/step - loss: 4.3417 - mae: 1.6738
Epoch 36/100
100/100 [==============================] - 0s 1ms/step - loss: 4.2691 - mae: 1.6579
Epoch 37/100
100/100 [==============================] - 0s 2ms/step - loss: 4.2078 - mae: 1.6470
Epoch 38/100
100/100 [==============================] - 0s 2ms/step - loss: 4.1606 - mae: 1.6381
Epoch 39/100
100/100 [==============================] - 0s 2ms/step - loss: 4.1203 - mae: 1.6292
Epoch 40/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0847 - mae: 1.6230
Epoch 41/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0589 - mae: 1.6182
Epoch 42/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0382 - mae: 1.6141
Epoch 43/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0188 - mae: 1.6109
Epoch 44/100
100/100 [==============================] - 0s 2ms/step - loss: 4.0089 - mae: 1.6098
Epoch 45/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9979 - mae: 1.6075
Epoch 46/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9891 - mae: 1.6055
Epoch 47/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9848 - mae: 1.6053
Epoch 48/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9806 - mae: 1.6044
Epoch 49/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9752 - mae: 1.6037
Epoch 50/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9739 - mae: 1.6038
Epoch 51/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9712 - mae: 1.6024
Epoch 52/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9690 - mae: 1.6024
Epoch 53/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9685 - mae: 1.6021
Epoch 54/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9667 - mae: 1.6021
Epoch 55/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9651 - mae: 1.6009
Epoch 56/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9656 - mae: 1.6019
Epoch 57/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6016
Epoch 58/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6012
Epoch 59/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9655 - mae: 1.6018
Epoch 60/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9639 - mae: 1.6016
Epoch 61/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9650 - mae: 1.6010
Epoch 62/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9651 - mae: 1.6017
Epoch 63/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9646 - mae: 1.6021
Epoch 64/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6019
Epoch 65/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9639 - mae: 1.6027
Epoch 66/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9656 - mae: 1.6013
Epoch 67/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9645 - mae: 1.6019
Epoch 68/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6024
Epoch 69/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6015
Epoch 70/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6022
Epoch 71/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9626 - mae: 1.6022
Epoch 72/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9632 - mae: 1.6015
Epoch 73/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6023
Epoch 74/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6017
Epoch 75/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6003
Epoch 76/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9648 - mae: 1.6022
Epoch 77/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9624 - mae: 1.6023
Epoch 78/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6019
Epoch 79/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6019
Epoch 80/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9648 - mae: 1.6018
Epoch 81/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9649 - mae: 1.6025
Epoch 82/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9631 - mae: 1.6021
Epoch 83/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9650 - mae: 1.6020
Epoch 84/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6020
Epoch 85/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6014
Epoch 86/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6017
Epoch 87/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6024
Epoch 88/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9646 - mae: 1.6016
Epoch 89/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9643 - mae: 1.6016
Epoch 90/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9636 - mae: 1.6019
Epoch 91/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6029
Epoch 92/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6026
Epoch 93/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6014
Epoch 94/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9623 - mae: 1.6019
Epoch 95/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9637 - mae: 1.6015
Epoch 96/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9641 - mae: 1.6017
Epoch 97/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9635 - mae: 1.6027
Epoch 98/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9644 - mae: 1.6024
Epoch 99/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9640 - mae: 1.6021
Epoch 100/100
100/100 [==============================] - 0s 2ms/step - loss: 3.9638 - mae: 1.6024
w =  [[1.99997306]
 [-1.01220131]]
b =  [2.88236618]
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/resource_variable_ops.py:1817: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
INFO:tensorflow:Assets written to: ./data/linear_model/1/assets
# 查看保存的模型文件
!ls {export_path+version}

assets saved_model.pb variables

# 查看模型文件相关信息
!saved_model_cli show --dir {export_path+str(version)} --all
MetaGraphDef with tag-set: ‘serve‘ contains the following SignatureDefs:

signature_def[‘__saved_model_init_op‘]:
  The given SavedModel SignatureDef contains the following input(s):
  The given SavedModel SignatureDef contains the following output(s):
    outputs[‘__saved_model_init_op‘] tensor_info:
        dtype: DT_INVALID
        shape: unknown_rank
        name: NoOp
  Method name is: 

signature_def[‘serving_default‘]:
  The given SavedModel SignatureDef contains the following input(s):
    inputs[‘inputs‘] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 2)
        name: serving_default_inputs:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs[‘outputs‘] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 1)
        name: StatefulPartitionedCall:0
  Method name is: tensorflow/serving/predict
WARNING: Logging before flag parsing goes to stderr.
W0413 05:10:30.262132 140384690243456 deprecation.py:506] From /usr/local/lib/python2.7/dist-packages/tensorflow_core/python/ops/resource_variable_ops.py:1786: calling __init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.

Defined Functions:
  Function Name: ‘__call__‘
    Option #1
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name=u‘inputs‘)
        Argument #2
          DType: bool
          Value: False
        Argument #3
          DType: NoneType
          Value: None
    Option #2
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name=u‘inputs‘)
        Argument #2
          DType: bool
          Value: True
        Argument #3
          DType: NoneType
          Value: None

  Function Name: ‘_default_save_signature‘
    Option #1
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name=u‘inputs‘)

  Function Name: ‘call_and_return_all_conditional_losses‘
    Option #1
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name=u‘inputs‘)
        Argument #2
          DType: bool
          Value: False
        Argument #3
          DType: NoneType
          Value: None
    Option #2
      Callable with:
        Argument #1
          inputs: TensorSpec(shape=(None, 2), dtype=tf.float32, name=u‘inputs‘)
        Argument #2
          DType: bool
          Value: True
        Argument #3
          DType: NoneType
          Value: None

二，安装 tensorflow serving

安装 tensorflow serving 有2种主要方法：通过Docker镜像安装，通过apt安装。

通过Docker镜像安装是最简单，最直接的方法，推荐采用。

Docker可以理解成一种容器，其上面可以给各种不同的程序提供独立的运行环境。

一般业务中用到tensorflow的企业都会有运维同学通过Docker 搭建 tensorflow serving.

无需算法工程师同学动手安装，以下安装过程仅供参考。

不同操作系统机器上安装Docker的方法可以参照以下链接。

Windows: https://www.runoob.com/docker/windows-docker-install.html

MacOs: https://www.runoob.com/docker/macos-docker-install.html

CentOS: https://www.runoob.com/docker/centos-docker-install.html

安装Docker成功后，使用如下命令加载 tensorflow/serving 镜像到Docker中

docker pull tensorflow/serving

三，启动 tensorflow serving 服务

!docker run -t --rm -p 8501:8501     -v "/Users/.../data/linear_model/"     -e MODEL_NAME=linear_model     tensorflow/serving & >server.log 2>&1

四，向API服务发送请求

可以使用任何编程语言的http功能发送请求，下面示范linux的 curl 命令发送请求，以及Python的requests库发送请求。

!curl -d ‘{"instances": [1.0, 2.0, 5.0]}‘     -X POST http://localhost:8501/v1/models/linear_model:predict
{
    "predictions": [[3.06546211], [5.01313448]
    ]
}
import json,requests
 
data = json.dumps({"signature_name": "serving_default", "instances": [[1.0, 2.0], [5.0,7.0]]})
headers = {"content-type": "application/json"}
json_response = requests.post(‘http://localhost:8501/v1/models/linear_model:predict‘, 
        data=data, headers=headers)
predictions = json.loads(json_response.text)["predictions"]
print(predictions)

 

参考：

开源电子书地址：https://lyhue1991.github.io/eat_tensorflow2_in_30_days/

GitHub 项目地址：https://github.com/lyhue1991/eat_tensorflow2_in_30_days

【tensorflow2.0】使用tensorflow-serving部署模型

标签：nbsp   地址   一个   target   服务   ram   art   ras   init   

原文地址：https://www.cnblogs.com/xiximayou/p/12690757.html
踩
(0)
赞
(0)
   
举报
评论 一句话评论（0）
登录后才能评论！
分享档案
更多>
2021年07月29日 (22)
2021年07月28日 (40)
2021年07月27日 (32)
2021年07月26日 (79)
2021年07月23日 (29)
2021年07月22日 (30)
2021年07月21日 (42)
2021年07月20日 (16)
2021年07月19日 (90)
2021年07月16日 (35)
周排行
mamicode.com排行更多图片
更多
友情链接
兰亭集智   国之画   百度统计   站长统计   阿里云   chrome插件   新版天听网
关于我们 - 联系我们 - 留言反馈
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码！