url Tensorflow Serving Python Request 超过了最大重试次数

Question

I've trained my own model and I've runned docker container with this code:

docker run -t --rm -d -p 8501:8501 -v "%cd%/serving/tensorflow_serving/servables/tensorflow/testdata/optik_gpu:/models/optik_gpu" -e MODEL_NAME=optik_gpu tensorflow/serving

I'm using Windows 10 as OS.

And I'm trying to post base64 string to the container to get a prediction. Here is my code as suggested on Google Cloud :

import base64 import io import json import requests

def container_predict(image_file_path, image_key, port_number=8501):

    with io.open(image_file_path, 'rb') as image_file:
        encoded_image = base64.b64encode(image_file.read()).decode('utf-8')

    instances = {
            'instances': [
                    {'image_bytes': {'b64': str(encoded_image)},
                     'key': image_key}
            ]
    }

    url = 'http://localhost:{}/v1/models/default:predict'.format(port_number)

    response = requests.post(url, data=json.dumps(instances))
    print(response.json())

container_predict("./test.jpg", 1)

When I run the script it gives this error:

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 157, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection
    raise err
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1016, in _send_output
    self.send(msg)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 956, in send
    self.connect()
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 184, in connect
    conn = self._new_conn()
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 169, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001A75A6B7668>: Failed to establish a new connection: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout

(optik3) E:\Programming\Tensorflow\optik>python try.py
Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 157, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\util\connection.py", line 84, in create_connection
    raise err
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\util\connection.py", line 74, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connectionpool.py", line 387, in _make_request    
    conn.request(method, url, **httplib_request_kw)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 1016, in _send_output
    self.send(msg)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\http\client.py", line 956, in send
    self.connect()
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 184, in connect
    conn = self._new_conn()
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connection.py", line 169, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x000001BDEB7466D8>: Failed to establish a new connection: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\adapters.py", line 449, in send
    timeout=timeout
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\urllib3\util\retry.py", line 436, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='localhost', port=4444): Max retries exceeded with url: /v1/models/default:predict (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001BDEB7466D8>: Failed to establish a new connection: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "try.py", line 39, in <module>
    container_predict("./test.jpg", 1, 4444)
  File "try.py", line 36, in container_predict
    response = requests.post(url, data=json.dumps(instances))
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\api.py", line 119, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "D:\ProgramData\Anaconda3\envs\optik3\lib\site-packages\requests\adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=4444): Max retries exceeded with url: /v1/models/default:predict (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001BDEB7466D8>: Failed to establish a new connection: [WinError 10061] Hedef makine etkin olarak reddettiğinden bağlantı kurulamadı'))

I don't know what is the problem.

Answer 1

I will preface this by saying I am new to all of this and am not sure how everything works.

Just had a similar problem and had a look to see what ports were open. Turns out that my docker container was on 8502 (don't know why as I thought I had specified it be on 8501 like in the tutorial here https://cloud.google.com/vision/automl/docs/containers-gcs-tutorial?hl=en_GB ). So once I changed the port to 8502 in the python code it started working.

Not sure if it will help you, but my code is now working... now I just need to figure out what is going on and why it happened!

Answer 2

The problem may be caused by the fact that you specified model name "optic_gpu" when launching docker container, but use "default" as model name when trying to do request. At least in that Google Cloud lesson they don't specify model name, and because of this use "default". So you may try to change link like this url = 'http://localhost:{}/v1/models/optic_gpu:predict'.format(port_number) and check if it helps.