使用Python解码protobuf文件时出现一些错误，错误为'google.protobuf.message.DecodeError：解码varint时太多字节。'

Question

At first I directly run my decode program， the command is:

python test_read_protext.py

the error is :

Traceback (most recent call last):
  File "test_read_protext.py", line 23, in <module>
    read_protext("buck1")
  File "test_read_protext.py", line 17, in read_protext
    cur_bucket_conf.ParseFromString(str(ret))
  google.protobuf.message.DecodeError: Error parsing message

I have find some some solutions on the web, and I saw some guy add the environment parameter，like PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python, so i run the program as:

PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python python test_read_protext.py 

but it did't work , the error is:

Traceback (most recent call last):
  File "test_read_protext.py", line 23, in <module>
    read_protext("buck1")
  File "test_read_protext.py", line 17, in read_protext
    cur_bucket_conf.ParseFromString(str(ret))
  File "/home/gaoxiang2/.local/lib/python2.7/site-  packages/google/protobuf/message.py", line 187, in ParseFromString
    return self.MergeFromString(serialized)
  File "/home/gaoxiang2/.local/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 1124, in MergeFromString
    if self._InternalParse(serialized, 0, length) != length:
  File "/home/gaoxiang2/.local/lib/python2.7/site-packages/google/protobuf/internal/python_message.py", line 1176, in InternalParse
    buffer, new_pos, wire_type)  # pylint: disable=protected-access
  File "/home/gaoxiang2/.local/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 948, in _DecodeUnknownField
    (data, pos) = _DecodeUnknownFieldSet(buffer, pos)
  File "/home/gaoxiang2/.local/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 927, in _DecodeUnknownFieldSet
    (data, pos) = _DecodeUnknownField(buffer, pos, wire_type)
  File "/home/gaoxiang2/.local/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 938, in _DecodeUnknownField
    (data, pos) = _DecodeVarint(buffer, pos)
  File "/home/gaoxiang2/.local/lib/python2.7/site-packages/google/protobuf/internal/decoder.py", line 135, in DecodeVarint
    raise _DecodeError('Too many bytes when decoding varint.')
google.protobuf.message.DecodeError: Too many bytes when decoding varint.

the proto file is :

syntax="proto3";

message StrategyInfo {
    string strategy_ins_name = 1;            
    int32 res_polling_num = 2;              
    int32 strategy_recall_upper_limit = 3;   
};

message SingleBucket {
    string bucket_name = 1;                  
    int32 bucket_priority = 2;               
    int32 bucket_recall_res_proportion = 3; 
    repeated StrategyInfo strategies = 4;    
};

message StrategyBucketConf {
    int32 bucket_num = 1;                    
    repeated SingleBucket buckets = 2;    
};

the real file is like:

bucket_num : 1
buckets {
    bucket_name : "hot_class"
    bucket_priority : 0
    bucket_recall_res_proportion : 60
    strategies {
        strategy_ins_name : "short_term_chot_v0"
        res_polling_num : 20
        strategy_recall_upper_limit : 100
    }
    strategies {
        strategy_ins_name : "recall_finish_hot"
        res_polling_num : 20
        strategy_recall_upper_limit : 100
    }
}

the decode program is:

def read_protext(msg) :
    cur_bucket_conf = bucket_conf.StrategyBucketConf()
    f = open(msg, 'rb')
    ret = f.read()  
    print ret
    cur_bucket_conf.ParseFromString(ret) #read from file
    f.close()
    print cur_bucket_conf.bucket_num

Answer 1

I have solve this question.

instead of

cur_bucket_conf.ParseFromString(ret) 

by

from google.protobuf import text_format
text_format.Parse(ret, cur_bucket_conf)