[英]how do you decode gtfs protobufs in node
I am trying to use https://github.com/dcodeIO/ProtoBuf.js to parse triments gtfs data. 我正在尝试使用https://github.com/dcodeIO/ProtoBuf.js解析gtfs数据片段。
Here is the code I have so far, it parses the .proto file correctly and creates the builder and has all the expected properties and methods, it throws an error when I try to decode any data with it. 这是我到目前为止的代码,它可以正确解析.proto文件并创建生成器,并具有所有预期的属性和方法,当我尝试使用它解码任何数据时,它将引发错误。
Error: Data must be corrupt: Buffer overrun
the proto file is from https://developers.google.com/transit/gtfs-realtime/gtfs-realtime-proto 原始文件来自https://developers.google.com/transit/gtfs-realtime/gtfs-realtime-proto
var ProtoBuf = require('protobufjs')
, request = require('request')
var transit = ProtoBuf.protoFromFile('gtfs-realtime.proto').build('transit_realtime')
request('http://developer.trimet.org/ws/V1/FeedSpecAlerts/?appID=618F30BB3062F39AF24AED9EC', parse)
function parse(err, res, body) {
try {
console.log(transit.FeedMessage.decode(res.body))
} catch(e) {
console.log(e)
}
}
Thanks to Brian Ferris I able to parse the first part of the header gtfs_realtime_version: "1"
but the parser fails on the next component (the time stamp uint64) 多亏了Brian Ferris,我能够解析标题
gtfs_realtime_version: "1"
的第一部分gtfs_realtime_version: "1"
但是解析器在下一个组件上失败(时间戳uint64)
Thanks to 谢谢
I kept finding your question when searching for the same issue you were having and hopefully I can help someone else. 在搜索您遇到的相同问题时,我一直在寻找您的问题,希望我能帮助其他人。 After scouring the internet for a lot longer than I should have, I've come up with something that works.
在搜寻互联网的时间长于我应有的时间之后,我想出了一些可行的方法。 I didn't quite understand the data until I had a working feed decoded.
在对有效的供稿进行解码之前,我不太了解数据。
It largely appears that this has to do with how the data is being read. 似乎很大程度上与读取数据的方式有关。 I'm not a NodeJS person so I don't know why, but it's dependent on how the data is read with
http
rather than request
for decoding. 我不是NodeJS使用者,所以我不知道为什么,但这取决于如何通过
http
读取数据,而不是request
解码。 I couldn't get the same method to work with request
for the data. 我无法使用相同的方法来处理数据
request
。
Part of this I found from https://github.com/dcodeIO/ProtoBuf.js/wiki/How-to-read-binary-data-in-the-browser-or-under-node.js%3F but I didn't quite yet understand how to use protobufjs
, so I'm putting a working example here for others. 我从https://github.com/dcodeIO/ProtoBuf.js/wiki/How-to-read-binary-data-in-the-browser-or-under-node.js%3F找到了部分内容,但我没有尚未了解如何使用
protobufjs
,因此我在这里为其他人提供了一个有效的示例。 Hope it helps. 希望能帮助到你。
var ProtoBuf = require('protobufjs');
var http = require("http");
// create a protobuf decoder
var transit = ProtoBuf.protoFromFile('gtfs-realtime.proto').build('transit_realtime');
// your protobuf binary feed URL
var feedUrl = "...";
// HTTP GET the binary feed
http.get(feedUrl, parse);
// process the feed
function parse(res) {
// gather the data chunks into a list
var data = [];
res.on("data", function(chunk) {
data.push(chunk);
});
res.on("end", function() {
// merge the data to one buffer, since it's in a list
data = Buffer.concat(data);
// create a FeedMessage object by decooding the data with the protobuf object
var msg = transit.FeedMessage.decode(data);
// do whatever with the object
console.log(msg);
});
});
I was able to get this to work (with New York MTA feeds, anyway) by forcing the request
module to have a null encoding, thus ensuring it returns a buffer instead of a string. 通过强制
request
模块具有空编码,我能够使它正常工作(无论如何与纽约MTA提要一起使用),从而确保它返回缓冲区而不是字符串。 Like so: 像这样:
request({
url: 'http://developer.trimet.org/ws/V1/FeedSpecAlerts/?appID=618F30BB3062F39AF24AED9EC'
encoding: null
}, parse)
Then the parsing appears to work fine. 然后,解析似乎可以正常工作。
From the Google Developers page at developers.google.com/transit/gtfs-realtime/code-samples . 在Google Developers页面的developers.google.com/transit/gtfs-realtime/code-samples上 。 Google has now made a Node.js npm module available to make things very easy:
Google现在提供了Node.js npm模块,使事情变得非常简单:
npm install gtfs-realtime-bindings
Here's Google's code snippet ( Apache 2.0 License ) 这是Google的代码段( Apache 2.0许可 )
var GtfsRealtimeBindings = require('gtfs-realtime-bindings');
var request = require('request');
var requestSettings = {
method: 'GET',
url: 'URL OF YOUR GTFS-REALTIME SOURCE GOES HERE',
encoding: null
};
request(requestSettings, function (error, response, body) {
if (!error && response.statusCode == 200) {
var feed = GtfsRealtimeBindings.FeedMessage.decode(body);
feed.entity.forEach(function(entity) {
if (entity.trip_update) {
console.log(entity.trip_update);
}
});
}
});
This does not answer your issue but you can get RT feed in text using url like http://developer.trimet.org/ws/V1/FeedSpecAlerts/appid/618F30BB3062F39AF24AED9EC/text/true 这不能解决您的问题,但是您可以使用诸如http://developer.trimet.org/ws/V1/FeedSpecAlerts/appid/618F30BB3062F39AF24AED9EC/text/true这样的网址获取RT Feed文本
I'm not a node expert, but the root message type of a GTFS-realtime feed is "FeedMessage": 我不是节点专家,但是GTFS实时供稿的根消息类型是“ FeedMessage”:
https://developers.google.com/transit/gtfs-realtime/reference https://developers.google.com/transit/gtfs-realtime/reference
You seem to be trying to parse the feed as an "Alert" message: 您似乎正在尝试将提要解析为“警告”消息:
console.log(transit.Alert.decode(res.body))
Maybe try changing Alert to FeedMessage and see what happens? 也许尝试将Alert更改为FeedMessage并查看会发生什么?
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.