简体繁体 English

RabbitMQ将消息排入队列

[英]RabbitMQ keep messages in queue

原文 2015-10-25 18:55:11 5 1 ruby/ rabbitmq/ race-condition/ bunny

I am streaming a tty's stdout and stderr to RabbitMQ (logs to be exact). 我正在将tty的stdout和stderr流式传输到RabbitMQ（确切地说是日志）。 These logs can be viewed on a website and while the content is streamed to RabbitMQ they are consumed by the webserver and forwarded to the client using WebSockets. 这些日志可以在网站上查看，并且在将内容流式传输到RabbitMQ时，它们将由Web服务器使用，并使用WebSockets转发给客户端。 Logs are immediately persisted after sending it to RabbitMQ. 将日志发送到RabbitMQ后立即保留。

When the user accesses the website the persisted logs are rendered and the consecutive parts are streamed using WebSockets. 当用户访问网站时，将呈现持久日志，并使用WebSockets传输连续部分。 The problem is that there is a race condition as the persisted logs might be missing chunks of the log that occurred between rendering the site and receiving the first chunk via WebSocket. 问题在于存在争用条件，因为持久日志可能会丢失在呈现站点与通过WebSocket接收第一个块之间发生的日志块。

My idea was to keep all chunks in the queue and send those via the WebSocket after connecting. 我的想法是将所有块保留在队列中，并在连接后通过WebSocket发送这些块。 Additionally I would add a worker to listen to some kind of a "finished" event which then takes everything in the queue and persists it at once. 另外，我将添加一个工作程序来侦听某种“完成”事件，然后该事件将队列中的所有内容都保留并立即保留。

The problem is that I don't know if this is possible using RabbitMQ or how. 问题是我不知道是否可以使用RabbitMQ或如何做到这一点。 Any ideas or other solutions? 有什么想法或其他解决方案吗？

I don't think it really matters but my stack is using Ruby Sinatra and the Bunny RabbitMQ client. 我认为这并不重要，但是我的堆栈使用的是Ruby Sinatra和Bunny RabbitMQ客户端。

1 个解决方案

While I agree with your general idea about picking up where you left off, after loading the intial page, what you're trying to do isn't something that should be done from RabbitMQ. 虽然我同意您关于从上次中断的地方开始的一般想法，但是在加载初始页面后，您要尝试做的事情不是RabbitMQ应该做的。

There are a lot of potential problems that this would cause, which I've outlined in a blog post, previously . 这会导致很多潜在的问题，我之前在博客中已经概述了这些问题。

Instead of trying to do this w/ RMQ, I would do this from a database layer. 与其尝试通过RMQ进行操作，不如从数据库层进行操作。

As you push things into the database, you have an ID - hopefully one that is sequential. 当您将内容推送到数据库中时，您将获得一个ID-希望该ID是顺序的。 If not, add a sequence to the entries. 如果不是，则将序列添加到条目中。

When you load the page for the user, send the current ID that they are at down to the browser. 当您为用户加载页面时，将他们当前所在的ID发送到浏览器。

After the page finishes loading and you're setting up the websocket connection, send the user's current spot in the list of messages via the websocket. 页面加载完成并设置了websocket连接后，请通过websocket在消息列表中发送用户的当前位置。 then the websocket connection can use that id to say "give me all the messages after this id, and start streaming them" 那么websocket连接可以使用该ID来说“在此ID之后给我所有消息，然后开始流式传输”

Again, this is not done via RabbitMQ (see my article on why this is a bad idea ), but via your database and sequential IDs. 同样，这不是通过RabbitMQ来完成的（请参阅我的文章，为什么这是一个坏主意），而是通过数据库和顺序ID来完成。