简体繁体 English

如何使用nodejs设置HTTP服务器进行日志记录？

[英]How to setup HTTP server with nodejs for logging?

原文 2016-09-22 19:20:40 9 1 php/ node.js/ amazon-web-services

I need to setup a logging service for my startup. 我需要为启动设置日志记录服务。 I and my team do not have the time and resources (monetary) to deploy a dedicated tool for logs. 我和我的团队没有时间和资源（金钱）来部署专用于日志的工具。 However I need to run analytics on user behavior. 但是，我需要对用户行为进行分析。 While I am going to learn the analytics bit and implement it down the line, I need to log the user activity so that I can pull out the data later and do the math. 当我要学习分析知识并逐步实现它时，我需要记录用户活动，以便稍后可以提取数据并进行数学计算。

I am using a SQL based (RDBMS) setup for my main application (which is a web service) and plan to log user activity in DynamoDB (AWS). 我正在为我的主应用程序（这是一个Web服务）使用基于SQL的（RDBMS）设置，并计划在DynamoDB（AWS）中记录用户活动。

Also, I plan to setup a local (on the same server) Node.Js server which can handle log requests coming from my PHP based app to make sure that the logging activity runs in another process and is handled gracefully without crashing or slowing down the main server code. 另外，我计划设置一个本地（在同一服务器上）Node.Js服务器，该服务器可以处理来自我基于PHP的应用程序的日志请求， 以确保日志记录活动在另一个进程中运行并且可以正常处理，而不会导致崩溃或减慢运行主服务器代码。 The idea here is - when a request comes in from the user, the PHP application would make call to the NodeJS server. 这里的想法是-当用户发出请求时，PHP应用程序将调用NodeJS服务器。 The NodeJS server will take the request and return immediately while queuing the request (like say 200 other requests) and the PHP app will continue to serve the request. NodeJS服务器将接受请求并在排队请求时立即返回 （例如200个其他请求），而PHP应用将继续为该请求提供服务。 The NodeJS server will go on logging everything in the queue one after the other. NodeJS服务器将继续依次记录队列中的所有内容。

The reason why I plan to do this is - NodeJS runs on a single thread and is event-loop based - hence, in case there is a sudden traffic spike, log requests would be queued and the data will be sent to Dynamo eventually without making the PHP application wait for the confirmation. 我计划执行此操作的原因是-NodeJS在单个线程上运行并且基于事件循环-因此，如果出现突然的流量高峰，日志请求将排队，并且数据最终将发送到Dynamo，而不会PHP应用程序等待确认。

Note : When I say user activity , I mean application-level logging like: 注意：当我说用户活动时 ，我的意思是应用程序级日志记录，例如：

User X from ABCD IP address tried to access an article with ID 123 at 2016-09-10 14:21:22 来自ABCD IP地址的用户X尝试在2016-09-10 14:21:22访问ID为123的文章
User Y Posted Comment #234 on Article #987 at 2016-09-10 18:30:02 using ABC app. 用户Y使用ABC应用在2016-09-10 18:30:02发表了关于文章＃987的评论＃234。
User C logged in to the web interface from EFGH IP address. 用户C从EFGH IP地址登录到Web界面。

Of course, the data would be in JSON format and the server will make calls to the AWS SDK. 当然，数据将采用JSON格式，服务器将调用AWS开发工具包。

Here, the NodeJs server would basically be acting as a log-proxy. 在这里，NodeJs服务器将基本上充当日志代理。 Based on the idea/scenario so far, I have the following questions: 根据目前的想法/场景，我有以下问题：

Is the decision to use DynamoDB for logging requests okay or am I making a mistake? 是否决定使用DynamoDB记录请求，还是我做错了？ The service fits within my budget. 这项服务符合我的预算。
What kind of limitations will I have with NodeJS in this scenario? 在这种情况下，NodeJS会有哪些限制？ Is nodejs okay for it? nodejs可以吗？
What Node modules should I use for this? 我应该为此使用哪些节点模块？ Will HTTP server do or are there modules openly available which are made for such scenarios? HTTP服务器会做这种情况吗？有没有开放的模块可供使用？
How to I make sure that this server listens only to requests coming from the same machine (loopback)? 如何确保该服务器仅侦听来自同一计算机的请求（环回）？

1 个解决方案

Here will go an TL;DR; TL; DR; answer: 回答：

1 - Is the decision to use DynamoDB for logging requests okay or am I making a mistake? 1-是否决定使用DynamoDB记录请求，还是我做错了？ The service fits within my budget. 这项服务符合我的预算。

DynamoDB is an nosql database service that provides fast and predictable performance with seamless scalability, IMO it's a good way to use when we talk about loggin, as expected you need a fast writing and fast reading, without relationship between logs messages (I think). DynamoDB是一种nosql数据库服务，可提供快速且可预测的性能以及无缝的可伸缩性，IMO是我们谈论loggin时的一种很好的使用方式，因为期望您需要快速写入和快速阅读，而日志消息之间没有关系（我认为）。 So you can use DynamoDB without problems here. 因此，您可以在这里毫无问题地使用DynamoDB。

2 - What kind of limitations will I have with NodeJS in this scenario? 2-在这种情况下，NodeJS会有哪些限制？ Is nodejs okay for it? nodejs可以吗？

NodeJS is (as you said) single thread and is event-loop based, if you have more requests than NodeJS can handle with, you should fork or something like 如您所说，NodeJS是单线程且基于事件循环，如果您的请求超出NodeJS的处理能力，则应使用fork 或类似的方法

I still think that you should use a tool like Logstash but if you sure that you can use NodeJS to do that, probably you will have success. 我仍然认为您应该使用Logstash之类的工具，但是如果您确定可以使用NodeJS这样做，则可能会成功。

3 - What Node modules should I use for this? 3-我应该为此使用哪些节点模块？ Will HTTP server do or are there modules openly available which are made for such scenarios? HTTP服务器会做这种情况吗？有没有开放的模块可供使用？

When I talk about logging strategies, the first thing that coming in my mind is "thousands of packages over the network", I don't know if you have this in your startup, anyway I suggest you to use UDP to send the logg messages, UDP (User Datagram Protocol) that is used primarily for establishing low-latency and loss tolerating connections between applications on the Internet. 当我谈论日志记录策略时，我想到的第一件事是“网络上有成千上万个软件包”，我不知道启动时是否有此包，无论如何，我建议您使用UDP来发送logg消息， UDP（用户数据报协议），主要用于在Internet上的应用程序之间建立低延迟和容错连接。 You can startup a UDP NodeJS listener using, for example dgram . 您可以使用，启动一个UDP监听器的NodeJS 的例子 DGRAM 。 To send the UDP log messages to your NodeJS server, you can use the Winston loggin library with the winston-udp plugin . 要将UDP日志消息发送到NodeJS服务器，可以将Winston loggin库与winston-udp插件一起使用。 Or, in your case, from PHP server, you can send UDP messages with socket or take a look here . 或者，就您而言，从PHP服务器，您可以使用套接字发送UDP消息或在此处查看。 Disclaimer: I'm not specialist in PHP, so I Googled and I found these solutions to send log messages over UDP, I'm not sure if this is the best way to do that :) 免责声明：我不是PHP的专家，所以我用Google搜索了，发现了这些解决方案可以通过UDP发送日志消息，我不确定这是否是最好的方法:)