如何将实时日志写入 AWS Glue 日志

Question

So my problem is sometimes my Glue job takes a lot of time to run or even fails all together because of some inexplicable issues. Is there a way to get messages printed out to the output or error logs so that I can at least track where the job is right now or was at the end?

I tried the solution in How do I write messages to the output log on AWS Glue?

But that does not seem to be working for me. The Output log generates a ton of logs and even when I search or filter for keywords in my message, I am still not able to find them.

Right now I am considering writing messages to a S3 file to get around this issue. But it got me wondering if there isn't actually any other way to display messages that I can check while the job is running?

Answer 1

您可以编辑作业以使用连续记录功能并在那里跟踪您的记录器