[英]Static Variables and Thread-Local Storage
Background: 背景:
I have discovered something of an interesting edge case relating to static memory initialization across multiple threads. 我发现了一些与多线程静态内存初始化有关的有趣边缘情况。 Specifically, I am using Howard Hinnant's TZ library which has been working fine for the rest of my code across many different threads. 具体来说,我使用的是Howard Hinnant的TZ库,它在许多不同的线程中对我的其余代码工作正常。
Now, I am developing a logging class which relies on yet another thread and condition variable. 现在,我正在开发一个依赖于另一个线程和条件变量的日志类。 Unfortunately, when I attempt to format a chrono time_point
using date::make_zoned(data::locate_zone("UTC"), tp)
the library crashes. 不幸的是,当我试图格式化一个时辰time_point
使用date::make_zoned(data::locate_zone("UTC"), tp)
库崩溃。 Upon digging through tz.cpp
, I find that the time zone database returned internally is evaluating to NULL
. 在挖掘tz.cpp
,我发现内部返回的时区数据库正在评估为NULL
。 This all comes from the following snippet: 这一切都来自以下代码段:
tzdb_list&
get_tzdb_list()
{
static tzdb_list tz_db = create_tzdb();
return tz_db;
}
As can be seen, the database list is stored statically. 可以看出,数据库列表是静态存储的。 With a few printf()s and some time with GDB I can see that the same db is returned for multiple calls from the main thread but returns NULL
when called from my logger thread. 使用一些printf()和一些GDB时间,我可以看到从主线程多次调用返回相同的db,但是从我的记录器线程调用时返回NULL
。
If, however, I change the declaration of tzdb_list
to: 但是,如果我将tzdb_list
的声明tzdb_list
为:
static thread_local tzdb_list tz_db = create_tzdb();
Everything works as expected. 一切都按预期工作。 This is not surprising as thread_local
will cause each thread to do the heavy-lifting of creating a standalone instance of tzdb_list
. 这并不奇怪,因为thread_local
将导致每个线程完成创建tzdb_list
的独立实例的tzdb_list
。 Obviously this is wasteful of memory and can easily cause problems later. 显然这会浪费内存,以后很容易引起问题。 As such, I really don't see this as a viable solution. 因此,我真的不认为这是一个可行的解决方案。
Questions: 问题:
What about the invocation of one thread versus another would cause static memory to behave differently? 一个线程与另一个线程的调用会导致静态内存的行为有何不同? If anything, I would expect the opposite of what is happening (eg. for the threads to 'fight' over initialized memory; not have one receive a NULL
pointer). 如果有的话,我会期望与正在发生的事情相反(例如,线程在初始化内存上“争夺”;没有一个接收到NULL
指针)。
How is it possible for a returned static reference to have multiple different values in the first place (in my case, valid memory versus NULL
)? 返回的静态引用如何首先有多个不同的值(在我的例子中,有效内存与NULL
)?
With thread_local
built into the library I get wildly different memory locations on opposite ends of the addressable region; 随着thread_local
内置到库中,我在可寻址区域的两端获得了截然不同的内存位置; why? 为什么? I suspect that this has to do with where thread memory is allocated versus the main process memory but do not know the exact details of thread allocation regions. 我怀疑这与线程内存分配的位置与主进程内存有关,但不知道线程分配区域的确切细节。
Reference: 参考:
My logging thread is created with: 我的日志记录线程创建时使用:
outputThread = std::thread(Logger::outputHandler, &outputQueue);
And the actual output handler / invocation of the library ( LogMessage
is just a typedef for std::tuple
): 而实际的输出处理程序/库的调用( LogMessage
只是std::tuple
的typedef):
void Logger::outputHandler(LogQueue *queue)
{
LogMessage entry;
std::stringstream ss;
while (1)
{
queue->pop(entry); // Blocks on a condition variable
ss << date::make_zoned(date::locate_zone("UTC"), std::get<0>(entry))
<< ":" << levelId[std::get<1>(entry)
<< ":" << std::get<3>(entry) << std::endl;
// Printing stuff
ss.str("");
ss.clear();
}
}
Additional code and output samples available on request. 可根据要求提供其他代码和输出样本。
EDIT 1 编辑1
This is definitely a problem in my code. 这绝对是我的代码中的一个问题。 When I strip everything out my logger works as expected. 当我删除所有内容时,我的记录器按预期工作。 What is strange to me is that my test case in the full application is just two prints in main and a call to the logger before manually exiting. 对我来说很奇怪的是,我在完整应用程序中的测试用例只是在main中打印两次,在手动退出之前调用logger。 None of the rest of the app initialization is run but I am linking in all support libraries at that point (Microsoft CPP REST SDK, MySQL Connector for C++ and Howard's date library (static)). 其余的应用程序初始化都没有运行,但我在此时链接所有支持库(Microsoft CPP REST SDK,MySQL Connector for C ++和Howard的日期库(静态))。
It is easy for me to see how something could be stomping this memory but, even in the "full" case in my application, I don't know why the prints on the main thread would work but the next line calling into the logger would fail. 我很容易看到有什么东西可以踩踏这个内存但是,即使在我的应用程序中的“完整”情况下,我也不知道为什么主线程上的打印会起作用,但下一行调用记录器会失败。 If something were going sideways at init I would expect all calls to break. 如果在初始阶段横向发生某些事情,我希望所有的电话都能打破。
I also noticed that if I make my logger static the problem goes away. 我还注意到,如果我使记录器保持静态,问题就会消失。 Of course, this changes the memory layout so it doesn't rule out heap / stack smashing. 当然,这会改变内存布局,因此不排除堆/堆栈粉碎。 What I do find interesting is that I can declare the logger globally or on the stack at the start of main()
and both will segfault in the same way. 我觉得有趣的是我可以在main()
的开头全局或堆栈上声明记录器,并且两者都会以相同的方式进行段错误。 If I declare the logger as static, however, both global and stack-based declaration work. 但是,如果我将logger声明为static,则全局和基于堆栈的声明都会起作用。
Still trying to create a minimal test case which reproduces this. 仍然试图创建一个再现这个的最小测试用例。
I am already linking with -lpthread
; 我已经用-lpthread
链接了; have been pretty much since the inception of this application. 自从这个应用程序开始以来已经非常多了。
OS is Fedora 27 x86_64 running on an Intel Xeon. 操作系统是在Intel Xeon上运行的Fedora 27 x86_64。 Compiler: 编译:
$ g++ --version
g++ (GCC) 7.3.1 20180130 (Red Hat 7.3.1-2)
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
It appears that this problem was caused by a bug in tz.cpp which has since been fixed . 看来这个问题是由tz.cpp中的一个错误引起的,后来修复了 。
The bug was that there was a namespace scope variable whose initialization was not guaranteed in the proper order. 错误是有一个命名空间范围变量,其初始化不能以正确的顺序保证。 This was fixed by turning that variable into a function-local-static to ensure the proper initialization order. 通过将该变量转换为函数local-static来确定正确的初始化顺序,从而解决了这个问题。
My apologies to all who might have been impacted by this bug. 我向所有可能受到这个bug影响的人道歉。 And my thanks to all those who have reported it. 我要感谢所有报道过的人。
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.