[英]Why does locking slow down this sequential file parser?
I wrote a simple reader and parser for a graph file format. 我为图形文件格式编写了一个简单的阅读器和解析器。 The problem is that it is incredibly slow.
问题是它非常慢。 Here are the relevant methods:
以下是相关方法:
Graph METISGraphReader::read(std::string path) {
METISParser parser(path);
std::pair<int64_t, int64_t> header = parser.getHeader();
int64_t n = header.first;
int64_t m = header.second;
Graph G(n);
node u = 0;
while (parser.hasNext()) {
u += 1;
std::vector<node> adjacencies = parser.getNext();
for (node v : adjacencies) {
if (! G.hasEdge(u, v)) {
G.insertEdge(u, v);
}
}
}
return G;
}
std::vector<node> METISParser::getNext() {
std::string line;
bool comment = false;
do {
comment = false;
std::getline(this->graphFile, line);
// check for comment line starting with '%'
if (line[0] == '%') {
comment = true;
TRACE("comment line found");
} else {
return parseLine(line);
}
} while (comment);
}
static std::vector<node> parseLine(std::string line) {
std::stringstream stream(line);
std::string token;
char delim = ' ';
std::vector<node> adjacencies;
// split string and push adjacent nodes
while (std::getline(stream, token, delim)) {
node v = atoi(token.c_str());
adjacencies.push_back(v);
}
return adjacencies;
}
To diagnose why it is so slow, I ran it in a profiler (Apple Instruments). 为了诊断它为什么这么慢,我在一个分析器(Apple Instruments)中运行它。 The results were surprising: It's slow because of locking overhead.
结果令人惊讶:由于锁定开销,它很慢。 The program spends over 90% of its time in
pthread_mutex_lock
and _pthread_cond_wait
. 该程序将超过90%的时间花在
pthread_mutex_lock
和_pthread_cond_wait
。
I have no idea where the locking overhead comes from, but I need to get rid of it. 我不知道锁定开销来自哪里,但我需要摆脱它。 Can you suggest next steps?
你能建议下一步吗?
EDIT: See the call stack expanded for _pthread_con_wait
. 编辑:查看为
_pthread_con_wait
扩展的调用堆栈。 I cannot figure out the source of the locking overhead by looking at this: 通过查看这个,我无法弄清楚锁定开销的来源:
Expand the call stack on the _pthread_cond_wait and pthread_mutex_lock calls to find out where the locking calls are invoked from. 展开_pthread_cond_wait和pthread_mutex_lock调用上的调用堆栈,以找出调用锁定调用的位置。
As a guess I'm going to say it's in all the unnecessary heap allocations you're doing. 作为一个猜测,我会说它在你所做的所有不必要的堆分配中。 The heap is a thread safe resource and on this platform the thread safety could be provided via mutexes.
堆是一个线程安全的资源,在这个平台上,线程安全可以通过互斥锁提供。
All functions that read data from an istream
will lock a mutex,
read data from a streambuf
and unlock the mutex
. 从
istream
读取数据的所有函数都将锁定mutex,
从streambuf
读取数据并解锁mutex
。 To eliminate that overhead, read the file directly from the streambuf
instead of the istream
and don't use stringstream
to parse the data. 要消除这种开销,请直接从
streambuf
而不是istream
读取文件,不要使用stringstream
来解析数据。
Here is a version of getline
that uses streambuf
instead of istream
这是一个使用
streambuf
而不是istream
的getline
版本
bool fastGetline(streambuf* sb, std::string& t)
{
t.clear();
for(;;) {
int c = sb->sbumpc();
switch (c) {
case '\n':
return true;
case '\r':
if(sb->sgetc() == '\n')
sb->sbumpc();
return true;
case EOF:
return !t.empty();
default:
t += (char)c;
}
}
声明:本站的技术帖子网页,遵循CC BY-SA 4.0协议,如果您需要转载,请注明本站网址或者原文地址。任何问题请咨询:yoyou2525@163.com.