C ++，OpenCV：在Windows上读取包含非ASCII字符的文件的最快方法

Question

I am writing a program using OpenCV that shall work on Windows as well as on Linux. Now the problem with OpenCV is, that its cv::imread function can not handle filepaths that contain non-ASCII characters on Windows. A workaround is to first read the file into a buffer using other libraries (for example std-libraries or Qt) and then read the file from that buffer using the cv::imdecode function. This is what I currently do. However, it's not very fast and much slower than just using cv::imread . I have a TIF image that is about 1GB in size. Reading it with cv::imread takes approx. 1s, reading it with the buffer method takes about 14s. I assume that imread just reads those parts of the TIF that are necessary for displaying the image (no layers etc.). Either this, or my code for reading a file into a buffer is bad.

Now my question is if there is a better way to do it. Either a better way with regard to OpenCV or a better way with regard to reading a file into a buffer.

I tried two different methods for the buffering, one using the std libraries and one using Qt (actually they both use QT for some things). They both are equally slow.:

Method 1

std::shared_ptr<std::vector<char>> readFileIntoBuffer(QString const& path) {

#ifdef Q_OS_WIN
    std::ifstream file(path.toStdWString(), std::iostream::binary);
#else
    std::ifstream file(path.toStdString(), std::iostream::binary);
#endif
    if (!file.good()) {
        return std::shared_ptr<std::vector<char>>(new std::vector<char>());
    }
    file.exceptions(std::ifstream::badbit | std::ifstream::failbit | std::ifstream::eofbit);
    file.seekg(0, std::ios::end);
    std::streampos length(file.tellg());
    std::shared_ptr<std::vector<char>> buffer(new std::vector<char>(static_cast<std::size_t>(length)));
    if (static_cast<std::size_t>(length) == 0) {
        return std::shared_ptr<std::vector<char>>(new std::vector<char>());
    }
    file.seekg(0, std::ios::beg);
    try {
        file.read(buffer->data(), static_cast<std::size_t>(length));
    } catch (...) {
        return std::shared_ptr<std::vector<char>>(new std::vector<char>());
    }
    file.close();
    return buffer;
}

And then for reading the image from the buffer:

std::shared_ptr<std::vector<char>> buffer = utility::readFileIntoBuffer(path);
cv::Mat image = cv::imdecode(*buffer, cv::IMREAD_UNCHANGED);

Method 2

QByteArray readFileIntoBuffer(QString const & path) {
    QFile file(path);
    if (!file.open(QIODevice::ReadOnly)) {
        return QByteArray();
    }
    return file.readAll();
}

And for decoding the image:

QByteArray buffer = utility::readFileIntoBuffer(path);
cv::Mat matBuffer(1, buffer.size(), CV_8U, buffer.data());
cv::Mat image = cv::imdecode(matBuffer, cv::IMREAD_UNCHANGED);

UPDATE

Method 3

This method maps the file into memory using QFileDevice::map and then uses cv::imdecode .

            QFile file(path);
            file.open(QIODevice::ReadOnly);
            unsigned char * fileContent = file.map(0, file.size(), QFileDevice::MapPrivateOption);
            cv::Mat matBuffer(1, file.size(), CV_8U, fileContent);
            cv::Mat image = cv::imdecode(matBuffer, cv::IMREAD_UNCHANGED);

However, also this approach didn't result in a shorter time than the other two. I also did some time measurements and found that reading the file in the memory or mapping it to the memory is actually not the bottleneck. The operation that takes the majority of the time is the cv::imdecode . I don't know why this is the case, since using cv::imread with the same image only takes a fraction of the time.

Potential Workaround

I tried obtaining an 8.3 pathname on Windows for files that contain non-ascii characters using the following code:

QString getShortPathname(QString const & path) {
#ifndef Q_OS_WIN
    return QString();
#else
    long length = 0;
    WCHAR* buffer = nullptr;
    length = GetShortPathNameW(path.toStdWString().c_str(), nullptr, 0);
    if (length == 0) return QString();
    buffer = new WCHAR[length];
    length = GetShortPathNameW(path.toStdWString().c_str(), buffer, length);
    if (length == 0) {
        delete[] buffer;
        return QString();
    }
    QString result = QString::fromWCharArray(buffer);
    delete[] buffer;
    return result;
#endif
}

However, I had to find out that 8.3 pathname generation is disabled on my machine, so it potentially is on others as well. So I wasn't able to test this yet and it does not seem to provide a reliable workaround. I also have the problem that the function doesn't tell me that 8.3 pathname generation is disabled.

Answer 1

There is an open ticket on this in OpenCV GitHub: https://github.com/opencv/opencv/issues/4292

One of the comments there suggest a workaround without reading the whole file to memory by using memory-mapped file (with help from Boost):

mapped_file map(path(L"filename"), ios::in);
Mat file(1, numeric_cast<int>(map.size()), CV_8S, const_cast<char*>(map.const_data()), CV_AUTOSTEP);
Mat image(imdecode(file, 1));