使用 VLC imem 從內存播放 h264 視頻文件但收到錯誤“主流錯誤：無法預填充緩沖區”

Question

我有一個加載到內存中的 h264 視頻文件，我嘗試使用參數“imem-cat=4”使用 imem 播放它，以便 vlc 將使用訪問模塊來解復用視頻，並且 vlc 啟動並接收我的 imem參數成功：

[0x7f38a0000e28] access_imem demux debug: Using get(0x404e1d), release(0x404e91), data(0x7fff5b4a9430), cookie("IMEM")

這個類別也意味着我不必提供 DTS 和 PTS。 VLC 的 imem 模塊沒有很好的文檔記錄，但我在幾個地方找到了提示，例如

https://forum.videolan.org/viewtopic.php?t=111917

https://forum.videolan.org/viewtopic.php?f=32&t=93842

在 python 中使用 libVLC 從內存播放視頻

我的 imem-get 函數只是在第一次調用時將緩沖區指針設置為視頻數據，返回 0，在任何進一步調用中，它返回 1 以指示沒有更多數據：

int MyImemGetCallback (void *data,
                   const char *cookie,
                   int64_t *dts,
                   int64_t *pts,
                   unsigned *flags,
                   size_t * bufferSize,
                   void ** buffer)
{

ImemData* imem = (ImemData*)data;
cookie = imem->cookieString;

if(imem == NULL || imem->allBuffered==true) //indicate all data has been get()ted
    return 1;

*buffer = (void*) imem->video;
bufferSize = (size_t*) &(imem->bytes);
imem->allBuffered=true;

return 0;
}

不幸的是，在第一次通話后，我收到以下錯誤：

[0x189cb18] main input debug: Creating an input for 'imem://'
[0x189cb18] main input debug: using timeshift granularity of 50 MiB, in path '/tmp'
[0x189cb18] main input debug: `imem://' gives access `imem' demux `' path `'
[0x189cb18] main input debug: creating demux: access='imem' demux='' location='' file='(null)'
[0x7f2808000e28] main demux debug: looking for access_demux module matching "imem": 20 candidates
[0x7f2808000e28] access_imem demux debug: Using get(0x404e1d), release(0x404e91), data(0x7ffe0da3b940), cookie("h264")
[0x7f2808000e28] main demux debug: no access_demux modules matched
[0x189cb18] main input debug: creating access 'imem' location='', path='(null)'
[0x7f2808001958] main access debug: looking for access module matching "imem": 25 candidates
[0x7f2808001958] access_imem access debug: Using get(0x404e1d), release(0x404e91), data(0x7ffe0da3b940), cookie("h264")
[0x7f2808001958] main access debug: using access module "access_imem"
[0x7f2808000e28] main stream debug: Using block method for AStream*
[0x7f2808000e28] main stream debug: starting pre-buffering
[0x7f2808000e28] main stream error: cannot pre fill buffer
[0x7f2808001958] main access debug: removing module "access_imem"
[0x189cb18] main input warning: cannot create a stream_t from access
[0x17d7298] main libvlc debug: removing all interfaces
[0x17d7298] main libvlc debug: exiting
[0x17d7298] main libvlc debug: no exit handler
[0x17d7298] main libvlc debug: removing stats

出於某種原因，似乎 vlc 無法訪問視頻數據，但錯誤消息不是很有幫助，通常指的是網絡流而不是內存位置。

有沒有人以這種方式成功使用 imem 或對可能出現的問題有任何想法？ 視頻從磁盤完美地在 VLC 中播放。 謝謝你的幫助。

編輯

看起來項目界面實際上可能不支持以這種方式播放。 但是，libVLC 提供了 libvlc_media_t 和 livblc_media_new_callbacks，它們可以讓我實現我想要的。 如果我得到它的工作，我會回來報告。

Answer 1

所以我無法讓 Imem 工作，但是在 VLC 論壇上，我指出了 3.0.0 版中可用的新 API。 我必須刪除我當前安裝的 vlc 和 libvlc-dev，並將 VLC 每日構建 PPA 添加到 Ubuntu 安裝，然后安裝這些版本。

API 是libvlc_media_new_callbacks ：

LIBVLC_API libvlc_media_t * libvlc_media_new_callbacks
(
    libvlc_instance_t *instance, 
    libvlc_media_open_cb open_cb, 
    libvlc_media_read_cb read_cb, 
    libvlc_media_seek_cb seek_cb, 
    libvlc_media_close_cb close_cb, 
    void *opaque
);

您必須實現這些回調中的每一個才能讓 VLC 訪問內存中的流。 盡管文檔指出實現 seek() 回調是不必要的，但沒有它我無法播放 h264 視頻。

open() 回調應該傳遞一個指向您的視頻數據的指針，我推薦一個容器類，以便您可以存儲與它一起讀取的最后一個字節的索引。

read() 回調用於將 len 字節讀入緩沖區，並為其傳遞指針。 在這里，您可以將 len 或更少字節寫入緩沖區並返回復制的字節數，阻塞直到准備好一些字節，返回 0 表示 EOF 或返回 -1 表示錯誤。

seek() 回調用於設置下一次 read() 將發生的字節索引。

最后 close() 不返回任何內容並用於整理。

下面是一個 read() 實現的例子：

class MemVideoData
{
    public:
        MemVideoData(char *data, int data_bytes) : video(data), bytes(data_bytes), lastByteIndex(0) {}   //init
        ~MemVideoData() {}
        char* video;   //pointer to video in memory
        int bytes;
        int lastByteIndex;
};

ssize_t memVideo_read(void *opaque, unsigned char* buf, size_t len)
{
  //TODO: block if not end of stream but no bytes available
  MemVideoData *mVid = (MemVideoData*) opaque;    //cast and give context
  int bytesToCopy=0;
  int bytesSoFar = mVid->lastByteIndex;
  int bytesRemaining = mVid->bytes - mVid->lastByteIndex;
  
  if(bytesRemaining >= len) bytesToCopy = len;   //at least as many bytes remaining as requested
  else if (bytesRemaining < len) bytesToCopy = bytesRemaining;    //less that requested number of bytes remaining
  else return 0;   // no bytes left to copy
  
  char *start = mVid->video;
  std::memcpy(buf,&start[mVid->lastByteIndex], bytesToCopy);    //copy bytes requested to buffer.
  mVid->lastByteIndex = mVid->lastByteIndex + bytesToCopy;    //increment bytes read count
  
  return bytesToCopy;
}

根據這里的要求，這是一個 Open 回調的示例：

int VideoPlayer::memVideo_open(void* opaque, void** datap, uint64_t* sizep)
{
   //TODO: get mutex on MemVideoData object pointed to by opaque
   MemVideoData *mVid = static_cast<MemVideoData*> (opaque); //cast opaque to our video state struct
   *sizep = (uint64_t) mVid->bytesTotal;    //set stream length
   *datap = mVid; // point to entire object. Think this was intended to point to the raw video data but we use the MemVideoData object in read() and seek()
   mVid->lastByteReadIndex=0;
   return 0;
}

Answer 2

看看下面我的 Qt 示例，它有效。 實際上我不想讀取整個文件並將其存儲在 ram 中。 所以我實現了 MediaDescriptor，因為我將在其中實現一個解密邏輯來讀取加密文件。 順便說一句，我使用了 libvlc 3.0.6 x64 預建庫，看起來運行良好。

媒體描述符.h

#pragma once

#include <QObject>
#include <fstream>

class MediaDescriptor : public QObject
{
    Q_OBJECT

public:
    MediaDescriptor(QString mediaFilePath);
    ~MediaDescriptor();

    bool tryOpen();
    uint64_t getMediaLength();
    uint64_t getMediaBytes(unsigned char *buffer, uint64_t length);
    void setSeek(uint64_t seek);

private:
    QString m_mediaFilePath;
    std::ifstream *m_mediaFile;
    uint64_t m_mediaLength;
    uint64_t m_seek;
};

媒體描述符文件

#include "MediaDescriptor.h"

MediaDescriptor::MediaDescriptor(QString mediaFilePath)
    : m_mediaFilePath(mediaFilePath), m_mediaFile(nullptr), m_mediaLength(0), m_seek(0)
{
}

MediaDescriptor::~MediaDescriptor()
{
    if (m_mediaFile)
    {
        m_mediaFile->close();
        delete m_mediaFile;
    }
}

bool MediaDescriptor::tryOpen()
{
    m_mediaFile = new std::ifstream(m_mediaFilePath.toStdString().c_str(), std::ios::binary | std::ios::ate);

    if (m_mediaFile->is_open())
    {
        m_mediaFile->seekg(0, m_mediaFile->end);
        m_mediaLength = m_mediaFile->tellg();
        return true;
    }

    delete m_mediaFile;
    return false;
}

uint64_t MediaDescriptor::getMediaLength()
{
    return m_mediaLength;
}

uint64_t MediaDescriptor::getMediaBytes(unsigned char *buffer, uint64_t length)
{
    // to do: decrytion logic
    if (m_mediaFile->is_open())
    {
        uint64_t len = length;
        if (m_seek + len > m_mediaLength)
            len = (size_t)(m_mediaLength - m_seek);

        if (len > 0)
        {
            m_mediaFile->seekg(m_seek);
            m_mediaFile->read((char*)&buffer[0], len);
            m_seek += len;
        }

        return len;
    }

    return -1;
}

void MediaDescriptor::setSeek(uint64_t seek)
{
    m_seek = seek;
}

VLCHelper.h

#pragma once

#include <QObject>
#include <QWidget>
#include <QTime>
#include <mutex>
#include "vlc/vlc.h"
#include "MediaDescriptor.h"

class VLCHelper : public QObject
{
    Q_OBJECT

public:
    ~VLCHelper();
    static VLCHelper& getInstance();

    int vlcMediaOpenCallback(void* opaque, void** datap, uint64_t* sizep);
    int vlcMediaReadCallback(void *opaque, unsigned char* buf, size_t len);
    int vlcMediaSeekCallback(void *opaque, uint64_t offset);
    void vlcMediaCloseCallback(void *opaque);

    void initMedia(MediaDescriptor &mediaDescriptor, QWidget *output = nullptr, int volume = 100, bool repeat = false);
    void destroyMedia();
    bool isMediaValid();

    Q_INVOKABLE void playPauseMedia(bool play);
    bool isMediaPlaying();
    Q_INVOKABLE void stopMedia();

    void setRepeatMedia(bool repeat);
    bool getRepeatMedia();

    void setMediaPosition(float position);
    float getMediaPosition();

    QTime getMediaTime();
    QTime getMediaTotalTime();

    void setMediaVolume(int volume);
    int getMediaVolume();

signals:
    void mediaEOFReached();
    void error(QString error);

private:
    VLCHelper();

    std::mutex m_callbackMutex;
    libvlc_instance_t *m_vlcInstance;
    libvlc_media_t *m_vlcMedia;
    libvlc_media_player_t *m_vlcMediaPlayer;
    bool m_repeat;
    bool m_stopRequested;
    MediaDescriptor *m_mediaDescriptor;
    QWidget *m_output;
};

VLCHelper.cpp

#include "VLCHelper.h"

#pragma region Callback Wrappers
extern "C" int vlcMediaOpenCallbackGateway(void* opaque, void** datap, uint64_t* sizep)
{
    return VLCHelper::getInstance().vlcMediaOpenCallback(opaque, datap, sizep);
}

extern "C" int vlcMediaReadCallbackGateway(void *opaque, unsigned char* buf, size_t len)
{
    return VLCHelper::getInstance().vlcMediaReadCallback(opaque, buf, len);
}

extern "C" int vlcMediaSeekCallbackGateway(void *opaque, uint64_t offset)
{
    return VLCHelper::getInstance().vlcMediaSeekCallback(opaque, offset);
}

extern "C" void vlcMediaCloseCallbackGateway(void *opaque)
{
    VLCHelper::getInstance().vlcMediaCloseCallback(opaque);
}
#pragma endregion

VLCHelper::VLCHelper()
    : QObject(nullptr),
    m_vlcInstance(nullptr),
    m_vlcMedia(nullptr),
    m_vlcMediaPlayer(nullptr),
    m_repeat(false),
    m_stopRequested(false)
{
}

VLCHelper::~VLCHelper()
{
}

VLCHelper& VLCHelper::getInstance()
{
    static VLCHelper ins;
    return ins;
}

void VLCHelper::initMedia(MediaDescriptor &mediaDescriptor, QWidget *output, int volume, bool repeat)
{
    m_mediaDescriptor = &mediaDescriptor;
    m_output = output;
    m_repeat = repeat;

    m_vlcInstance = libvlc_new(0, NULL);

    m_vlcMedia = libvlc_media_new_callbacks(
        m_vlcInstance,
        vlcMediaOpenCallbackGateway,
        vlcMediaReadCallbackGateway,
        vlcMediaSeekCallbackGateway,
        vlcMediaCloseCallbackGateway,
        0
    );

    m_vlcMediaPlayer = libvlc_media_player_new_from_media(m_vlcMedia);

#if defined(Q_OS_WIN)
    libvlc_media_player_set_hwnd(m_vlcMediaPlayer, output ? (void*)output->winId() : nullptr);
#elif defined(Q_OS_MAC)
    libvlc_media_player_set_nsobject(m_vlcMediaPlayer, output ? (void*)output->winId() : nullptr);
#elif defined(Q_OS_LINUX)
    libvlc_media_player_set_xwindow(m_vlcMediaPlayer, output ? (void*)output->winId() : nullptr);
#endif

    libvlc_audio_set_volume(m_vlcMediaPlayer, volume);

    m_mediaDescriptor->setSeek(0);
}

void VLCHelper::destroyMedia()
{
    if (!m_vlcInstance)
        return;

    if (m_vlcMediaPlayer)
    {
        libvlc_media_player_stop(m_vlcMediaPlayer);
        libvlc_media_player_release(m_vlcMediaPlayer);
        m_vlcMediaPlayer = nullptr;
    }

    libvlc_release(m_vlcInstance);
    m_vlcInstance = nullptr;
}

bool VLCHelper::isMediaValid()
{
    return m_vlcInstance && m_vlcMedia && m_vlcMediaPlayer;
}

void VLCHelper::playPauseMedia(bool play)
{
    m_stopRequested = false;

    if (isMediaValid())
        play ? libvlc_media_player_play(m_vlcMediaPlayer) : libvlc_media_player_pause(m_vlcMediaPlayer);
    else
        emit error("TO DO");
}

bool VLCHelper::isMediaPlaying()
{
    if (isMediaValid())
        return libvlc_media_player_is_playing(m_vlcMediaPlayer);

    return false;
}

void VLCHelper::stopMedia()
{
    m_stopRequested = true;

    if (isMediaValid())
        libvlc_media_player_stop(m_vlcMediaPlayer);
    else
        emit error("TO DO");
}

void VLCHelper::setRepeatMedia(bool repeat)
{
    m_repeat = repeat;
}

bool VLCHelper::getRepeatMedia()
{
    return m_repeat;
}

void VLCHelper::setMediaPosition(float position)
{
    if (isMediaValid())
        libvlc_media_player_set_position(m_vlcMediaPlayer, position);
    else
        emit error("TO DO");
}

float VLCHelper::getMediaPosition()
{
    if (isMediaValid())
        return libvlc_media_player_get_position(m_vlcMediaPlayer);
    else
        emit error("TO DO");

    return -1.0;
}

QTime VLCHelper::getMediaTime()
{
    if (isMediaValid())
        return QTime::fromMSecsSinceStartOfDay((int)libvlc_media_player_get_time(m_vlcMediaPlayer));
    else
        emit error("TO DO");

    return QTime();
}

QTime VLCHelper::getMediaTotalTime()
{
    if (isMediaValid())
        return QTime::fromMSecsSinceStartOfDay((int)libvlc_media_player_get_length(m_vlcMediaPlayer));
    else
        emit error("TO DO");

    return QTime();
}

void VLCHelper::setMediaVolume(int volume)
{
    if (isMediaValid())
        libvlc_audio_set_volume(m_vlcMediaPlayer, volume);
    else
        emit error("TO DO");
}

int VLCHelper::getMediaVolume()
{
    if (isMediaValid())
        return libvlc_audio_get_volume(m_vlcMediaPlayer);
    else
        emit error("TO DO");

    return -1;
}

int VLCHelper::vlcMediaOpenCallback(void* opaque, void** datap, uint64_t* sizep)
{
    std::lock_guard<std::mutex> lock(m_callbackMutex);

    // optional, if comment out libvlc will trigger the 'vlcMediaReadCallback' as more often but for less buffer length
    *sizep = m_mediaDescriptor->getMediaLength();

    return 0;
}

int VLCHelper::vlcMediaReadCallback(void *opaque, unsigned char* buf, size_t len)
{
    std::lock_guard<std::mutex> lock(m_callbackMutex);

    return m_mediaDescriptor->getMediaBytes(buf, len);
}

int VLCHelper::vlcMediaSeekCallback(void *opaque, uint64_t offset)
{
    std::lock_guard<std::mutex> lock(m_callbackMutex);

    // optional, but important for some media types which holds meta data end of the file, for example: .mp4
    m_mediaDescriptor->setSeek(offset);

    return 0;
}

void VLCHelper::vlcMediaCloseCallback(void *opaque)
{
    std::lock_guard<std::mutex> lock(m_callbackMutex);

    m_mediaDescriptor->setSeek(0);

    if (!m_stopRequested)
    {
        emit mediaEOFReached();
        QMetaObject::invokeMethod(&getInstance(), "stopMedia");

        if(m_repeat)
            QMetaObject::invokeMethod(&getInstance(), "playPauseMedia", Q_ARG(bool, true));
    }
}

以及播放器小部件中的用法：

    VLCHelper::getInstance().initMedia(*m_mediaDescriptor, ui.frame_video);
    connect(&VLCHelper::getInstance(), SIGNAL(mediaEOFReached()), this, SLOT(vlcMediaEOFReached()));
    connect(&VLCHelper::getInstance(), SIGNAL(error(QString)), this, SLOT(vlcError(QString)));

...

void PlayerWidget::on_pushButton_media_play_pause_clicked()
{
      VLCHelper::getInstance().playPauseMedia(!VLCHelper::getInstance().isMediaPlaying());
}

void PlayerWidget::on_pushButton_media_stop_clicked()
{
    VLCHelper::getInstance().stopMedia();
}

void PlayerWidget::timer_timeout()
{
    bool isValid = VLCHelper::getInstance().isMediaValid();

    if (isValid)
    {
        if (VLCHelper::getInstance().isMediaPlaying())
        {
            // update media position
            ui.horizontalSlider_media_position->blockSignals(true);
            ui.horizontalSlider_media_position->setValue((int)(VLCHelper::getInstance().getMediaPosition() * 1000.0f));
            ui.horizontalSlider_media_position->blockSignals(false);

            // update media volume
            ui.horizontalSlider_media_volume->blockSignals(true);
            ui.horizontalSlider_media_volume->setValue(VLCHelper::getInstance().getMediaVolume());
            ui.horizontalSlider_media_volume->blockSignals(false);

            // update media time
            ui.label_media_time->setText(VLCHelper::getInstance().getMediaTime().toString());

            // update media total time
            ui.label_media_time_total->setText(VLCHelper::getInstance().getMediaTotalTime().toString());
        }
    }

    ui.horizontalSlider_media_volume->setEnabled(isValid);
    ui.pushButton_media_stop->setEnabled(isValid);
    ui.pushButton_media_repeat->setEnabled(isValid);
}

Answer 3

每當您將流傳遞給 libvlc 時，它都需要執行打開、讀取、查找、關閉流操作。
以下是執行此操作的函數：

using namespace std;

int Open(void* opaque, void** datap, uint64_t* sizep)
{
    ifstream *file = (ifstream *)(opaque); 
    file->seekg(0, file->beg);
    *sizep = file->tellg(); 
    *datap = opaque;
    return 0;
}

SSIZE_T Read(void *opaque, unsigned char* buffer, size_t length)
{
    ifstream *file = (ifstream *)(opaque);
    file->read((char *)(buffer), length);
    return file->gcount();
}

int Seek(void *opaque, uint64_t offset)
{
    ((ifstream *)(opaque))->seekg(offset);    //Seek Stream to offset
    return 0;                                 //Success
}

void Close(void *opaque)
{
    ((ifstream *)(opaque))->close();
}

並調用libvlc_media_new_callbacks如下：

const char *FilePath = "C:\\folder\\video.mp4";
ifstream *FileStream = new std::ifstream(FilePath, std::ios::binary | std::ios::ate);
Libvlc_Media = libvlc_media_new_callbacks(
    Instance,
    Open,
    Read,
    Seek,
    Close,
    FileStream
);

我是在@cube45 的幫助下實現的。 在這里閱讀整個討論。 在登錄和加入 libvlc 服務器之前。 更多信息在這里。