Linux PCIe DMA driver

Question

I'm currently writing a driver for a PCIe device that should send data to a Linux system using DMA. As far as I can understand my PCIe device needs a DMA controller (DMA master) and my Linux system too (DMA slave). Currently the PCIe device has no DMA controller and should not get one. That confuses me.

A. Is the following possible?

1. PCIe device sends interrupt
2. Wait for interrupt in the Linux driver
3. Start DMA transfer from memory mapped PCIe registers to Linux system DMA.
4. Read the data from memory in userspace

I have everything setup for this, the only thing I miss is how to transfer the data from the PCIe registers to the memory.

B. Which system call (or series of) do I need to call to do a DMA transfer?

C. I probably need to setup the DMA on the Linux system but what I find points to code that assumes there is a slave, eg struct dma_slave_config.

The use case is collecting data from the PCIe device and make it available in memory to userspace.

Any help is much appreciated. Thanks in advance!

Answer 1

DMA, by definition, is completely independent of the CPU and any software (ie OS kernel) running on it. DMA is a way for devices to perform memory reads and writes against host memory without the involvement of the host CPU.

The way DMA usually works is something like this: software will allocate a DMA accessible region in memory and share the physical address with the device, say, by performing memory writes against the address space associated with one of the device's BARs. Then, the device will perform a DMA read or write against that block of memory. When that operation is complete, the device will issue an interrupt to the device driver so it can handle the data and/or free the memory.

If your device does not have the capability of issuing a DMA read or write against host memory, then you'll have to interact with it with using the CPU only. Discrete DMA controllers have not been a thing for a very long time.