CUDA kernel printf() 在终端中不产生 output，在分析器中工作

Question

Consider the following program:

#include <cuda/api_wrappers.hpp>

namespace kernels {
template <typename T>
__global__ void print_stuff()
{
        printf("This is a plain printf() call.\n");
}
} // namespace kernels

int main()
{
        auto launch_config { cuda::make_launch_config(2,2) };
        cuda::launch(::kernels::print_stuff<int>, launch_config);
        cuda::outstanding_error::ensure_none();
}

(it uses the cuda-api-wrappers library).

The program compiles and runs. However, if I run in in a terminal, it prints nothing; while if I run it via nvvp, the console shows me:

This is a plain printf() call.
This is a plain printf() call.
This is a plain printf() call.
This is a plain printf() call.

... as expected (2 blocks x 2 threads = 4 lines).

What is/could be the reason am I not getting the four lines printed on the terminal as well?

Notes:

• I realize the fault may theoretically be with the library, of which I am the author. So "it has to be the library" is a legitimate answer, but you need to explain why it can't be anything else.
• No warnings when compiling with nvcc -Xcompiler -Wall -Xcompiler -Wextra .
• I use Devuan GNU/Linux 3 (beowulf; equivalent of Debian Buster).
• My hardware: An AMD64 Intel CPU; a GTX 1050 Ti card.
• nVIDIA Driver version: 430.50; CUDA version: 10.1.105.
• cuda-memcheck does not complain about the program.

Answer 1

You are implicitly, and mistakenly, assuming a certain order of occurrences when main() is done. Specifically, you're assuming that because the default stream is synchronous, everything having to do with your kernel is over and done with by the time the next line of code after the kernel launch gets executed. That is not 100% true - as @RobertCrovella suggests; specifically, it's not guaranteed that the device's printf() buffer will be ferried back into host memory and dumped into the standard output stream before control returns to your program.

You will need to synchronize the (default, current) CUDA device with the host, ie execute:

cuda::device::current::get().synchronize();

or at least synchronize the device's default stream:

cuda::device::current::get().default_stream().synchronize();

and this will ensure the printf() results make it to standard output.

Now, nvvp instruments your execution in some way (probably just by having the profiler running - but nvprof instruments the execution through the hooks which are the CUDA runtime API calls). So, the behavior is different when you run your program that way.

---

Somewhat-related question: The behavior of stream 0 (default) and other streams .