CUDA推力库中的功能是否隐式同步？

Question

I met some problems When using functions in thrust library, I am not sure if I should add cudaDeviceSynchronize manually before it. For example,

double dt = 0;
kernel_1<<<blocks, threads>>>(it);
dt = *(thrust::max_element(it, it + 10));
printf("%f\n", dt);

Since kernel_1 is non-blocking, host will execute the next statement. The problem is I am not sure if the thrust::max_element is blocking. If it is blocking, then it works well; otherwise, will host skip it and execute the "printf" statement?

Thanks

Answer 1

Your code is broken in at least 2 ways.

1. it is presumably a device pointer:

    kernel_1<<<blocks, threads>>>(it); ^^ 

   it is not allowed to use a raw device pointer as an argument to a thrust algorithm:

    dt = *(thrust::max_element(it, it + 10)); ^^ 

   unless you wrap that pointer in a thrust::device_ptr or else use the thrust::device execution policy explicitly as an argument to the algorithm. Without any of these clues, thrust will dispatch the host code path (which will probably seg fault) as discussed in the thrust quick start guide .

2. If you fixed the above item using either thrust::device_ptr or thrust::device , then thrust::max_element will return an iterator of a type consistent with the iterators passed to it. If you pass a thrust::device_ptr it will return a thrust::device_ptr . If you use thrust::device with your raw pointer, it will return a raw pointer. In either case, it is illegal to dereference such in host code:

    dt = *(thrust::max_element(it, it + 10)); ^ 

   again, I would expect such usage to seg fault.

Regarding asynchrony, it is safe to assume that all thrust algorithms that return a scalar quantity stored in stack variable are synchronous. That means the CPU thread will not proceed beyond the thrust call until the stack variable has been populated with the correct value

Regarding GPU activity in general, unless you use streams, all GPU activity is issued to the same (default) stream. This means that all CUDA activity will be executed in-order, and a given CUDA operation will not begin until the preceding CUDA activity is complete. Therefore, even though your kernel launch is asynchronous, and the CPU thread will proceed onto the thrust::max_element call, any CUDA activity spawned from that call will not begin executing until the previous kernel launch is complete. Therefore, any changes made to the data referenced by it by kernel_1 should be finished and completely valid before any CUDA processing in thrust::max_element begins. And as we've seen, thrust::max_element itself will insert synchronization.

So once you fix the defects in your code, there should not be any requirement to insert additional synchronization anywhere.

Answer 2

This function does not seem to be async.

Both of these pages explain the behaviour of max_element() and they do not explicit it as async, so I would assume it is blocking :

• thrust : Extrema
• max_element

Since it is using an iterator to treat all the elements and find the maximum of the values, I can not think about it to be async.

You can still use cudaDeviceSynchronize to try it for real, but do not forget to set the corresponding flag on your device.