CUDA study 2-5. Data Transfer Overhead

Data Transfer Overhead

The data copying process introduced in CUDA programming between the PC and the graphics card is an additional overhead compared to the traditional CPU processing. In fact, fetching data from memory is a slow operation in PC computations. Transferring data from the DRAM of the PC to the graphics card’s memory is dependent on the data transfer specifications and speed of the PC, determining the time required.

Therefore, programs with a low number of computations may be faster when utilizing the CPU.

Ideal Conditions for Applying CUDA

The performance of the processing unit and the bandwidth between the memory processor become crucial as the need for extensive computations on large datasets increases. In conclusion, if the processing time using the CPU, when combined with the data transfer time, is greater than the processing time using the GPU, then using CUDA is advantageous.

Following this, we will study how to reduce data transfer overhead by using streams.