Memory APIs¶
device_memory_allocate 【required】¶
Definition¶
C_Status (*device_memory_allocate)(const C_Device device, void** ptr, size_t size)
Description¶
It allocates the device memory.
Parameter¶
device - the device to be used
ptr - the address of the allocated device memory
size - the size of the device memory needed to be allocated (in byte)
device_memory_deallocate 【required】¶
Definition¶
C_Status (*device_memory_deallocate)(const C_Device device, void* ptr, size_t size)
Description¶
It deallocates the device storage.
Parameter¶
device - the device to be used
ptr - the address of the device memory needed to be deallocated
size - the size of the device memory needed to be deallocated (in byte)
host_memory_allocate 【optional】¶
Definition¶
C_Status (*host_memory_allocate)(const C_Device device, void** ptr, size_t size)
Description¶
It allocates pinned host memory.
Parameter¶
device - the device to be used
ptr - the address of allocated host memory
size - the size of memory needed to be allocated (in byte)
host_memory_deallocate 【optional】¶
Definition¶
C_Status (*host_memory_deallocate)(const C_Device device, void* ptr, size_t size)
Description¶
It deallocates the pinned host memory.
Parameter¶
device - the device to be used
ptr - the address of host memory needed to be deallocated
size - the size of memory needed to be deallocated (in byte)
unified_memory_allocate 【optional】¶
Definition¶
C_Status (*unified_memory_allocate)(const C_Device device, void** ptr, size_t size)
Description¶
It allocates unified memory.
Parameter¶
device - the device to be used
ptr - unified memory address
size - the size of memory needed to be allocated (in byte)
unified_memory_deallocate 【optional】¶
Definition¶
C_Status (*unified_memory_deallocate)(const C_Device device, void** ptr, size_t size)
Description¶
It deallocates unified memory.
Parameter¶
device - the device to be used
ptr - the address of unified memory needed to be deallocated
size - the size of memory needed to be deallocated (in byte)
memory_copy_h2d 【optional】¶
Definition¶
C_Status (*memory_copy_h2d)(const C_Device device, void* dst, const void* src, size_t size)
Description¶
It copies synchronous memory from the host to the device.
Parameter¶
device - the device to be used
dst - the address of destination device memory
src - the address of the source host memory
size - the size of memory needed to be copied (in byte)
memory_copy_d2h 【optional】¶
Definition¶
C_Status (*memory_copy_d2h)(const C_Device device, void* dst, const void* src, size_t size)
Description¶
It copies synchronous memory from the device to the host.
Parameter¶
device - the device to be used
dst - the address of the destination host memory
src - the address of the source device memory
size - the size of memory needed to be copied (in byte)
memory_copy_d2d 【optional】¶
Definition¶
C_Status (*memory_copy_d2d)(const C_Device device, void* dst, const void* src, size_t size)
Description¶
It copies synchronous memory in the device.
Parameter¶
device - the device to be used
dst - the address of the destination device memroy
src - the address of the source device memory
size - the size of memory needed to be copied (in byte)
memory_copy_p2p 【optional】¶
Definition¶
C_Status (*memory_copy_p2p)(const C_Device dst_device, const C_Device src_device, void* dst, const void* src, size_t size)
Description¶
It copies synchronous memory between devices.
Parameter¶
dst_device - the destination device
src_device - the source device
dst - the address of destination device memory
src - the address of source device memory
size - the size of memory needed to be copied (in byte)
async_memory_copy_h2d 【optional】¶
Definition¶
C_Status (*async_memory_copy_h2d)(const C_Device device, C_Stream stream, void* dst, const void* src, size_t size)
Description¶
It copies asynchronous memory from the host to the device. If it is not implemented, PaddlePaddle will be replace it with a synchronous API.
Parameter¶
device - the device to be used
stream - it is executed on that stream.
dst - the address of destination device memory
src - the address of source host memory
size - the size of memory neeeded to be copied (in byte)
async_memory_copy_d2h 【optional】¶
Definition¶
C_Status (*async_memory_copy_d2h)(const C_Device device, C_Stream stream, void* dst, const void* src, size_t size)
Description¶
It copies asynchronous memory from device to host. If it is not implemented, PaddlePaddle will replace it with a synchronous API.
Parameter¶
device - the device to be used
stream - It is executed on the stream.
dst - the address of destination host
src - the address of source device
size - the size of memory needed to be copied
async_memory_copy_d2d 【optional】¶
Definition¶
C_Status (*async_memory_copy_d2d)(const C_Device device, C_Stream stream, void* dst, const void* src, size_t size)
Description¶
It copies asynchronous memory in the device. If it is not implemented, PaddlePaddle will replace it with a synchronous API.
Parameter¶
device - the device to be used
stream - the stream to be used
dst - the address of destination device memory
src - the address of source device memory
size - the size of memory needed to be copied (in byte)
async_memory_copy_p2p 【optional】¶
Definition¶
C_Status (*async_memory_copy_p2p)(const C_Device dst_device, const C_Device src_device, C_Stream stream, void* dst, const void* src, size_t size)
Description¶
It copies asynchronous memory between devices. If it is not implemented, PaddlePaddle will replace it with a synchronous API.
Parameter¶
dst_device - the destination device
src_device - the source device
stream - the stream to be used
dst - the address of destination device memory
src - the address of source device memory
size - the size of memory needed to be copied (in byte)
device_memory_set 【optional】¶
Definition¶
C_Status (*device_memory_set)(const C_Device device, void* ptr, unsigned char value, size_t size)
Description¶
It uses the value to pad the memory of a device. If it is not implemented, PaddlePaddle will take its place with memory_copy_h2d.
Parameter¶
device - the device to be used
ptr - the address of the padding
value - padded value
size - padding size (in byte)
device_memory_stats 【optional】¶
Definition¶
C_Status (*device_memory_stats)(const C_Device device, size_t* total_memory, size_t* free_memory)
Description¶
It counts the memory using condition.
Parameter¶
device - the device to be used
total_memory - total memory (in byte)
free_memory - free memory (in byte)
device_min_chunk_size 【optional】¶
Definition¶
C_Status (*device_min_chunk_size)(C_Device device, size_t* size)
Description¶
It checks the minimum size of device memory chunks (in byte). In order not to call the device API to frequently apply for/ deallocate memory, PaddlePaddle manages the device memory. When there is an application, memory will be first allocated from the managed memory. When there is an application for memory whose size is “size”, the size of the allocated memory is size + extra_padding_size and it will be aligned with min_chunk_size, the minimum size of memory chunks.
device_max_chunk_size 【optional】¶
Definition¶
C_Status (*device_max_chunk_size)(C_Device device, size_t* size)
Description¶
The size of the memory allocated from that managed by PaddlePaddle is no more than the maximum size of device memory chunks (in byte). Otherwise, the device API will be invoked for allocation. If this API is not implemented, the size of the memory is device_max_alloc_size, the maximum size of allocatable device memory.
device_max_alloc_size 【optional】¶
Definition¶
C_Status (*device_max_alloc_size)(C_Device device, size_t* size)
Description¶
It checks the maximum size (in byte) of allocatable device memory. If it is not implemented, the memory size will be equal to that of the current available memory.
device_extra_padding_size 【optional】¶
Definition¶
C_Status (*device_extra_padding_size)(C_Device device, size_t* size)
Description¶
It allocates the extra padding size of device memory. If it is not implemented, the size will be set to 0 by default. In order not to call the device API to frequently apply for/ deallocate memory, PaddlePaddle manages the device memory. When there is an application, memory will be first allocated from the managed memory. When there is an application for memory whose size is “size”, the size of the allocated memory is size + extra_padding_size and it will be aligned with min_chunk_size, the minimum size of memory chunks.
device_init_alloc_size 【optional】¶
Definition¶
C_Status (*device_init_alloc_size)(const C_Device device, size_t* size)
Description¶
It checks the size of allocated device memory (in byte) after initialization. If it is not implemented, the size will be equal to device_max_alloc_size, the maximum size of allocatable device memory.
device_realloc_size 【optional】¶
Definition¶
C_Status (*device_realloc_size)(const C_Device device, size_t* size)
Description¶
It checks the size of reallocated device memory (in byte). If it is not implemented, the memory size will be equal to device_max_alloc_size, the maximum size of allocatable device memory.