Memory handling

In general, processing functions should not allocate their own memory. Instead, they should be able to work with externally-managed memory that could be under the control of an execution engine or another library, and to maintain flexibility, we should make minimal assumptions about how this will be done.

For these reasons, we need to be able to supply our functions with pointers to externally-managed memory: we do this by wrapping the raw pointer in a simple structure, together with some metadata that describes the memory at that location. When passing them to a processing function, this provides the information we need to know how these wrapped pointers should be handled.

The pointer and its metadata are encapsulated in the sdp_Mem structure, which is an opaque type in order to keep all its members private. The functions are all exported using a simple C interface so they can be used from any environment (and further wrapped, if deemed necessary).

Include the header “ska-sdp-func/utility/sdp_mem.h” to use these functions.

Structures

struct sdp_Mem

#include “ska-sdp-func/utility/sdp_mem.h”

Simple structure to wrap a pointer to allocated memory.

This structure is an opaque type to keep implementation details private. It exists as a way to allow processing functions to work with memory that may be allocated and owned by someone else.

Call sdp_mem_create_wrapper() to create a wrapper for memory which has already been allocated, and is therefore assumed to be owned by someone else.

Call sdp_mem_create() to allocate memory which is owned by the wrapper itself.

We wrap a raw pointer since this is a universally-supported way of accessing memory. The pointer is stored together with metadata that describes the memory at that location, namely:

The data type of each element (see sdp_MemType);
The location of the memory (whether in system RAM or on a GPU; see sdp_MemLocation);
The total number of elements allocated;
Whether the wrapper owns the memory to which it points. This determines whether the wrapped memory will be deallocated when calling sdp_mem_free() and the reference count reaches zero.
Whether the memory should be considered read-only.

The following are also stored, for working with tensors:

The number of dimensions.
The size of each dimension.
The stride in each dimension, in bytes (for Python compatibility).

enum sdp_MemType

Enumerator to specify the data type of memory elements.

Values:

enumerator SDP_MEM_CHAR = 1: Char type (1 byte).

enumerator SDP_MEM_INT = 2: Integer type (4 bytes).

enumerator SDP_MEM_FLOAT = 4: Single precision floating point type (4 bytes).

enumerator SDP_MEM_DOUBLE = 8: Double precision floating point type (8 bytes).

enumerator SDP_MEM_COMPLEX = 32: Complex flag.

enumerator SDP_MEM_COMPLEX_FLOAT = SDP_MEM_FLOAT | SDP_MEM_COMPLEX : Single precision complex floating point type (8 bytes).

enumerator SDP_MEM_COMPLEX_DOUBLE = SDP_MEM_DOUBLE | SDP_MEM_COMPLEX : Double precision complex floating point type (16 bytes).

enum sdp_MemLocation

Enumerator to specify the location of allocated memory.

Values:

enumerator SDP_MEM_CPU: Memory is on the host.

enumerator SDP_MEM_GPU: Memory is on the GPU.

Functions

sdp_Mem *sdp_mem_create(sdp_MemType type, sdp_MemLocation location, int32_t num_dims, const int64_t *shape, sdp_Error *status)

Allocate a multi-dimensional block of memory.

The shape of the memory block is given by the shape parameter, which is an array of length num_dims elements.

Parameters:

type – Enumerated element data type of memory to allocate.
location – Enumerated memory location.
num_dims – Number of dimensions.
shape – Size of each dimenion, in elements.
status – Error status.

Returns:

sdp_Mem* Handle to allocated memory.

sdp_Mem *sdp_mem_create_wrapper(void *data, sdp_MemType type, sdp_MemLocation location, int32_t num_dims, const int64_t *shape, const int64_t *stride, sdp_Error *status)

Wraps a pointer to a multi-dimensional array which is owned elsewhere.

Since it is owned by someone else, the memory will not be deallocated when the handle is freed.

The shape of the memory block is given by the shape parameter, which is an array of length num_dims elements. The stride (in bytes) for each dimension can also be optionally specified using the stride parameter - if NULL, then this will be computed from the shape.

Parameters:

data – Raw pointer to wrap.
type – Enumerated element data type of memory at data.
location – Enumerated memory location of memory at data.
num_dims – Number of dimensions.
shape – Size of each dimenion, in elements.
stride – Stride of each dimension, in bytes. May be NULL.
status – Error status.

Returns:

sdp_Mem* Handle to wrapped memory.

sdp_Mem *sdp_mem_create_alias(const sdp_Mem *src)

Create a shallow copy, or an alias, of a block’s metadata.

Parameters:: src – Handle to source memory block.
Returns:: sdp_Mem* Handle to aliased memory.

sdp_Mem *sdp_mem_create_copy(const sdp_Mem *src, sdp_MemLocation location, sdp_Error *status)

Create a deep copy of a memory block in the specified location.

Parameters:

src – Handle to source memory block.
location – Enumerated memory location for destination.
status – Error status.

Returns:

sdp_Mem* Handle to copied memory.

void sdp_mem_clear_contents(sdp_Mem *mem, sdp_Error *status)

Clears contents of a memory block by setting all its elements to zero.

Parameters:

mem – Handle to memory to clear.
status – Error status.

void sdp_mem_copy_contents(sdp_Mem *dst, const sdp_Mem *src, int64_t offset_dst, int64_t offset_src, int64_t num_elements, sdp_Error *status)

Copies memory contents from one block to another.

Parameters:

dst – Handle to destination memory block.
src – Handle to source memory block.
offset_dst – Start offset (number of elements) into destination block.
offset_src – Start offset (number of elements) from source block.
num_elements – Number of elements to copy.
status – Error status.

void sdp_mem_copy_contents_async(sdp_Mem *dst, const sdp_Mem *src, int64_t offset_dst, int64_t offset_src, int64_t num_elements, sdp_CudaStream *stream, sdp_Error *status)

Copies memory contents from one block to another.

This function uses cudaMemcpyAsync for transfers that involve GPU memory.

Parameters:

dst – Handle to destination memory block.
src – Handle to source memory block.
offset_dst – Start offset (number of elements) into destination block.
offset_src – Start offset (number of elements) from source block.
num_elements – Number of elements to copy.
stream – Handle to CUDA stream to use.
status – Error status.

void *sdp_mem_data(sdp_Mem *mem)

Returns a raw pointer to the memory wrapped by the handle.

Parameters:: mem – Handle to memory block.
Returns:: void* Raw pointer to allocated memory.

const void *sdp_mem_data_const(const sdp_Mem *mem)

Returns a raw const pointer to the memory wrapped by the handle.

Parameters:: mem – Handle to memory block.
Returns:: void* Raw const pointer to allocated memory.

void *sdp_mem_gpu_buffer(sdp_Mem *mem, sdp_Error *status)

Returns a pointer to the GPU buffer, if memory is on the GPU.

This is needed when launching a GPU kernel with sdp_launch_cuda_kernel().

The status flag is set and a null pointer is returned if the wrapped memory is not on the GPU.

Parameters:

mem – Handle to memory block.
status – Error status.

Returns:

void* Pointer used for GPU buffer when launching a kernel.

const void *sdp_mem_gpu_buffer_const(const sdp_Mem *mem, sdp_Error *status)

Returns a pointer to the GPU buffer, if memory is on the GPU.

This is needed when launching a GPU kernel with sdp_launch_cuda_kernel().

The status flag is set and a null pointer is returned if the wrapped memory is not on the GPU.

Parameters:

mem – Handle to memory block.
status – Error status.

Returns:

void* Const pointer used for GPU buffer when launching a kernel.

void sdp_mem_free(sdp_Mem *mem)

Decrements the reference counter.

If the reference counter reaches zero, any memory owned by the handle will be released, and the handle destroyed.

This is an alias for sdp_mem_ref_dec().

Parameters:: mem – Handle to memory block.

int32_t sdp_mem_is_c_contiguous(const sdp_Mem *mem)

Returns true if the dimension strides are C contiguous.

Parameters:: mem – Handle to memory block.
Returns:: True if dimension strides are C contiguous.

int32_t sdp_mem_is_floating_point(const sdp_Mem *mem)

Returns true if data element are in floating point format.

Parameters:: mem – Handle to memory block.
Returns:: True if data elements are in floating point.

int32_t sdp_mem_is_complex(const sdp_Mem *mem)

Returns true if data elements are of complex type.

Parameters:: mem – Handle to memory block.
Returns:: True if data type is complex.

int32_t sdp_mem_is_complex4(const sdp_Mem *mem)

Returns true if data are complex 4-vectors, or 2-by-2 matrices.

Parameters:: mem – Handle to memory block.
Returns:: True if data type is complex-4.

int32_t sdp_mem_is_double(const sdp_Mem *mem)

Returns true if data elements are double precision.

Parameters:: mem – Handle to memory block.
Returns:: True if data elements are double precision.

int32_t sdp_mem_is_matching(const sdp_Mem *mem1, const sdp_Mem *mem2, int32_t check_location)

Returns true if the metadata between two objects is consistent.

Checks are done on the data type, number of dimensions, strides, and dimension sizes.

Parameters:

mem1 – Handle to first memory block.
mem2 – Handle to second memory block.
check_location – If true, check the data locations also match.

Returns:

True if memory blocks have matching meta-data.

int32_t sdp_mem_is_read_only(const sdp_Mem *mem)

Returns true if the read-only flag is set.

Parameters:: mem – Handle to memory block.
Returns:: True if memory should be considered to be read-only.

sdp_MemLocation sdp_mem_location(const sdp_Mem *mem)

Returns the enumerated location of the memory.

Parameters:: mem – Handle to memory block.
Returns:: sdp_MemLocation The enumerated memory location.

int32_t sdp_mem_num_dims(const sdp_Mem *mem)

Returns the number of dimensions in the memory block.

Parameters:: mem – Handle to memory block.
Returns:: int32_t The number of dimensions in the block.

int64_t sdp_mem_num_elements(const sdp_Mem *mem)

Returns the total number of elements in the memory block.

Parameters:: mem – Handle to memory block.
Returns:: int64_t The total number of data elements in the block.

void sdp_mem_random_fill(sdp_Mem *mem, sdp_Error *status)

Fills memory with random values between 0 and 1. Useful for testing.

Parameters:

mem – Handle to memory block.
status – Error status.

void sdp_mem_ref_dec(sdp_Mem *mem)

Decrement the reference counter.

If the reference counter reaches zero, any memory owned by the handle will be released, and the handle destroyed.

This is an alias for sdp_mem_free().

Parameters:: mem – Handle to memory block.

sdp_Mem *sdp_mem_ref_inc(sdp_Mem *mem)

Increment the reference counter.

Call if ownership of the handle needs to be transferred without incurring the cost of an actual copy.

Parameters:: mem – Handle to memory block.
Returns:: sdp_Mem* Handle to memory block (same as mem).

void sdp_mem_set_read_only(sdp_Mem *mem, int32_t value)

Set the flag specifying whether the memory should be read-only.

Parameters:

mem – Handle to memory block.
value – True or false value of the read-only flag to set.

int64_t sdp_mem_shape_dim(const sdp_Mem *mem, int32_t dim)

Returns the number of elements in the specified dimension.

The slowest varying dimension is the first index (0), and the fastest varying dimension is the last index (num_dims - 1).

Parameters:

mem – Handle to memory block.
dim – The dimension index to return.

Returns:

int64_t Number of elements in the specified dimension.

int64_t sdp_mem_stride_bytes_dim(const sdp_Mem *mem, int32_t dim)

Returns the stride (in bytes) of the specified dimension.

The slowest varying dimension is the first index (0), and the fastest varying dimension is the last index (num_dims - 1).

This is in bytes for compatibility with Python.

Parameters:

mem – Handle to memory block.
dim – The dimension index to return.

Returns:

int64_t Stride in bytes for the specified dimension.

int64_t sdp_mem_stride_elements_dim(const sdp_Mem *mem, int32_t dim)

Returns the stride (in elements) of the specified dimension.

The slowest varying dimension is the first index (0), and the fastest varying dimension is the last index (num_dims - 1).

Parameters:

mem – Handle to memory block.
dim – The dimension index to return.

Returns:

int64_t Stride in elements for the specified dimension.

sdp_MemType sdp_mem_type(const sdp_Mem *mem)

Returns the enumerated data type of the memory.

Parameters:: mem – Handle to memory block.
Returns:: sdp_MemType The enumerated element data type.

int64_t sdp_mem_type_size(sdp_MemType type)

Returns the size of one element of a data type, in bytes.

Parameters:: type – Enumerated data type.
Returns:: int64_t Size of data type in bytes.

const char *sdp_mem_location_name(sdp_MemLocation location)

Returns a string representation of memory location.

Parameters:: location – Enumerated memory location.
Returns:: String representation.

const char *sdp_mem_type_name(sdp_MemType type)

Returns a string representation of data type.

Parameters:: type – Enumerated memory data type.
Returns:: String representation.

void sdp_mem_check_writeable_at(const sdp_Mem *mem, sdp_Error *status, const char *expr, const char *func, const char *file, int line)

Checks that an array is writeable.

Use the sdp_mem_check_writeable macro to automatically fill expr, func, file and line by call location. status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
status – Output error status.
expr – Expression string to report in error message.
func – Function to report in error message.
file – File name to report in error message.
line – Line to report in error message.

void sdp_mem_check_c_contiguity_at(const sdp_Mem *mem, sdp_Error *status, const char *expr, const char *func, const char *file, int line)

Checks that an array is C contiguous.

Use the sdp_mem_check_c_contiguity macro to automatically fill expr, func, file and line by call location. status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
status – Output error status.
expr – Expression string to report in error message.
func – Function to report in error message.
file – File name to report in error message.
line – Line to report in error message.

void sdp_mem_check_location_at(const sdp_Mem *mem, sdp_MemLocation expected_location, sdp_Error *status, const char *expr, const char *func, const char *file, int line)

Checks that an array resides in the expected memory space.

Use the sdp_mem_check_location macro to automatically fill expr, func, file and line by call location. status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
expected_location – Expected memory location.
status – Output error status.
expr – Expression string to report in error message.
func – Function to report in error message.
file – File name to report in error message.
line – Line to report in error message.

void sdp_mem_check_num_dims_at(const sdp_Mem *mem, int64_t expected_num_dims, sdp_Error *status, const char *expr, const char *func, const char *file, int line)

Checks that an array has expected number of dimensions.

Use the sdp_mem_check_num_dims macro to automatically fill expr, func, file and line by call location. status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
expected_num_dims – Expected number of dimensions.
status – Output error status.
expr – Expression string to report in error message.
func – Function to report in error message.
file – File name to report in error message.
line – Line to report in error message.

void sdp_mem_check_dim_size_at(const sdp_Mem *mem, int32_t dim, int64_t size, sdp_Error *status, const char *expr, const char *func, const char *file, int line)

Checks that a given dimension of an array has the expected size.

Use the sdp_mem_check_dim_size macro to automatically fill expr, func, file and line by call location. status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
dim – Dimension index to check.
size – Expected size of specified dimension.
status – Output error status.
expr – Expression string to report in error message.
func – Function to report in error message.
file – File name to report in error message.
line – Line to report in error message.

void sdp_mem_check_shape_at(const sdp_Mem *mem, int32_t expected_num_dims, const int64_t *expected_shape, sdp_Error *status, const char *expr, const char *func, const char *file, int line)

Checks that an array has the expected shape.

Use the sdp_mem_check_shape macro to automatically fill expr, func, file and line by call location. status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
expected_num_dims – Expected number of dimensions.
expected_shape – Expected size of each dimension.
status – Output error status.
expr – Expression string to report in error message.
func – Function to report in error message.
file – File name to report in error message.
line – Line to report in error message.

void sdp_mem_check_shape_dim_at(const sdp_Mem *mem, int32_t dim, const int64_t expected_shape, sdp_Error *status, const char *expr, const char *func, const char *file, int line)

Checks that an array has the expected shape for a certain dimension.

Use the sdp_mem_check_shape_dim macro to automatically fill expr, func, file and line by call location. status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
expected_num_dims – Expected number of dimensions.
expected_shape – Expected size of each dimension.
status – Output error status.
expr – Expression string to report in error message.
func – Function to report in error message.
file – File name to report in error message.
line – Line to report in error message.

void sdp_mem_check_same_shape_at(sdp_Mem *mem, int32_t dim, sdp_Mem *mem2, int32_t dim2, sdp_Error *status, const char *func, const char *expr, const char *expr2, const char *file, int line)

Checks that dimensions of two given memory objects have the same size.

status will be set if the check fails. Note that this function will not fail if the dimension in question does not exist on either memory object, use sdp_mem_check_num_dims additionally. Use sdp_mem_check_same_shape macro to automatically fill func, expr, file and line by call location.

Parameters:

mem – Handle to memory block to check
dim – Dimension to check
mem2 – Handle to memory block to check against
dim2 – Dimension to check against
status – Output error status
func – Function to report in error message
expr – Expression string to report in error message
expr2 – Second expression string to report in error message
file – File name to report in error message
line – Line to report in error message

void sdp_mem_check_type_at(const sdp_Mem *mem, sdp_MemType expected_type, sdp_Error *status, const char *expr, const char *func, const char *file, int line)

Checks that an array has the expected data type.

Use the sdp_mem_check_type macro to automatically fill expr, func, file and line by call location. status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
expected_type – Expected data type.
status – Output error status.
expr – Expression string to report in error message.
func – Function to report in error message.
file – File name to report in error message.
line – Line to report in error message.

sdp_mem_check_writeable(mem, status)

Checks that an array is writeable.

status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
status – Output error status.

sdp_mem_check_c_contiguity(mem, status)

Checks that an array is C contiguous.

status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
status – Output error status.

sdp_mem_check_location(mem, expected_location, status)

Checks that an array resides in the expected memory space.

status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
expected_location – Expected memory location.
status – Output error status.

sdp_mem_check_num_dims(mem, expected_num_dims, status)

Checks that an array has expected number of dimensions.

status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
expected_num_dims – Expected number of dimensions.
status – Output error status.

sdp_mem_check_dim_size(mem, dim, size, status)

Checks that a given dimension of an array has the expected size.

status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
dim – Dimension index to check.
size – Expected size of specified dimension.
status – Output error status.

sdp_mem_check_shape(mem, expected_num_dims, expected_shape, status)

Checks that an array has the expected shape.

status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
expected_num_dims – Expected number of dimensions.
expected_shape – Expected size of each dimension.
status – Output error status.

sdp_mem_check_shape_dim(mem, expected_num_dims, expected_shape, status)

Checks that an array has the expected shape for a certain dimension.

status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
expected_num_dims – Expected number of dimensions.
expected_shape – Expected size of each dimension.
status – Output error status.

sdp_mem_check_same_shape(mem, dim, mem2, dim2, status)

Checks that a dimension of given memory has expected size.

status will be set if the check fails. Note that this function will not fail if the dimension in question does not exist, use sdp_mem_check_num_dims additionally.

Parameters:

mem – Handle to memory block to check
dim – Dimension to check
mem2 – Handle to memory block to check against
dim2 – Dimension to check against
status – Output error status

sdp_mem_check_type(mem, expected_type, status)

Checks that an array has the expected data type.

status will be set if the check fails.

Parameters:

mem – Handle to memory block to check.
expected_type – Expected data type.
status – Output error status.

Views (C++ only)

template<typename num_t, int32_t num_dims> using sdp_MemViewCpu = sdp_MemView<num_t, num_dims, SDP_MEM_CPU>

template<typename num_t, int32_t num_dims> using sdp_MemViewGpu = sdp_MemView<num_t, num_dims, SDP_MEM_GPU>

template<typename num_t, int32_t num_dims, sdp_MemLocation loc> struct sdp_MemView

#include “ska-sdp-func/utility/sdp_mem_view.h”

Utility class for accessing simple strided arrays.

Use sdp_mem_check_and_view to safely obtain views from sdp_Mem object, then sdp_MemView::operator()() to address memory contents:

sdp_MemViewCpu<double, 1> view; sdp_Error status;
sdp_mem_check_and_view(mem, &view, &status);
if (*status) return;

double sum = 0;
for (int i = 0; i < view.shape[0]; i++) {
    sum += view(i);
}

Note that sdp_MemView::operator()() will perform bounds checks unless NDEBUG is defined.

Public Functions

inline sdp_MemView(): Default constructor

inline num_t &operator()(): Operator for accessing data contained in 0-dimensional array views.

inline num_t &operator()(int64_t i0): Operator for accessing data contained in 1-dimensional array views.

inline num_t &operator()(int64_t i0, int64_t i1): Operator for accessing data contained in 2-dimensional array views.

inline num_t &operator()(int64_t i0, int64_t i1, int64_t i2): Operator for accessing data contained in 3-dimensional array views.

inline num_t &operator()(int64_t i0, int64_t i1, int64_t i2, int64_t i3): Operator for accessing data contained in 4-dimensional array views.

inline num_t &operator()(int64_t i0, int64_t i1, int64_t i2, int64_t i3, int64_t i4): Operator for accessing data contained in 5-dimensional array views.

Public Members

num_t *ptr: Pointer to first array element

int64_t shape[num_dims]: Size of array in every dimension

int64_t stride[num_dims]: Memory stride in dimension (in units of element type size)

template<typename num_t, int32_t num_dims, sdp_MemLocation loc> void sdp_mem_check_and_view_at(sdp_Mem *mem, sdp_MemView<num_t, num_dims, loc> *view, sdp_Error *status, const char *func, const char *expr, const char *file, int line)

Attempts to generate view of memory object.

This checks that the memory:

has the expected type
has the expected number of dimension
is located in the right memory space
is writeable

status will be set if a check fails. Use sdp_mem_check_and_view macro to automatically fill func, expr, file and line by call location.

Parameters:

mem – Handle to memory block to view
view – Handle to memory block to view
status – Output error status.
func – Function to report in error message
expr – Expression string to report in error message
file – File name to report in error message
line – Line to report in error message

template<typename num_t, int32_t num_dims, sdp_MemLocation loc> void sdp_mem_check_and_view_at(sdp_Mem *mem, sdp_MemView<const num_t, num_dims, loc> *view, sdp_Error *status, const char *func, const char *expr, const char *file, int line)

Attempts to generate constant view of memory object.

This checks that the memory:

has the expected type
has the expected number of dimension
is located in the right memory space

status will be set if a check fails. Use sdp_mem_check_and_view macro to automatically fill func, expr, file and line by call location.

Parameters:

mem – Handle to memory block to view
view – Handle to memory block to view
status – Output error status.
func – Function to report in error message
expr – Expression string to report in error message
file – File name to report in error message
line – Line to report in error message

sdp_mem_check_and_view(mem, view, status)

Attempts to generate view of memory object.

This checks that the memory:

has the expected type
has the expected number of dimension
is located in the right memory space
is writeable (if non-const view type)

status will be set if a check fails.

Parameters:

mem – Handle to memory block to view
view – Handle to memory block to view
status – Output error status.