Memory Layout of Arrays for Variables

The term Variable means a physical quantity (pressure, density, etc.) or a geometric quantity (distance, coordinate, etc.) or another quantity.

A variable can have many values which can be gathered in a container which is equivalent to a mathematical vector. Many variables are not gathered in a single container. As examples, mesh coordinates are stored separately in three vectors, and not in a matrix. Also, primitive variables are stored separately in vectors, and not in a matrix.

So a variable may be stored in the container Instant as a numpy array.

Even if the variable vector has only one mathematical dimension, the values of the variable may be related to a specific topology.

For structured grids, an array may have more than one topological dimension, say N. Then, the shape of the array is the tuple of array dimensions. Indeed, a structured grid can be represented as a topologically regular array of points. Each point or element in this N-cube topology can be addressed with N independent indices. As an example, for a two dimensional structured grid, the number of topological dimensions is two, the topology is a square, and there are two indices i and j.

To browse the values of a one-dimensional topological array, there is no ambiguity since there is only one index. But, for a multi-dimensional topological array, the dimensions can be accessed in different ways.

With antares, the point order increases the fastest as one moves from the last to the first dimension. Applied to the above example, the point order increases in j fastest, then i.

Interaction with the VTK library

Some treatments requires the VTK library. The VTK library has its own underlying data structures. Then, antares data arrays must be converted into VTK data structures. This is particularly important for multidimensional arrays where the memory layout may be different, and then copies of arrays may be required. Indeed, with VTK, the point order increases in i fastest, then j.

From VTK < 8.1, it was not possible to avoid copies of arrays for coordinates since they were required as a matrix (Array of Structures) by the library. For VTK > 8.1, it is now possible to avoid copies of arrays for coordinates since they can be passed as vectors (Structure of Arrays) by the library.

The other important point to avoid array copies is the memory layout of the numpy arrays. If the fortran-contiguous memory layout may not induced any copy in the best case, whereas the C-contiguous memory layout will surely imply copies.

Another important point is the datatype of the array values. If the datatype is the single precision floating-point type (float32, 4 memory bytes), then a copy is made to get a double precision floating-point type (float64, 8 memory bytes).

For technical details, you can refer to the following functions:

antares.utils.VtkUtilities.instant_to_vtk(instant, coordinates, dtype='float64', memory_mode=False)

Convert an antares instant to a vtk object.

Parameters:

  • coordinates (list(str)) – the coordinates of the current instant

  • dtype (str) – the data type used to write the file

antares.utils.VtkUtilities.instant_to_vtk_sgrid(instant, coordinates, dtype='float64', memory_mode=False, missing_coordinates=None)

Convert an structured instant to a vtkStructuredGrid object.

An instant has a shape with many dimensions, e.g (nx, ny) in a 2D configuration. It can be viewed as a matrix with nx lines and ny columns. The elements of this matrix are then ordered in a C-order in which the index on the last dimension (ny) changes first. The VTK library requires the opposite: the index on the first dimension (nx) must change first, like a fortran order. That is the reason of the transpose operation performed on variables. Note that the above has nothing to do with the memory layout of the data.

This matrix must be stored in memory. If the memory layout is a fortran contiguous layout, then data can be passed to the VTK library without copy. With nympy, there are at least 3 equivalent ways to do it:

  • arr.T.reshape(-1) # order=’C’ is implicit

  • arr.reshape(-1, order=’F’)

  • arr.ravel(order=’F’)

If the memory layout is a C contiguous layout, then data can not be passed to the VTK library without copy.

If the memory layout is not contiguous at all, then data can not be passed to the VTK library without copy.

Parameters:

  • coordinates (list(str)) – The coordinate names of the current instant.

  • dtype (str) – The data type used to write the file.

  • missing_coordinates (list(str)) – A list with the coordinate name that is missing to create a 3D coordinate system. List of one element.

antares.utils.VtkUtilities.instant_to_vtk_ugrid(instant, coordinates, dtype='float64', memory_mode=False, missing_coordinates=None)

Convert an unstructured instant to a vtkUnstructuredGrid object.

Parameters:

  • coordinates (list(str)) – The coordinate names of the current instant.

  • dtype (str) – The data type used to write the file.

  • missing_coordinates (list(str)) – A list with the coordinate name that is missing to create a 3D coordinate system. List of one element.