Next: 2 Coupling scalar values Up: 5 Additional notes on Previous: 5 Additional notes on
As described elsewhere, OASIS3-MCT leverages the MCT 2.11 coupling infrastructure developed at Argonne National Laboratory. That infrastructure is designed to couple fields on static grids between model components. The fields can be decomposed using MPI across mutiple processes, and the decomposition and number of proceses involved can be arbitrary in each component. MCT supports both communication of data between unique MPI non-overlapping communicators and within a single MPI communicator, although these two operations are functionally independent within MCT.
MCT also provides the ability to map (i.e. interpolate or regrid) data between grids as long as the interpolation is linear and can be computed by a linear sparse matrix multiply. MCT does not compute interpolation weights, but it has interfaces that allow those weights to be passed into MCT. Within MCT, mapping and communication are also treated as independent features.
OASIS3-MCT supports both mapping and communication of data through MCT. As a results, mapping and coupling are implemented, in many ways, as separate steps in the underlying implementation. A coupling field that also requires mapping will carry out the mapping on either the source or destination component on the associated processes, while coupling between components will be done either before or after mapping. In other words, a coupling field can be interpolated to the destination grid on the source processes then communicated to the destination component OR a coupling field can be communicated to the destination component on the source grid then mapped to the destination grid on the destination component. At the present time, it is not possible to map the field as part of the communication rearrangement, although in theory, that capability should be possible to implement in the future, and OASIS3-MCT developers are considering it. The separation of mapping and communication is handled by the OASIS layer. Users only need to be aware of a few options that can be set to fine tune the performance of these operations.
OASIS3-MCT also imposes a few other constraints on the usage. MCT does NOT support haloed communication. There must be a 1-to-1 relationship between grid point values on the source grid and on the destination grid. A user cannot send a single grid point value to multiple destination gridcells or processes via OASIS3-MCT. The partitions defined by the user that define the field decomposition cannot reference the same global gridcell more than once.
OASIS3-MCT does not support dynamically varying grids nor dynamically varying decompositions at the present time. MCT is NOT currently setup to support those features. In the future, it is possible that some dynamic grid capabilities might be supported through OASIS3-MCT but requirements and implementation are still being assessed.
Since the OASIS3-MCT_4.0 release, OASIS3-MCT includes a mixed MPI+OpenMP parallel version of the SCRIP library for the calculation of the remapping weights (see section 4.3). But besides this aspect, MCT itself, and therefore the communication layer in OASIS, has only minimal OpenMP support at the present time. Users are discouraged from calling OASIS3-MCT oasis_put and oasis_get from a threaded region. And it is not possible to define a partition (decomposition) across multiple active threads. Most of the work that OASIS3-MCT does related to decomposition rearrangement for communication or mapping per se is MPI-based. There are some floating point operations in the sparse matrix multiple as well as in some of the OASIS3-MCT transforms; but generally, these are highly parallel and would often not benefit from use of OpenMP. OASIS3-MCT is designed primarily to support large memory parallel applications that require efficient and scalable coupling and mapping capabilities.
MCT only supports coupling of integer or floating point data. Fields based on logicals and character strings cannot be coupled.