Up to Specific issues in real coupled models
Hello, I have a development question. I noticed that the user documentation has a footnote starting with "The model may call MPI Init explicitly...". I was wondering if this could be used to pass in MPI_THREAD_MULTIPLE for instance to allow multithreaded MPI communication in the coupled models without affecting internal operations of OASIS? Thank you for your time.
Hi, We are not expert of MPI+OpenMP but we have a user who indeed couples two codes with OASIS3-MCT, one of which being MPI+OpenMP. In his case, in the mixed MPI+OpenMP code calls MPI_Init (via the call to oasis_init). Then in the OpenMP part of the code, only the master thread of each MPI task calls the OASIS3-MCT routines (oasis_init_comp, oasis_def_partition, oasis_def_var, oasis_enddef, oasis_put, oasis_get, …) for the part of the coupling field treated by the whole MPI task (not only for the part of the coupling field treated by the thread itself). An OMP scatter is done after an Oasis_recv and an OMP gather is done before an Oasis_send. And this works fine. In his case, as one code is mixed MPI+OpenMP and the other is only MPI, he has to explicitly assign cores to each MPI task of his two codes. For example, if one MPI task of the mixed MPI+OpenMP code has 4 thread, he assign 4 cores to this MPI task; but for the other code that is only MPI, each MPI task will have one core assigned. It seems that this explicit assignment is possible on the Bullx Curie. On platforms where the explicit assignment is not possible, the number of cores per MPI task for both codes will have to be the number of threads associated to each MPI task in the mixed MPI+OpenMP code (and this will lead to a waste of resources for the MPI only code). Best regards, Laure
