Up to Specific issues in real coupled models
Hi all, Coupling cost (interpolation and communication) could be very computational high when the global conservation method is used without activating any optimisation option. The results for EC-Earth obtained by the BSC group in Barcelona show that using 128 processes for IFS the coupling time is small comparing to other calculations, but using 512 processes for IFS the coupling time becomes very significant and represent a big portion of the time-step of IFS. This is due to several communications one-to-all/all-to one and to the fact that the global conservation calculation is done, at least partially, only for one IFS master process. This implementation increments the overhead when more and more processes are used. It represents more than a 50% of time execution when 512 processes are used. Coupling field gathering, an option offered in OASIS3-MCT (see "Support to couple multiple fields via a single communication” in the User Guide), can be used to optimise coupling exchange between components. Results obtained by the BSC group in Barcelona show that gathering all the fields which use similar coupling transformations, the execution time of the coupling is reduced a 40%. This happens because OASIS is able to do communications and interpolation of all the fields gathered at the same time. A reproducibility test, taking into account the chaotic nature of climate models and the small differences introduced by parallel operations such as a reduction, has been done. The test shows that differences are less than 1% when comparing the simulations using and not using the field gathering optimisation, which is of the order of the difference between two different parallel simulations using exactly the same options. The “opt” option of OASIS3-MCT can be used to optimise the global conservation (“CONSERV”post-processing operation). Results obtained by the BSC group in Barcelona show that using this option, the coupling done from IFS to NEMO is reduced by 90%. The reason is that global communications (gather/scatter and reduction) are used instead of all-to-one/one-to-all MPI communications. Besides, conservation calculation is done among all IFS processes instead of using one IFS master process for these calculations. Again a reproducibility test (see previous paragraph) has been done. The test shows that differences are less than 1% when comparing the simulations using and not using the “opt” option, which is of the order of the difference between two different parallel simulations using exactly the same options. Best regards, Sophie
