Up to Transformations and interpolations
Hej I am working on a coupled single column model (SCM) version of EC-Earth (OpenIFS SCM, NEMO C1D coupled using oasis3-mct). I have a version of the model which runs fine but I would like to modify the coupling transformation (currently no interpolation) to transform from the OpenIFS single point to the NEMO C1D 3x3 (co-located) grid, and visa versa. At present I expand the OpenIFS single point to a 3x3 grid internally within OpenIFS before passing the fields to the coupler. Any suggestion on how to make this transform within oasis3-mct would be greatly appreciated. Regards Hamish
Hi Hamish, You should use your own remapping and address file reproducing what you do in OpenIFS before passing the fields to the coupler and use the option MAPPING in OASIS (see the User Guide https://oasis.cerfacs.fr/wp-content/uploads/sites/114/2021/02/GLOBC-TR-oasis3mct_UserGuide3.0_052015.pdf section 4.3). Imagine that you expand your single point in OpenIFS to a 3x3 grid and then you decide which source point contribute to each target grid and the corresponding weight (see the User Guide for the remapping and address file format, section 5.4) to construct yourself your remapping file. Best regards, Laure
Hej Laure Thanks you very much for the helpful feedback. You are quite correct, there was one line in the OpenIFS coupling configuration with an incorrect setting. Now that is fixed the coupling is working nicely - thanks again for the assistance. Regards, Hamish
My apologies - it has taken some time to get back to testing the SCM version of EC-Earth. Unfortunately I am running into a problem when I try the remapping using oasis - and I am having a little difficulty finding what I am doing wrong. The error message I get (err.000000) is: [1] MCT::m_SparseMatrixPlus:: FATAL--length of vector x different from column count of sMat.Length of x = 9 Number of columns in sMat = 1 [1] 001.MCT(MPEU)::die.: from MCT::m_SparseMatrixPlus::initDistributed_() [1] application called MPI_Abort(MPI_COMM_WORLD, 2) - process 1 Any suggestions you have would be greatly appreciated. Hamish
Hi Hamish, It seems that your field (put or received) does not have the correct dimension to perform the multiplication. Are you sur that the fields you send (receive) are define on the correct grid ? Best regards, Laure
Hi, I am using OASIS-mct3 to couple COSMO-CLM to the ocean model NEMO Nordic and have a similar error: MCT::m_SparseMatrixPlus:: FATAL--length of vector x different from column count of sMat.Length of x = 111946 Number of columns in sMat = 323737 So I know that the fields do not have the correct dimension but I am wondering why and how I can avoid this error. sMat=323737 is the total number of surface grid points of NEMO (whole domain, “dry” points inclusive) while x = 111946 is the number of only the “wet” surface grid points. I told NEMO to only calculate the wet points to save computational time and I guess OASIS gets this number of points while it expects to receive the whole domain. How can I tell OASIS to expect only the wet points and not all points? Best regards Manuel
Hi Manuel, If you really want to send only the wet points as coupling field, this means that your coupling auxiliary grid data files grids.nc, masks.nc, areas.nc (see the User Guide https://oasis.cerfacs.fr/wp-content/uploads/sites/114/2021/02/GLOBC_TR_oasis3mct_UserGuide_4.0.pdf section 5) has to describe only the wet points. Furthermore, the weights files used for the interpolation also has to consider a grid with only the wet point. The current problem probably comes from the fact that there is a mismatch between the dimension of the coupling field and the remapping matrix. Where does your remapping weight file comes from? Is it computed online in the initialisation or is it pre-computed before the run? I hope this helps? Regards, Sophie
Hi Sophie, Thanks for your answer! My remapping weight files are computed online in the initialisation, but they have the wrong NEMO grid dimension (619*523; whole domain wet+dry). So I think you are correct, that is the error source! The masks.nc seems to be correct, the wet points are masked with 0, the dry points with 1. But how can I let grids.nc and areas.nc only describe the wet points? The NEMO-variables in these files have the dimension of the whole domain (dry+wet). Have I mask these variables? Maybe this is the reason for the wrong grid dimension of the weight file! Could be the dimension of the oasis restart file also a possibility? Best regards Manuel
Manuel, The array of points you send through the oasis_put has to be coherent with the partition defined with oasis_def_partition (see 2.2.3 of the User guide https://oasis.cerfacs.fr/wp-content/uploads/sites/114/2021/02/GLOBC_TR_oasis3mct_UserGuide_4.0.pdf). If you want to send only a subarray of wet points, you have to adjust the description of the partition so to fit the grid of wet points only. And you would have to use the argument isize (= 619*523 in your case) of oasis_def_partition to tell oasis that the sum of the local partition will not cover the whole grid described in grids.nc, masks.nc, areas.nc . This should work but we have never tested that specific case ourselves. We have just used "isize" when processes used to run nemo do not cover partitions that would include only dry points (one can activate that in nemo namelist by setting jpnij different from jpni*jpnj). We have never tested that so to exclude dry point within running processes. We will test that offline with a toy model but this will take us some time as our to-do list is long! With best regards, Sophie
Hi Sophie, "We have just used "isize" when processes used to run nemo do not cover partitions that would include only dry points (one can activate that in nemo namelist by setting jpnij different from jpni*jpnj)." That's exactly what I want to do! I also set jpnij different from jpni*jpnj. I will try to adjust the partition as you described and hope it works. Thank you and best regards Manuel
Hi Sophie, it is still unclear to me how I can adjust the description of the partition. Have I to do this in the oasis-sourcecode in ‘mod_oasis_part.F90’? The parameter isize, which is named ‘ig_size’ here, is set in subroutine „oasis_part_zero“ to ig_size=-1, can I just change it to ig_size= 619*523? And I havent found the part where I can define my partition through ‘ig_paral’. Where can I set this parameter and which of the five types described in the User-Guide would be the best choice for my problem? Thanks a lot in advance and best regards, Manuel
Dear Manuel, Of course, you should not change the OASIS code itself (mod_oasis_part.F90). But you would have to go in the nemo sources, where the different nemo process define their partition calling oasis_def_partition, see cpl_oasis3.F90 in NEMOGCM/NEMO/OPA_SRC/SBC . If you describe only the wet points, I suppose that the grid would be unstructured and you would have to use an orange partition, see the User Guide https://oasis.cerfacs.fr/wp-content/uploads/sites/114/2021/02/GLOBC_TR_oasis3mct_UserGuide_4.0.pdf, section 2.2.3 . I hope this helps? Regards, Sophie
Hi Sophie, Thanks a lot for your help! But if I get the explanation of the partitioning right, I suppose that a box partition is the right choice. Its ok to have rectangular processes. And of course its ok to calculate the dry points of a process, when parts of this process are “wet”. I want that only the processes which are completely dry don’t do any calculations and these are the processes which should not be sent through oasis. Sorry if you got me wrong here! You mentioned before that you “just used "isize" when processes used to run nemo do not cover partitions that would include only dry points (one can activate that in nemo namelist by setting jpnij different from jpni*jpnj)”. That’s exactly what I want to do. So I guess I only have to change isize to isize=619*523. Could you tell me where I can do this? In subroutine “oasis_part_zero” it is set to -1, but I guess that is not the place where I have to adjust it? Best regards Manuel
Hi Manuel, If you want that only the processes which are completely dry don’t do any calculations, you don't have to change anything in the nemo code (unless you use a very old version) and it is supported (we do this all the time). The only thing you have to do is to set jpnij different from jpni*jpnj in nemo namelist. And you have to launch nemo on jpnij number of cores. Of course, you cannot set anything, but there are tools coming with nemo to help you choose the appropriate jpnij, jpni, jpnj. Regards, Sophie
Hi Sophie, mhh okay, I set jpnij different from jpni*jpnj: jpni = 27 jpnj = 21 jpnij = 195 With this combination, NEMO does the partitioning right. Nevertheless, OASIS produces the above mentioned error. Must I adjust something in the namcouple namelist or somewhere else? In namcouple, I changed the number of NEMO grid points for the respective field to the number of points of the wet PEs, but that did not help. Best regards Manuel
Hi Manuel, It is very hard for me to debug without seeing your code and everything. I guess the error you refer to is: "MCT::m_SparseMatrixPlus:: FATAL--length of vector x different from column count of sMat.Length of x = 111946 Number of columns in sMat = 323737" Is x = 111946, the number of wet points? If so, it looks like you send and define a coupling field with only the wet points, which you should not do. You should define and send a coupling field with all points. The "only" difference when you set jpnij different from jpni*jpnj is that nothing will happen for the domains that would be covered only by dry points but this should be totally transparent for you. Besides setting jpnij different from jpni*jpnj and launching nemo on a reduced set of cores, you should not do anything special. With best regards, Sophie
