Cerfacs Enter the world of high performance ...

CERFACS’ computing resources

Resources – last update: Sept. 2019 –

Two computers provide to CERFACS an aggregate peak capacity of about 880 Tflop/s for processing our main simulation requirements. To these internal resources are added those of our partners (Meteo-France and CCRT). To afford an additional Support to our research activities (thesis and ANR projects), the resources allocated through GENCI’s calls  on the three national centers (Cines, Idris and TGCC) significantly extend our academic resources. These resources are complemented by our participation in international calls (ex. Prace and Incite programs).

CERFACS’ Internal resources

Kraken Cluster (577 peak Tflop/s)

Compute Partition (498 peak Tflop/s): The Kraken cluster includes 185 compute nodes, each of them with two Intel Xeon Gold 6140 processors (18 cores skylake processor at 2.3 Ghz) and 96 GB DDR4 memory.

2 computing nodes, each of them with two AMD Rome processors (64 cores at 2 Ghz) and 256 GO memory

Pre/Post processing Partition (79 peak Tflop/s):

Deep Learning and AI support : 2 nodes accelerated with one Nvidia V100 + 1 node accelerated with 4 Nvidia V100 interconnected with Nvlink + 1 node accelerated with 1 Nvidia Titan4 (optimized for inferences)

Vizualisation support : 5 nodes with 288 GO memory with Nvidia Tesla M60 card. NICE environment provides remote display to internal / external user’s.

Big memory support : one node with 768 GB memory used for large mesh generation + one node with 1.5 PO of memory dedicated to climate modeling.

All nodes of Pre/Post processing partition are bi-socket nodes with Intel Xeon Gold 6140 processors.

Internal network, storage and software environment: The interconnection network is a non-blocking Omnipath Network. An internal GPFS file system offers to users a 0.5 PO scratch dir capacity. Software environment includes intel development compilers, libraries and tools; TotalView and DDT debuggers; and SLURM job manager. Integrated by Lenovo and serviware, this cluster is in production mode since May 2018.

Nemo Cluster (300 peak Tflop/s)

Nemo3

Compute Partition (276 peak Tflop/s): The Nemo cluster includes 288 compute nodes, each of them with two Intel E5-2680 processors (12 cores haswell processor at 2.5 Ghz) and 64 GB DDR4 memory.

Pre/Post processing partition (13 peak Tflop/s): 12 post-processing nodes with 256 GB memory and Nvidia accelerator + one node with 512 GB memory used for large mesh generation. All these nodes are bi-socket Intel E5-2680.

Knight Landing Partition (11 peak Tflop/s): A four nodes partition of Intel Knight Landing processors (64 cores @ 1.3 Ghz) allow researchers to port and optimize their codes in this environment.

Internal network, storage and software environment: The interconnection network is a non-blocking FDR Infiniband network. An internal GPFS file system offers to users a 1 PO scratch dir capacity. Software environment includes intel development compilers, libraries and tools; TotalView and DDT debuggers; and SLURM job manager. Integrated by Lenovo and serviware, this cluster has been inaugurated on September 30th, 2015.

Scylla Cluster (Big Data Post-Processing)

Inaugurated in February 2019 Scylla cluster is dedicated to big data files management and post-processing. Mainly CMIP5 and CMIP6 (Coupled Model Intercomparison Project Phase 5 and 6) data computed by Cerfacs’ researchers in the Frame of GIEC activities are managed on this cluster.

This cluster is also shared with other research Cerfacs’ teams needing big storage management capacities close to post-processing nodes.

Storage Capacity : 1.4 Po user space. DSS solution (based on IBM Spectrum Scale offerings). 2 nodes dedicated to Metadata management on SSD disks ans 2 nodes dedicated to data management stored on 166 disks each of then with 12 TO capacity.

Pre/Post processing partition :

3 bi-socket Intel Gold 6126 (14 cores @ 2.6 Ghz) with 384 Go memory,

1 bi-socket Intel Gold 6126 with 768 GO memory.

Each of these node is equiped with a Nvidia P4000 accelerator.

Central NAS Server

A central NFS server with a capacity of 1.2 PO is accessible from all clusters and workstations. Its function is to provide a secondary archiving service used by internal and external servers hosting the results of numerical simulation. This technical solution is supported by a 2 LENOVO GPFS Servers associated to a DDN SFA7700 storage solution.

CERFACS’ External computers acces

Météo-France and CEA CCRT extend our simulation capacity through the access to their supercomputers in the frame of partnerships.

  • Météo-France research supercomputer (Beaufix): 1 836 nodes bi-socket Xeon Broadwell 20c @ 2.2 Ghz – 2.59 Pflop/s. On 2018 and 2019 a special allocation of 60 Mh has been allocated by Météo-France to Cerfacs’ researchers in the frame of common GIEC simulations.
  • CCRT supercomputer (Cobalt): 1 422 nodes bi-socket Xeon Broadwell 14c @ 2.4 Ghz + 252 nodes bi-socket Xeon Skylake 20c @ 2.4 Ghz

Through numerous collaboration and support of Genci, Prace and Incite CERFACS accesses multiple external computers. Genci allows our doctoral students to access national resources centers:

Prace attributes the resources to support our borders simulations.

NEWS

Young PhD day (JDD) – Thursday 8 October 2020

Brigitte Yzel |  24 September 2020

  Ph.D. Students' Day (JDD 2020) Thursday 8 October 2020 Virtual event open to our partners and associates   Planning   The 2020 edition of the PhD Students' Day will take place on October 8th, 2020. This year, this day of the PhD students of the Cerfacs, will take place mainly in visio in order to respect all the health safety measures relating to Covid-19. This day will be held in a virtual way through oral presentations filmed and broadcast on a private link, which we will distribute to all participants. Everyone will thus be able to discover at a distance the subjects developed by the doctoral students. At the same time, the "Posters session" will be opened on the morning of the JDD, by sending all the posters in pdf format by e-mail to all the participants for consultation, and by opening the "50-minute question session" on a channel whose links we will send you. This will allow participants to ask questions about the posters of the session. Finally, we have included in this 5th edition a great novelty, by allowing several students to present their subject in the popularised form of "My thesis in 3 minutes", guided by Matthieu Pouget, actor and theatre director. This day will end with the vote for the "Best poster" and the "Best presentation of My thesis in 3 minutes". Participation in the event is free of charge. However, for logistical reasons, registration, only by invitation, is mandatory.   Organising Committee:  Scientific committee  Nicolas Venkovic Victor Xing Inscription &  information Jade Schweiger Brigitte Yzel  Read more


Sparse Days 2020, Cerfacs

Brigitte Yzel |  24 September 2020

Sparse Days Meeting 2020  23 - 24 November 2020 Virtual Workshop   The annual Sparse Days meeting will be held this year as a virtual event, on 23 - 24 November 2020. Sadly, because of the pandemic, we will not be able to have a normal face-to-face Sparse Days at Cerfacs this year. We will be holding an online version from 3pm to 7pm on 23 and 24 November. The timing is chosen to accommodate participants from North America since, at normal Sparse Days, we have a good representation from there. A registration form  is available on this conference website where you can also submit a title and an abstract for any proposed talk. We will probably be using the Webex system and will post details about how to connect to all people who register.Read more

ALL NEWS