RENCI has a 24,000+ square foot facility including a 2,000 square foot data center at 100 Europa Drive, Chapel Hill.
Europa Data Center
- 2,000 square feet of floor space on an 18-inch raised floor
- 600 kVA Commercial power
- 375 kVA UPS power
- 30 kVA Generator power
- 134 Tons dedicated cooling
- Room for 40 Racks of High-performance Computing, Storage, and Networking
Computational Infrastructure
RENCI began operations as an organization starting in 2004. Since that time, the organization has acquired various computational systems to support projects and activities. The following is a list of the major computational infrastructure currently active at RENCI.
HPC (Hatteras)
Hatteras is a 3744-core cluster running CentOS Linux and the SLURM resource manager. Hatteras is segmented into several independent sub-clusters with varying architectures. It is capable of concurrently running one 512-way parallel job and two 1488-way parallel jobs.
Hatteras’ sub-clusters have the following configurations:
- Chassis 8 (512 interconnected cores)
- 16 x Dell M630 Half-Height Blade Server
- Two Intel Xeon E5-2683v4 CPUs (2.1GHz, 16-core)
- 256GB 2400MHz RAM
- 100GB SSD for local I/O
- 56Gb/s Mellanox (4X FDR) Interconnect
- Rack R1-04 GPU (48 interconnected cores)
- 1 x Dell PowerEdge R750xa Servers
- Two Intel Xeon Gold 6342R CPU (2.80GHz, 24-core)
- 1.5TB 3200MHz RAM
- Three NVIDIA Tesla A100
- 100Gb/s Mellanox (4X EDR) Interconnect
- Rack R1-05 Compute (1488 interconnected cores)
- 31 x Dell PowerEdge R640 Servers
- Two Intel Xeon Gold 6240R CPU (2.40GHz, 24-core)
- 192GB 2933MHz RAM
- 100Gb/s Mellanox (4X EDR) Interconnect
- Rack R1-05 Largemem (96 interconnected cores)
- 2 x Dell PowerEdge R640 Servers
- Two Intel Xeon Gold 6240R CPU (2.40GHz, 24-core)
- 1.5TB 2933MHz RAM
- 100Gb/s Mellanox (4X EDR) Interconnect
- Rack R1-05 GPU (96 interconnected cores)
- 2 x Dell PowerEdge R740xd Servers
- Two Intel Xeon Gold 6240R CPU (2.40GHz, 24-core)
- 192GB 2933MHz RAM
- Two NVIDIA Tesla V100
- 100Gb/s Mellanox (4X EDR) Interconnect
- Rack R1-06 Compute (1488 interconnected cores)
- 31 x Dell PowerEdge R640 Servers
- Two Intel Xeon Gold 6240R CPU (2.40GHz, 24-core)
- 192GB 2933MHz RAM
- 100Gb/s Mellanox (4X EDR) Interconnect
- Rack R1-06 Largemem (96 interconnected cores)
- 2 x Dell PowerEdge R640 Servers
- Two Intel Xeon Gold 6240R CPU (2.40GHz, 24-core)
- 1.5TB 2933MHz RAM
- 100Gb/s Mellanox (4X EDR) Interconnect
- Rack R1-06 GPU (96 interconnected cores)
- 2 x Dell PowerEdge R740xd Servers
- Two Intel Xeon Gold 6240R CPU (2.40GHz, 24-core)
- 192GB 2933MHz RAM
- Two NVIDIA Tesla V100
- 100Gb/s Mellanox (4X EDR) Interconnect
Kubernetes
RENCI manages one Kubernetes cluster in the Europa Data Center named Sterling and several project-specific clusters on Google Kubernetes Engine (GKE).
The Sterling cluster consists of the following hardware:
- 12x Dell PowerEdge R640 servers, each with 96 CPU and 1.5TB RAM
- 3x Dell R[7,8]40 Servers with 96 CPU and 1.5TB RAM
- 6x Nvidia Ampere A100 GPUs
- 11x 4TB NVMe Drives
The ASHE OpenShift Kubernetes cluster in Manning Datacenter consists of the following hardware:
- 6x Dell PowerEdge R640 servers, each with 96 CPU and 1.5TB RAM
We configure our GKE clusters to take advantage of several fully automated features to provide a wide variety of resources to the application. These features include:
- Node Types: compute vs. memory optimized, GPU capable, local storage
- Auto Scaling: scales number of nodes based on resource utilization (CPU, memory, GPU)
- Node Pools: specific node pools are utilized depending on resource need
- Node Repair: auto repair process is initiated for unhealthy nodes
Storage Infrastructure
The RENCI Storage Infrastructure includes:
- NetApp Clustered Data ONTAP
- FAS8300 Europa node HA Pair
- AFF-A300 Europa node HA Pair
- FAS8300 Manning node HA Pair
- AFF-A400 Manning node HA Pair
