HiPerGator

HiPerGator Hardware Specification Sheet

View the most recent version of the HiPerGator hardware spec sheet

HiPerGator 1.0:
  • 21,000 cores
  • 64 terabytes of RAM
  • 2 petabytes of disk storage
  • Maximum speed of 157 teraflops = 157 trillion floating point operations
HiPerGator 2.0:
  • 30,000 cores
  • 120 terabytes of RAM
  • 1 petabyte of disk storage
  • Maximum speed of 1,100 teraflops – 1,100 trillion floating point operations
HiPerGator 2.0

To put these numbers in context, consider the following comparisons:

  • The 1,100-teraflop speed of HiPerGator 2.0 is the same as the combined speed of 600 PlayStation 4 and 840 Xbox One gaming stations, if they could be made to work together effectively and efficiently on large data and complex problems.
  • The human brain is estimated to have the power equivalent of 20 petaflops.
  • The 120 terabytes of RAM (not the 1 petabyte of long-term disk storage) of HiPerGator 2.0 can hold 240 million books.

Nodes and processors

Year Family Node Model Chip Model Nodes Sockets Cores RAM (GB)
Westmere X5675 8 2 12 94
Westmere Dell C6100 X5675 16 2 12 92
Sandy Supermicro X9DRG E5-S2643 7 2 8 62
2015 Haswell Dell SOS6320 E5-S2648 900 2 32 125
Haswell E5-2683 11 2 28 125
Nehalem X7560 1 2 16 125
Sandy E5-4607 1 4 24 750
Skylake X8186 1 4 192 1510
2013 Dhabi Dell C6145 O6378 160 4 64 250
Opteron Atlus 2840 O6220 64 2 16 60
Opteron O6220 14 2 16 29
Magny O6174 1 4 48 496

Network and node interconnect

The nodes are connected by Mellanox 56Gbit/s FDR InfiniBand interconnect for fast data access to storage and for distributed memory parallel processing. The core switches use the 100 Gbit/s EDR InfiniBand standard. There is a 10 Gbit/s Ethernet for management and for providing the operating system to the diskless nodes, and a separate 10 Gbit/s Ethernet network connecting the nodes to support the planned new services of running virtual machines and virtual cluster on HiPerGator.

Storage

The storage options for HiPerGator are described on the Storage Types policy page. The current total size of HiPerGator storage is 2 petabytes (PB). The default path for blue storage is /ufrc. The default path for orange storage is /orange.

Electrical power and performance measures

Some researchers are interested in monitoring electrical power consumption of their computations. The SLURM scheduler generates a power summary at the of a job; by using SLURM directives you can control many details of your job, such as whether all tasks run on a single node and whether the node has the Intel processors or AMD processors. However, because a HiPerGator is a production system, we do not allow users to change the frequency of processors, or to do other research that would involve full dedication of hardware and root access.