Research Computing 2021 End-of-Year Status
By Jon Akers
📅 December 23, 2021
HiPerGator and HiPerGator-RV status
- Both HiPerGator 3.0, with its 70,000 cores and 608 NVIDIA low-end GPU cards, and HiPerGator AI, the NVIDIA SuperPOD with140 DGX A100 nodes holding 1,120 top-of-the-line NVIDIA Ampere A100 GPUs, have been in full production since June 2021.
- With the AI initiative at UF, many people have been storing more data so that both the Blue and Orange file systems needed expansion. In August, Blue was expanded from 4 PB to 7.1 PB and Orange grew from 3 PB to 5.6 PB.
- The Orange file system usage kept growing, however, and we purchased more disks this month, December 2021, to expand it further to 16 PB in January 2022.
- This month, December 2021,, the computing environment for research on restricted data was migrated to the new HiPerGator 3.0 hardware. We used this transition to update the branding: ResVault is now called HiPerGator-RV.
- The Catalyst awards made by the UF Office of Research in Summer of 2020 included allocations of resources on HiPerGator. These projects will present their findings and results at the upcoming HiPerGator Symposium in the Spring 2022.
- Collaboration with NVAITC, the NVIDIA AI Technology Center, is very strong. The two permanent scientist/engineers assigned by NVIDIA to UF work closely with UFIT staff in training and consultation of UF faculty. The NVAITC will provide special support for projects that need it. Proposals are by invitation only; contact RC staff to discuss your ideas. Several such projects are already being worked on.
- Elias Eldayrie, UF's CIO, has sent a letter to all deans inviting them to ask RC staff to present options and resources to college assemblies and departmental faculty meetings
- RC and NNAITC staff have worked with numerous faculty to prepare proposals for NSF, NIH, DOE and other opportunities.
- UFIT and UFII organized and hosted 20 AI focused events, workshops, and Birds-of-a-Feather sessions with hundreds of people from UF and partner institutions in Florida and the SEC.
- UF and NVIDIA jointly organized the AI Bootcamp this fall for UF students and researchers.
- This past summer, a student club for AI was created, called GAITOR Club. UFIT has been working with them and supporting them with resources and training.
Summary of services
- HiPerGator and HiPerGator AI offer an advanced HPC (High-Performance Computing) environment, based on the standard Linux command-line and batch-job model for access and use.
- Graphical User Interfaces (GUI) are offered for various tools: Galaxy, Matlab, R-Studio, JupyterHub, and OpenOnDemand to name a few.
- Note that HiPerGator is not a cloud environment with the ability to configure and run VMs. There is no support for Windows.
- HiPerGator-RV offers computing, including HPC using GPUs for simulation and AI, on restricted data.
- For security reasons, different users and user groups are isolated from each other by working inside virtual machines.
- As a result, HiPerGator-RV does support both Windows and Linux.
What is coming?
- From the news, security of systems and data is on everybody's mind. The result is a much higher level of scrutiny from all angles about cybersecurity controls. More and more contracts UF needs to enter require some assurance that data is properly safeguarded.
- UF is one of the leaders in the Nation and in the world in providing a secure computing environment, namely HiPerGator-RV, that has been assessed by external auditors to meet stringent compliance requirements, such as HIPAA, FISMA, and CUI.
- The use of HiPerGator-RV comes with severe restrictions because of the security controls. We are therefore investing in making the whole HiPerGator certified for work with PHI, which is a less stringent control than FISMA and CUI. This will be the goal for 2022.
- With that certification, standard Linux, command-line, batch-job, and GUI tools can be used on PHI after approval and training has been received.
- Note that there still will be no support for virtual machines or Windows on HiPerGaator outside of HiPerGator-RV.