Leveraging private container networks for increased user isolation and flexibility on HPC clusters - Université de Bordeaux
Communication Dans Un Congrès Année : 2024

Leveraging private container networks for increased user isolation and flexibility on HPC clusters

Résumé

To address the increasing complexity of modern scientific computing workflows, HPC clusters must be able to accommodate a wider range of workloads without compromising their efficiency in processing batches of highly parallel jobs. Cloud computing providers have a long history of leveraging all forms of virtualization to let their clients easily and securely deploy complex distributed applications and similar capabilities are now expected from HPC facilities. In recent years, containers have been progressively adopted by HPC practitioners to facilitate the installation of applications along with their software dependencies. However little attention has been given to the use of containers with virtualized networks to securely orchestrate distributed applications on HPC resources. In this article, we describe a way to leverage network virtualization to benefit from the flexibility and isolation typically found in a cloud environment while being as transparent and as easy to use as possible for people familiar with HPC clusters. Users are automatically isolated in their own private network which prevents unwanted network accesses and allows them to easily define network addresses so that components of a distributed workflow can reliably reach each other. We describe the implementation of this approach in the pcocc (private cloud on a compute cluster) container runtime. We evaluate both its overhead as well as its benefits for representative use-cases on a Slurm based cluster.

Fichier principal
Vignette du fichier
wocc24_final.pdf (212.63 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04740275 , version 1 (16-10-2024)

Licence

Identifiants

  • HAL Id : hal-04740275 , version 1

Citer

Lise Jolicoeur, François Diakhaté, Raymond Namyst. Leveraging private container networks for increased user isolation and flexibility on HPC clusters. WOCC 2024 - 2nd International Workshop on Converged Computing on Edge, Cloud, and HPC, May 2024, Hamburg, Germany. ⟨hal-04740275⟩
0 Consultations
0 Téléchargements

Partager

More