Communication Dans Un Congrès Année : 2024

Designing Quality MPI Correctness Benchmarks: Insights and Metrics

Résumé

Several MPI correctness benchmarks have been proposed to evaluate the quality of MPI correctness tools. The design of such a benchmark comes with different challenges, which we address in this paper. First, an imbalance in the proportion of correct and erroneous codes in the benchmarks requires careful metric interpretation (recall, accuracy, F1 score). Second, tools that detect errors but do not report additional information, like the affected source line or class of error, are less valuable. We extend the typical notion of a true positive with stricter variants that consider a tool’s helpfulness. We introduce a new noise metric to consider the amount of distracting error reports. We evaluate those new metrics with MPI-BugBench, on the MPI correctness tools ITAC, MUST, and PARCOACH. Third, we discuss the complexities of hand-crafted and automatically generated benchmark codes and the additional challenges of non- deterministic errors.
Fichier principal
Vignette du fichier
main.pdf (206.87 Ko) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-04878332 , version 1 (20-01-2025)

Identifiants

  • HAL Id : hal-04878332 , version 1

Citer

Tim Jammer, Simon Schwitanski, Emmanuelle Saillard, Alexander Hück, Joachim Jenke, et al.. Designing Quality MPI Correctness Benchmarks: Insights and Metrics. 8th International Workshop on Software Correctness for HPC Applications (Correctness '24), Nov 2024, Atlanta, United States. ⟨hal-04878332⟩
0 Consultations
0 Téléchargements

Partager

More