Analysis of gene set enrichment is an essential part of functional omics studies. Here, we complement the established evaluation metrics of gene set enrichment algorithms with a novel approach to assess the practical reproducibility of scientific results obtained from gene set enrichment tests when applied to related data from different studies.