On the Limitations of Neural Collapse for the Detection of Out-of-Distribution Observations: Can We Do Any Better?

In the last decade AI systems have become increasingly popular in real-world applications. As such, ensuring their reliability and safety in open-world settings remains a critical task for the field. Within AI safety, Out-of-Distribution (OoD) detection aims to prevent AI models from making overconfident (and potentially dangerous) predictions when presented with inputs outside the domain of their training data. Recently, the Neural Collapse (NC) phenomenon has been employed to detect OoD instances in neural networks by relying solely on the similarity between the latent vectors and the weights of the classification layer, with no noticeable impact on inference speed. While NC-based OoD detection framework has shown competitive performance, we argue that several overlooked limitations restrict the robustness and broader applicability of this approach. In this paper we present a critical analysis of these limitations, identifying three core challenges: 1) the dependency on the generalization of the base model, which may lead to unreliable overconfident estimates; 2) structural dataset issues that break the NC assumption, particularly in large-scale datasets like ImageNet; and 3) the alignment between the out-of-distribution datasets used for benchmarking and the requirements of the AI model’s operational design domain, which may affect the fairness of performance comparisons. As a result of our analysis, we provide learned lessons and outline workarounds to overcome these limitations in real-world scenarios, showing how minor adjustments to the NC-based OoD detection framework can potentially improve its effectiveness.