The SRIDA map

SRIDA pillar 3

Software & Science Enablers

Enabling AI, portability and reproducibility

Challenge

Scientific software represents decades of accumulated knowledge and investment, yet faces growing tensions between longevity requirements and rapid technological change. Major experiments like HL-LHC and SKA will operate for 10-50 years, requiring software that remains functional across multiple generations of hardware and computing paradigms. Simultaneously, AI is transforming scientific workflows, encompassing ML/DL methods used for surrogate modelling and event classification, and increasingly LLM-based and agentic systems that can support code generation, infrastructure operations, and documentation, but most applications remain at proof-of-concept stage rather than production deployment.

Four interconnected challenges constrain the software layer bridging infrastructure and science:

  • AI maturity gap: Machine learning methods are increasingly central to simulation, reconstruction, and analysis, yet most infrastructures treat them as add-ons rather than integral capabilities. Scaling from demonstration to operational deployment is challenging, often dependent on small teams rather than sustainable capacity. The gap between AI potential and AI reality represents a strategic bottleneck.
  • Portability limitations: Scientific codes optimised for one architecture often cannot exploit another without substantial rewriting. The computing landscape now spans CPUs, GPUs, FPGAs and emerging accelerators, each with different programming models and each with typical evolution cycles of 2-5 years. Taking advantage of heterogeneous resources requires software that is portable across centres and automatically exploits available hardware capabilities; this can be realized by careful porting to all the architectures of interest (e.g., EESSI), or by using high-level frameworks designed for portability (e.g. Kokkos, SYCL/oneAPI, OpenMP target offload, or ALPACA).
  • Reproducibility requirements: Scientific insights must be reproducible over a long period requiring preservation of complete computational context including software, workflows, documentation and execution environments. Current practices often fail to capture sufficient information for later reproduction.
  • Long-term preservation: Instrument lifetimes of 10-50 years require software (together with the complete execution environment) that survives across hardware generations and organisational changes. Code developed today must remain executable and maintainable through multiple technology transitions.

Priorities Overview

Pillar 3 enables AI and accelerated computing methods, software portability across heterogeneous architectures, reproducibility of scientific insights, and code preservation across instrument lifetimes. Four priorities address the software lifecycle from development through long-term preservation.

P07: AI. Establish AI as first-class infrastructure across the European compute and data continuum. AI surrogate models can deliver two to three orders of magnitude computational cost reduction for high-impact HEP and RA use cases. Realising this depends on an integrated AI ecosystem that spans HPC, HTC, cloud, and instrument-edge resources, supports federated learning and FAIR model sharing, and is not concentrated in a single class of facility. The strategic lever is active engagement with European AI policy (RAISE, the AI Office, the AI Act), supported by resource-accounting reform that recognises GPU and AI-Factory allocations as pledged for the major experiments (Detail: Annex A, P07).

P08: Software Portability. Develop scientific codes using portable programming models that execute efficiently across diverse architectures (CPUs, GPUs, FPGAs, and emerging accelerators). Adopt hardware abstraction layers such as Kokkos, SYCL, and Alpaka for architecture-agnostic code; package applications in containers (e.g. Apptainer) or modules (e.g., Lmod); and follow FAIR principles for performance portability and reusable software (e.g., FAIR4RS). Build on European tooling already in production, including EESSI through the EuroHPC Centre-of-Excellence MultiXscale and the EuroHPC Federation Platform's Federated Software Catalogue, and on the EOSC Cloud Container Platform for containerised application deployment. (Detail: Annex A, P08)

P09: Open Science and Reproducibility. Ensure scientific workflows and results remain reproducible over 5-10 year timescales through preservation of (a sufficiently) complete computational context, including standardised metadata, provenance tracking, and FAIR-compliant practices for software and workflows alongside data. Connect with e.g., EVERSE for research software best-practice consolidation and with the DORA/sfdora research assessment principles for career recognition based on FAIR and Open Science contributions. Open data alliances (IVOA, IHDEA, IPDA) provide established community standards that the priority builds on. (Detail: Annex A, P09)

P10: Software Preservation and Sustainability. Ensure scientific codes remain functional across the 20-50 year lifetimes of major instruments (HL-LHC programme to its successors, SKA1 to SKA2, LOFAR through LOFAR 2.0) through systematic preservation and modernisation, with documentation as an integral part of preservation policies and Software Management Plans. HEP's long collaborative legacy from LHC informs emerging RA infrastructures such as SRCNet (still in build-out, so the comparison must be drawn carefully). The priority can be supported by e.g., EVERSE, the SPECTRUM Knowledge Hub, the ESCAPE ESFRI Cluster, and networks such as Radionet, with advocacy from DORA and the ADORE declaration of the Research Software Alliance. Promote open-source and open-standards policies to prevent sovereignty issues and avoid proprietary vendor lock-in. (Detail: Annex A, P10)

Interdependencies: The four priorities form a coherent approach to sustainable scientific software. Portability (P08) enables reproducibility (P09) by decoupling software from specific environments. AI/ML integration (P07) depends on portable software methods (P08) to deploy models across heterogeneous resources. Long-term preservation (P10) requires the documentation and standardisation practices developed for reproducibility (P09).

Recommendations

For Scientific Communities and Research Infrastructures

  • Invest in transitioning AI applications from proof-of-concept to production deployment (P07), including evaluation of LLM-assisted ("vibe-coding") workflows as a possible accelerator for porting scientific codes to new architectures and for sustaining them across hardware generations (P10)
  • Engage with the RAISE AI for Science pillar to secure HEP and RA representation and contribute use cases for AI Factory pilot workloads (P07)
  • Adopt portable programming models and (micro-)architecture optimized distribution for new software developments (P08)
  • Implement FAIR Principles for research softwares (e.g., FAIR4RS) guidelines for software documentation and preservation (P09, P10)
  • Engage with the i.a., the EVERSE software-quality framework and the EOSC science-cluster software repositories such as the ESCAPE Open-source Scientific Software and Service Repository (OSSR) (P08, P09, P10)

For e-Infrastructures and Service Providers

  • Deploy AI platforms as integral infrastructure capabilities, not add-ons (P07)
  • Provide federated-learning infrastructure for privacy- and sovereignty-constrained training, and operate AI Factory pilots that integrate HEP and RA workloads with EuroHPC AI infrastructure (P07)
  • Support common software repositories and distribution systems (P08)
  • Align service offerings with the community-driven and maintained software-excellence frameworks and science-cluster software repositories (e.g., EVERSE, ESCAPE OSSR (P08, P09))
  • Provide execution environments and support workflow tooling that enable reproducibility and provenance capture (P09)

For Policy Makers and Funding Bodies

  • Fund transition of AI applications from proof-of-concept to production deployment (P07)
  • Fund inference infrastructure integration across distributed computing resources (P07)
  • Require reproducibility standards in funded research (P09)
  • Fund software preservation as an essential investment component of research and digital infrastructures (P10)
  • Reform resource-accounting frameworks so that GPU and AI-Factory allocations are accepted as pledged resources within WLCG and SRCNet accounting (P07)
  • Engage the AI Office on AI Act application to scientific AI, including model documentation, dataset transparency, and risk classification for safety-relevant operational AI such as detector control and observatory operations (P07)
  • Sustain pan-European software-excellence frameworks such as EVERSE, science-cluster software repositories such as the ESCAPE Open-source Scientific Software and Service Repository (OSSR), and optimized software compilation, packaging and distribution such as EESSI as cross-priority enablers for software quality, training, and FAIR-for-software practices (P08, P09, P10)

Priorities

On the map