Data infrastructure at RespiQ
Why data infrastructure matters
Each time our device analyses a sample, it records high-dimensional data. To interpret this data, we also need to know the measurement context: what was measured, under which conditions, with which device configuration, and which reference it should be compared to. If that information is missing or recorded inconsistently, the measurement cannot be reliably analysed, compared with other measurements, or reproduced.
In our previous articles, we described how we use machine learning to detect disease biomarkers at parts-per-billion levels, and why our models must be traceable to features with a clear scientific meaning. Both depend on the data infrastructure that connects our lab work to our analysis.
The problem we are solving
Reproducibility is a well-known challenge in science (1). In machine learning, this matters even more, as a model can only be as reliable as the data it is built on. Therefore, collecting, verifying, and managing that data is a substantial part of the work (2). At RespiQ, this work is part of the feedback loop between experimentation and data science that we described in our first blog post. The speed of that loop depends on how quickly new measurements can be found, checked, and analysed. This is why our data infrastructure is designed to make these steps fast and consistent.
Furthermore, regulators expect medical AI to be developed with well-managed, representative and traceable data, as described in the Good Machine Learning Practice principles for medical device development (5) and the data governance requirements of the EU AI Act (6). Validated data, access control, traceability, and reproducible analysis are requirements for both R&D and the final clinical product.
The RespiQ data platform: measurements from the lab are stored in the RespiQ data lake, where they are made available to the RespiQ library and to our partners. We then use the results to design the next experiment.
The RespiQ datalake
The RespiQ datalake is our central storage system for measurement data. Every measurement is stored together with structured information about the experiment: what was measured, how, under which conditions, and as part of which study.
This information is validated when the data is uploaded. The platform checks that records are complete and consistent with our experimental rules, so errors are found at upload rather than during analysis. For example, a breath measurement cannot be stored without being linked to a study participant, and access to that data is restricted to authorised users, in full compliance with GDPR regulations.
Authorised users can search, browse, and visualise experiments in the browser, including comparing measured spectra with reference emission lines from the NIST Atomic Spectra Database (3). When we introduce new devices, protocols or projects, the data model is updated, and existing records are migrated, so older measurements remain usable alongside new ones.
This design follows the FAIR principles for scientific data management: data should be Findable, Accessible, Interoperable, and Reusable (4).
Each measurement is stored with the information needed to interpret it: the sample, the measurement conditions, the device setup, quality checks, provenance, and access permissions.
The RespiQ library
The RespiQ library is the proprietary Python package. These have been developed and refined through our own research and cover the full process from raw measurement to result: assessing data quality, preprocessing, extracting features with a chemical interpretation, and building and evaluating models. Many of these methods are specific to our technology and are not available in standard software packages. Like the datalake, the library is built to meet the regulatory expectations described above: it is tested, documented, and version-controlled, so every method is validated before use and every change is recorded.
The library connects directly to RespiQ datalake. Authorised users can select all experiments that match a set of conditions, load the data together with its metadata, and run our standard analysis workflows in a few lines of code.
Keeping our methods in one shared library has the following benefits:
Speed: results are available shortly after the data is uploaded, without manual data preparation.
Consistency: all users follow the same validated methods, so results from different users, time periods, and device versions can be compared directly.
Shared methods: when a method is developed or improved, it is validated once and then made available to the whole team. New team members can use the existing methods from the start.
Traceability: each result can be traced back to the measurements and the version of the methods that produced it, so analyses can be reproduced.
Traceability is also a requirement for the interpretability approach described in our previous article. To explain why a model reached a conclusion, we need to know which data it was trained on and how that data was processed.
Access for partners
We work with clinical, academic, and industrial partners. To support these collaborations, RespiQ datalake provides programmatic access through an API. Our partners receive personal access keys with permissions managed by RespiQ, and a dedicated Python client allows them to query experiments, load data, and review quality metrics in their own analysis environment.
This means our partners use the same data and definitions as our team. There’s no need to exchange and reconcile files manually, and each party can independently verify the results of joint studies.
How this speeds up R&D
RespiQ datalake connects the work of experimental scientists and data scientists: measurements are uploaded with their metadata, loaded for analysis, and the results are used to improve the device and plan the next experiment.
As an example, consider a new device configuration being tested in the lab. After the measurements are uploaded, the system validates the data, and ensures that quality metrics are available. A data scientist can analyse the new data directly and compare it with previous configurations measured under similar conditions. If the analysis shows where the signal is strongest or where measurements are less reliable, this can be used to plan our next experiments.
Because each result is linked to the data that produced it, new experiments can be compared directly with earlier ones. A shorter loop allows us to run more iterations of experiments and analysis, which helps us improve detection limits faster.
Next steps
As we extend our work to new biomarkers, applications, and patient populations, the amount and variety of our data will increase. We will continue to develop the RespiQ datalake and the respiQ library to support this, so that new data remains easy to find, analyse, and reproduce.
RespiQ is developing AI-powered breath analysis technology for early disease detection and continuous health monitoring. Our data platform stores every measurement with its experimental context, supporting reproducible analysis, secure collaboration with partners, and faster iteration between the lab and our models.
References
Baker, M. 1,500 scientists lift the lid on reproducibility. Nature 533, 452–454 (2016). doi: 10.1038/533452a
Sculley, D. et al. Hidden Technical Debt in Machine Learning Systems. Advances in Neural Information Processing Systems 28, 2503–2511 (2015).
Kramida, A., Ralchenko, Yu., Reader, J. & NIST ASD Team. NIST Atomic Spectra Database (version 5.12), [Online]. National Institute of Standards and Technology, Gaithersburg, MD (2024). doi: 10.18434/T4W30F
Wilkinson, M. D. et al. The FAIR Guiding Principles for scientific data management and stewardship. Sci Data 3, 160018 (2016). doi: 10.1038/sdata.2016.18
U.S. Food and Drug Administration, Health Canada & Medicines and Healthcare products Regulatory Agency. Good Machine Learning Practice for Medical Device Development: Guiding Principles (2021).
Regulation (EU) 2024/1689 of the European Parliament and of the Council laying down harmonised rules on artificial intelligence (Artificial Intelligence Act), Article 10: Data and data governance. Official Journal of the European Union (2024).
Written by Chandan Sreedhara, 08/10/2026
Reviewed by Laura Verga & Anouk Post, 09/10/2026
