JMIR Publications

ABSTRACT

Background:

The increasing use of real-time health data from wearable devices and self-reported questionnaires offers significant opportunities for preventive care in ageing populations. However, current health data platforms often lack integrated traceability mechanisms, model versioning, and seamless management of heterogeneous data streams, which are critical for clinical accountability, regulatory compliance, and reproducibility.

Objective:

This work presents DeltaTrace, a unified big data health platform built on open source technologies. It integrates data traceability, real-time and batch processing, and core features such as visualization and scheduling. Designed for scalability and auditability, the platform supports robust versioning of data and models to enable effective health data management.

Methods:

DeltaTrace is evaluated using continuous physiological signals from wearable devices and questionnaire data ingested in batch mode, combining synthetic and real data from the LifeSnaps dataset. Performance tests are conducted on a single CPU-only server with 8-core and 24-core configurations to measure ingestion, visualization, aggregation, and anomaly detection latencies.

Results:

The platform supports continuous processing for approximately 1,500 users with delay times under 10 minutes. Ingestion and visualization tasks consistently operated between 4.7 and 5.8 minutes, while statistical aggregation and anomaly detection required less than 5.6 and 10.5 minutes, respectively. Increasing cores from 8 to 24 improved ingestion and cleaning latency by up to 25% and anomaly detection performance by up to 50% under high loads.

Conclusions:

DeltaTrace shows a scalable and modular architecture that ensures reliable, traceable processing of health data under minimal hardware requirements, with flexibility to improve performance via additional resources. This enables timely clinical interventions for chronic disease management in ageing populations.