Currently submitted to: JMIR Biomedical Engineering
Date Submitted: Nov 4, 2026
Open Peer Review Period: Nov 17, 2026 - Jan 12, 2027
(currently open for review)
Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.
On-Device Dietary Self-Monitoring With Open-Weight Vision–Language Models: Model Selection and Evaluation on Three Public Datasets
ABSTRACT
Background:
Vision–language models (VLMs) can estimate the energy of a meal from one photograph, but the usual arrangement sends the photograph to an external service, and a continuous record of meals is personal information in its own right. Small open-weight VLMs can now run on personal devices, but how the design choices of such a system affect accuracy has not been examined systematically on public data.
Objective:
We aimed to develop PLANS (Personal-device Local Assessment of Nutrients System), in which a small open-weight VLM runs entirely on a personal device so that a person can record their own energy intake without submitting photographs or dietary data to any network-based service, and to examine how the parameter count of the model, 4-bit quantization, the execution platform and the food weight supplied by the user affect accuracy, with current commercial models as a reference.
Methods:
Fifteen open-weight VLMs (2–32 billion parameters) were compared on a server on three public image sets with reference values (SNAPMe, ACETADA, Nutrition5k; 2,868 images). Qwen3-VL 2B, 4B and 8B at 4 bits were run on a desktop computer and the 4B model on a tablet without network access. Energy was evaluated by median absolute percentage error (MedAPE); the food weight supplied as text and six commercial models (as a reference) were also examined.
Results:
At 4 bits the 4B model gave MedAPE of 36.1%, 38.0% and 43.8% on the three sets: a few points above the commercial models on SNAPMe, within their range on Nutrition5k, and below all of them on the multi-item ACETADA trays, where 8B (23.9%) was closer. Tablet and desktop estimates were identical for 86–89% of images on the three sets. Supplying the true food weight as text lowered MedAPE of the 4-bit 4B model to 22.4% on ACETADA and 40.0% on Nutrition5k, whereas overstating the weight by 25% or adding 150 g (an unsubtracted container weight) worsened it on Nutrition5k (48.8% and 69.0%); a same-stack BF16 control showed that the loss of accuracy at 4 bits on Nutrition5k was due to quantization rather than to the execution stack.
Conclusions:
Fully local estimation with a small open-weight VLM is feasible on a tablet, but accuracy depends on the image set and the execution configuration, and weight input does not help uniformly. The 4B model at 4 bits was selected as the basic configuration for on-device dietary self-monitoring.
Citation
Request queued. Please wait while the file is being generated. It may take some time.
Copyright
© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.