Maintenance Notice

Due to necessary scheduled maintenance, the JMIR Publications website will be unavailable from Wednesday, July 01, 2020 at 8:00 PM to 10:00 PM EST. We apologize in advance for any inconvenience this may cause you.

Who will be affected?

Currently submitted to: JMIR Biomedical Engineering

Date Submitted: Nov 4, 2026
Open Peer Review Period: Nov 17, 2026 - Jan 12, 2027
(currently open for review)

Warning: This is an author submission that is not peer-reviewed or edited. Preprints - unless they show as "accepted" - should not be relied on to guide clinical practice or health-related behavior and should not be reported in news media as established information.

On-Device Dietary Self-Monitoring With Open-Weight Vision–Language Models: Model Selection and Evaluation on Three Public Datasets

  • Shinichi Nakagawa; 
  • Akira Yamamoto

ABSTRACT

Background:

Vision–language models (VLMs) can estimate the energy of a meal from one photograph, but the usual arrangement sends the photograph to an external service, and a continuous record of meals is personal information in its own right. Small open-weight VLMs can now run on personal devices, but how the design choices of such a system affect accuracy has not been examined systematically on public data.

Objective:

We aimed to develop PLANS (Personal-device Local Assessment of Nutrients System), in which a small open-weight VLM runs entirely on a personal device so that a person can record their own energy intake without submitting photographs or dietary data to any network-based service, and to examine how the parameter count of the model, 4-bit quantization, the execution platform and the food weight supplied by the user affect accuracy, with current commercial models as a reference.

Methods:

Fifteen open-weight VLMs (2–32 billion parameters) were compared on a server on three public image sets with reference values (SNAPMe, ACETADA, Nutrition5k; 2,868 images). Qwen3-VL 2B, 4B and 8B at 4 bits were run on a desktop computer and the 4B model on a tablet without network access. Energy was evaluated by median absolute percentage error (MedAPE); the food weight supplied as text and six commercial models (as a reference) were also examined.

Results:

At 4 bits the 4B model gave MedAPE of 36.1%, 38.0% and 43.8% on the three sets: a few points above the commercial models on SNAPMe, within their range on Nutrition5k, and below all of them on the multi-item ACETADA trays, where 8B (23.9%) was closer. Tablet and desktop estimates were identical for 86–89% of images on the three sets. Supplying the true food weight as text lowered MedAPE of the 4-bit 4B model to 22.4% on ACETADA and 40.0% on Nutrition5k, whereas overstating the weight by 25% or adding 150 g (an unsubtracted container weight) worsened it on Nutrition5k (48.8% and 69.0%); a same-stack BF16 control showed that the loss of accuracy at 4 bits on Nutrition5k was due to quantization rather than to the execution stack.

Conclusions:

Fully local estimation with a small open-weight VLM is feasible on a tablet, but accuracy depends on the image set and the execution configuration, and weight input does not help uniformly. The 4B model at 4 bits was selected as the basic configuration for on-device dietary self-monitoring.


 Citation

Please cite as:

Nakagawa S, Yamamoto A

On-Device Dietary Self-Monitoring With Open-Weight Vision–Language Models: Model Selection and Evaluation on Three Public Datasets

JMIR Preprints. 04/11/2026:116178

DOI: 10.2196/preprints.116178

URL: https://preprints.jmir.org/preprint/116178

Download PDF


Request queued. Please wait while the file is being generated. It may take some time.

© The authors. All rights reserved. This is a privileged document currently under peer-review/community review (or an accepted/rejected manuscript). Authors have provided JMIR Publications with an exclusive license to publish this preprint on it's website for review and ahead-of-print citation purposes only. While the final peer-reviewed paper may be licensed under a cc-by license on publication, at this stage authors and publisher expressively prohibit redistribution of this draft paper other than for review purposes.