MealMeter: Using Multimodal Sensing and Machine Learning for Automatically Estimating Nutrition Intake
Asiful Arefeen, Samantha Fessler, Sayyed Mostafa Mostafavi, Carol S Johnston, Hassan Ghasemzadeh · 2025
Read MealMeter: Using Multimodal Sensing and Machine Learning for Automatically Estimating Nutrition Intake by Asiful Arefeen, Samantha Fessler, Sayyed Mostafa Mostafavi, Carol S Johnston, Hassan Ghasemzadeh (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-11683.txt.
Introduction
Asiful Arefeen, Samantha Fessler, Sayyed Mostafa Mostafavi, Carol Johnston, and Hassan Ghasemzadeh built a meal-tracking system in 2025 that estimates nutrition from body signals. Food logs are slow and error-prone, yet precise intake matters for diabetes and weight care. Their system, MealMeter, reads wearables and phones instead.
Twelve participants supplied glucose traces, heart rhythm variation, motion data, and context cues around labelled meals. Light learning models turned those signals into carbohydrate, protein, and fat estimates. The combined sensing beat baselines, including a large foundation model, with carbohydrate error near thirteen grams on average. The authors see automated tracking as support for personal nutrition plans and metabolic care.
Catalog record
| CREATOR | Asiful Arefeen, Samantha Fessler, Sayyed Mostafa Mostafavi, Carol S Johnston, Hassan Ghasemzadeh |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 19 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-11683
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
MealMeter: Using Multimodal Sensing and Machine Learning for Automatically Estimating Nutrition Intake Asiful Arefeen, Samantha Fessler, Sayyed Mostafa Mostafavi, Carol S Johnston, Hassan Ghasemzadeh Abstract Accurate estimation of meal macronutrient composition is a pre-perquisite for precision nutrition, metabolic health monitoring, and glycemic management. Traditional dietary assessment methods, such as self-reported food logs or diet recalls are time-intensive and prone to inaccuracies and biases. Several existing AI-driven frameworks are data intensive. In this study, we propose MealMeter, a machine learning driven method that leverages multimodal sensor data of wearable and mobile devices. Data are collected from 12 12 participants to estimate macronutrient intake. Our approach integrates physiological signals (e.g., continuous glucose, heart rate variability), inertial motion data, and environmental cues to model the relationship between meal intake and metabolic responses. Using lightweight machine learning models trained on a diverse dataset of labeled meal events, MealMeter predicts the composition of carbohydrates, proteins, and fats with high accuracy. Our results demonstrate that multimodal sensing combined with machine learning significantly improves meal macronutrient estimation compared to the baselines including foundation model and achieves average mean absolute errors (MAE) and average root mean squared relative errors (RMSRE) as low as 13.2 13.2 grams and 0.37 0.37 , respectively, for carbohydrates. Therefore, our developed system has the potential to automate meal tracking, enhance dietary interventions, and support personalized nutrition strategies for individuals managing metabolic disorders such as diabetes and obesity. Clinical relevance — MealMeter is a clinically relevant tool for precision nutrition, glycemic management and obesity management because it offers accurate meal tracking through multimodal sensing and machine learning. I Introduction Fig. 1: Pipeline: MealMeter performs multi-modal fusion using data from Dexcom G6 CGM and Empatica E4 followed by feature engineering and light-weight model training for dietary assessment. Managing obesity, diabetes and cardiovascular disease (CVD) remains a continuous challenge for Americans. The age-adjusted prevalence of obesity among U.S. adults was 41.8%, and severe obesity registered at 9.2 according to prepandemic national survey data collected from 2017 to 2020 [ 1 ] . Obesity is closely associated with other serious chronic health conditions; for instance, almost 23% of U.S. adults with obesity have diabetes while 58% have high blood pressure, which is a major risk factor for CVDs [ 1 ] . Addressing the root cause—obesity—requires a proactive approach to controlling consumption patterns and adopting healthy lifestyles [ 2 ] . Meal monitoring facilitates the management and prevention the aforementioned chronic diseases. Accurate tracking of dietary intake enables individuals and healthcare providers to make informed decisions about self-nutrition, which is vital for weight management, blood sugar control and reduction of CVD risk factors. Traditional dietary assessment techniques like self-reported food diaries are prone to inaccuracies due to inaccurate memory retention and subjective reporting [ 3 ] . However, advanced digital health technologies, including wearable devices and artificial intelligence (AI), offer innovative solutions for more precise and real-time dietary monitoring and to reduce laborious food logging [ 4 ] . Existing wearable technologies record physiological signals including heart rate variability, skin temperature and physical activity levels. Although these metrics offer indirect insights into metabolic responses to food intake, exact tracking of protein and fat consumption has remained challenging to date. Recent research explored the feasibility of advanced sensors and AI to estimate macronutrient intake by analyzing patterns in physiological data [ 4 ] . Continuous glucose monitors (CGMs), for instance, can provide insights into blood sugar spikes due to carbohydrate intakes [ 5 ] . High intakes of fat and high glycemic carbohydrates reduce heart rate variability [ 6 ] . Prior research [ 7 ] aimed to develop sophisticated methods to assess the impact of various macronutrients on physiological parameters, which could eventually lead to more precise dietary monitoring tools through wearable sensing. Recent research focused on leveraging AI and wearable devices to enhance dietary assessment accuracy, predict abnormal health events and provide interventions [ 8 , 9 , 10 , 11 ] . Zhang et al. [ 12 ] created embedding from food photographs using vision transformers, fused them with corresponding glycemic response embeddings and fed these patches to a regression model for calorie counting. Shao et al. [ 13 ] used Red, Green, Blue-Depth of food images for precise estimation of portion sizes. Speech2Health [ 14 ] used Natural Language Processing (NLP) techniques to convert spoken language to text and search the user-reported food name in a reference nutrition database to compute calorie intake. In another study, a five-point Gaussian filter was applied and area under the curve (AUC) was computed to capture the trend of the post-meal glycemic response [ 15 ] . Later, this trend was fed to a multi-task neural network for carbohydrate, protein and fat estimation. Several techniques have been also proposed to accurately detect eating events leveraging sensor systems, although these techniques could not provide solutions for meal tracking, [ 16 , 17 , 18 ] . MealMeter is the first of its kind to integrate CGM and wristband-based multimodal sensing (Figure 1 ) to analyze the relationship between meal macronutrients and physiological signals for dietary assessment. Leveraging this synergy with computationally simple machine learning, MealMeter offers a fast, comprehensive and data-driven approach to tracking dietary intake and metabolic response. Therefore, contributions made by MealMeter can be summarized as follows- • For the purpose of developing MealMeter, we collected data from 12 healthy individuals equipped with CGM sensor and Empatica E4 to capture metabolic response to customized meals for three non-consecutive days. • We used the data from the user study to train lightweight regression models to map meal macronutrients from physiological signals and explore associations among these parameters. II Methods II-A Data Collection Fig. 2: Plot shows collected CGM signal from a subject as a response to the food items fed at different timestamps. It also shows 90-minute long pre- and post-meal windows as inputs to the pipeline. Data were collected from an ongoing trial where we recruited 12 healthy adults aged 23-31 years with BMI 20.08-31.84 to record device-derived responses to standardized meals. Participants had no diagnosed chronic metabolic or thyroid conditions, did not use recreational drugs, limited alcohol consumption to no more than two servings per day, and were not involved in competitive athletics or resistance training. All participants provided written consent to the study. The trial was approved by the Institutional Review Board at Arizona State University (IRB #15102). This study had three non-consecutive laboratory monitoring days. Upon enrollment, participants were equipped with wearable devices, including the Dexcom G6 CGM and the Empatica E4 wristband worn on their dominant arm. The CGM device collected blood glucose concentrations (BGL) at 5-minute intervals while the E4 recorded 3-axis acceleration (Acc), electrodermal activity (EDA), heart rate (HR), blood volume pulse (BVP) and skin temperature (TEMP) at different sampling frequencies ranging from 4Hz to 64Hz. Trained research personnel provided instructions on device usage and study study-related dos and don’ts. During the three 10-hour laboratory monitoring sessions (8:00 AM–6:00 PM), participants arrived after a 12-hour overnight fast and received customized meals tailored to their resting energy requirements, calculated using the Mifflin-St Jeor equation [ 19 ] . Meals were classified as hypercaloric, eucaloric, or hypocaloric and consisted of commercial foods and