Three-Dimensional Shape Reconstruction and Total Weight Estimation of a Mixed Dish Using RGB–ToF Depth Fusion - A Study of Bulgogi -
The accurate quantification of food intake is important for objective dietary assessment and nutritional management. This study evaluates an Red–Green–Blue (RGB) time-of-flight (ToF) depth-fusion framework for estimating the total weight of Bulgogi, a mixed Korean dish. Seventy samples of Bulgogi were prepared using two ratios of beef-to-vegetables (50:50 and 80:20) and three total-weight ratios (100, 200, and 300 g). RGB and depth data were acquired using three RGB–ToF cameras and processed using geometric alignment, food-region masking, plate reference-depth estimation, height-map generation, apparent volume calculation, multi-camera aggregation, and regression-based calibration. The model performance was assessed using sample-level nested cross-validation with outer five-fold evaluation and inner four-fold tuning. Camera-derived information from each physical sample was aggregated prior to cross-validation to prevent data leakage. The mean food height was strongly associated with total weight measured (R2=0.9342), whereas the maximum height showed a lower explanatory power (R2=0.7634). In held-out out-of-fold predictions, the baseline volume–density model achieved an mean absolute error (MAE) of 15.03 g, root mean squared error (RMSE) of 17.62 g, mean absolute percentage error (MAPE) of 10.98%, and R2 of 0.9553. The calibrated regression model improved performance to an MAE of 6.70 g, RMSE of 9.36 g, MAPE of 3.58%, and R2 of 0.9874. These findings support the feasibility of RGB–ToF depth fusion for total weight estimation of mixed food under the tested conditions.