Analysis & Numerical Simulation of Indian Food Image Classification Using Convolutional Neural Network
Main Article Content
Abstract
Recognition of Indian food can be assumed to be a fine-grained visual task owing to recognition property of various food classes. It is therefore important to provide an optimized approach to segmentation and classification for different applications based on food recognition. Food computation mainly utilizes a computer science approach which needs food data from various data outlets like real-time images, social flat-forms, food journaling, food datasets etc, for different modalities. In order to consider Indian food images for a number of applications we need a proper analysis of food images with state-of-art-techniques. The appropriate segmentation and classification methods are required to forecast the relevant and upgraded analysis. As accurate segmentation lead to proper recognition and identification, in essence we have considered segmentation of food items from images. Considering the basic convolution neural network (CNN) model, there are edge and shape constraints that influence the outcome of segmentation on the edge side. Approaches that can solve the problem of edges need to be developed; an edge-adaptive As we have solved the problem of food segmentation with CNN, we also have difficulty in classifying food, which has been an important area for various types of applications. Food analysis is the primary component of health-related applications and is needed in our day to day life. It has the proficiency to directly predict the score function from image pixels, input layer to produce the tensor outputs and convolution layer is used for self- learning kernel through back-propagation. In this method, feature extraction and Max-Pooling is considered with multiple layers, and outputs are obtained using softmax functionality. The proposed implementation tests 92.89% accuracy by considering some data from yummly dataset and by own prepared dataset. Consequently, it is seen that some more improvement is needed in food image classification. We therefore consider the segmented feature of EA-CNN and concatenated it with the feature of our custom Inception-V3 to provide an optimized classification. It enhances the capacity of important features for further classification process. In extension we have considered south Indian food classes, with our own collected food image dataset and got 96.27% accuracy. The obtained accuracy for the considered dataset is very well in comparison with our foregoing method and state-of-the-art techniques..
Downloads
Article Details

This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
References
Giovanni Maria Farinella, Dario Allegra, F. Stanco, A Benchmark Dataset to Study the Representation of Food Images, Workshop on Assistive Computer Vision and Robotics (ACVR 2014) Volume: 8927, September 2015, pp 584-599, DOI: 10.1007/978-3-319-16199-0_41.
GwoGiun Lee; Chin-Wei Huang; Jia-Hong Chen; Shih-Yu Chen; Hsiu-Ling Chen, AIFood: A Large Scale Food Images Dataset for Ingredient Recognition, IEEE Region 10 Conference (TENCON), 2019, DOI: 10.1109/TENCON.2019.8929715
XinChen , Yu Zhu , Hua Zhou, Liang Diao, and Dongyan Wang , ChineseFoodNet: A Large-scale Image Dataset for Chinese Food Recognition, arXiv:1705.02743v3 [cs.CV] 15 Oct 2017.
Y. He, N. Khanna, C. J. Boushey and E. J. Delp, Snakes assisted food image segmentation, 2012 IEEE 14th International Workshop on Multimedia Signal Processing (MMSP), Banff, AB, 2012, pp. 181-185, doi: 10.1109/MMSP.2012.6343437.
Y. Wang, C. Liu, F. Zhu, C. J. Boushey and E. J. Delp, Efficient superpixel based segmentation for food image analysis, 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, 2016, pp. 2544-2548, doi: 10.1109/ICIP.2016.7532818.
M. Anthimopoulos, J. Dehais, P. Diem and S. Mougiakakou, Segmentation and recognition of multi-food meal images for carbohydrate counting, 13th IEEE International Conference on BioInformatics and BioEngineering, Chania, 2013, pp. 1-4, doi: 10.1109/BIBE.2013.6701608.
Y. He, C. Xu, N. Khanna, C. J. Boushey and E. J. Delp, Analysis of food images: Features and classification, 2014 IEEE International Conference on Image Processing (ICIP), Paris, 2014, pp. 2744-2748, doi: 10.1109/ICIP.2014.7025555.
Y. He, N. Khanna, C. J. Boushey and E. J. Delp, Image segmentation for image-based dietary assessment: A comparative study, International Symposium on Signals, Circuits and Systems ISSCS2013, Iasi, 2013, pp. 1-4, doi: 10.1109/ISSCS.2013.6651268.
S. Aslan, G. Ciocca and R. Schettini, Semantic segmentation of food images for automatic dietary monitoring, 2018 26th Signal Processing and Communications Applications Conference (SIU), Izmir, 2018, pp. 1-4, doi: 10.1109/SIU.2018.8404824.
Y. He, C. Xu, N. Khanna, C. J. Boushey and E. J. Delp, Food image analysis: Segmentation, identification and weight estimation, 2013 IEEE International Conference on Multimedia and Expo (ICME), San Jose, CA, 2013, pp. 1-6, doi: 10.1109/ICME.2013.6607548.
M. Bolaños and P. Radeva, Simultaneous food localization and recognition, 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun, 2016, pp. 3140-3145, doi: 10.1109/ICPR.2016.7900117.
StergiosChristodoulidis, MariosAnthimopoulos and StavroulaMougiakakou, Food Recognition for Dietary Assessment Using Deep Convolutional Neural Networks, ICIAP 2015: New Trends in Image Analysis and Processing -- ICIAP 2015 Workshops pp 458- 465
Joachim Dehais, MariosAnthimopoulos and StavroulaMougiakakou, Dish Detection and Segmentation for Dietary Assessment on Smartphones, ICIAP 2015: New Trends in Image Analysis and Processing -- ICIAP 2015 Workshops pp 433-440
J. Lee, A. Banerjee and S. K. S. Gupta, MT-Diet: Automated smartphone based diet assessment with infrared images, 2016 IEEE International Conference on Pervasive Computing and Communications (PerCom), Sydney, NSW, 2016, pp. 1-6, doi: 10.1109/PERCOM.2016.7456506.
ElnazJahani, Heravi H.H and Aghdam D.P, An optimized convolutional neural network with bottleneck and spatial pyramid pooling layers for classification of foods Pattern Recognition Letters, Volume 105, 1 April 2018, Pages 50-58
B. S. Anami, Dayanand G. Savakar,Effect of Foreign Bodies on Recognition and Classification of Bulk Food Grains Image Samples, Journal of Applied Computer Science andMathematics, Volume 3(6), Pages: 77-83, 2019.
S. J. Minija and W. R. S. Emmanuel, Food image classification using sphere shaped Support vector machine, 2017 International Conference on Inventive Computing and Informatics (ICICI), Coimbatore, 2017, pp. 109-113, doi: 10.1109/ICICI.2017.8365314.
Aslan S., Ciocca G., Schettini R. (2017) On Comparing Color Spaces for Food Segmentation. In: Battiato S., Farinella G., Leo M., Gallo G. (eds) New Trends in Image Analysis and Processing – ICIAP 2017. ICIAP 2017.
Knez S., Šajn L. (2015) Food Object Recognition Using a Mobile Device: State of the Art. In: Murino V., Puppo E., Sona D., Cristani M., Sansone C. (eds) New Trends in Image Analysis and Processing -- ICIAP 2015 Workshops. ICIAP 2015.
R. Dinic and T. Stütz, EatAR Tango: Results on the Accuracy of Portion Estimation, 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR- Adjunct), Nantes, 2017, pp. 284-287, doi: 10.1109/ISMAR-Adjunct.2017.90.
J. Sun, K. Radecka and Z. Zilic, Exploring Better Food Detection via Transfer Learning, 2019 16th International Conference on Machine Vision Applications (MVA), Tokyo, Japan, 2019, pp. 1-6, doi: 10.23919/MVA.2019.8757886.
G. Su, D. Li, Y. Wang, L. Wang and M. Zhang, Chinese Dish Segmentation Based on Local Variation Driven Superpixel Grouping and Region Analysis, 2018 International Conference on Audio, Language and Image Processing (ICALIP), Shanghai, 2018, pp. 41-45, doi: 10.1109/ICALIP.2018.8455779.
M. Chiang, C. Wu, J. Feng, C. Fang and S. Chen, Food Calorie and Nutrition Analysis System based on Mask R-CNN, 2019 IEEE 5th International Conference on Computer and Communications (ICCC), Chengdu, China, 2019, pp. 1721-1728, doi: 10.1109/ICCC47050.2019.9064257.
F. Konstantakopoulos, E. I. Georga, K. Klampanas, D. Rouvalis, N. Ioannou and D. I. Fotiadis, Automatic Estimation of the Nutritional Composition of Foods as Part of the GlucoseML Type 1 Diabetes Self-Management System, 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE), Athens, Greece, 2019, pp. 470-473, doi: 10.1109/BIBE.2019.00091.
Wang X., Rohani N., Manerikar A., Katsagellos A., Cossairt O., Alshurafa N. (2017) Distinguishing Nigerian Food Items and Calorie Content with Hyperspectral Imaging. In: Battiato S., Farinella G., Leo M., Gallo G. (eds) New Trends in Image Analysis and Processing – ICIAP 2017. ICIAP 2017.
Y. Lu, T. Stathopoulou, M. F. Vasiloglou, S. Christodoulidis, Z. Stanga and S. Mougiakakou, An Artificial Intelligence-Based System to Assess Nutrient Intake for Hospitalised Patients, IEEE Transactions on Multimedia, doi: 10.1109/TMM.2020.2993948.
Aslan, S., Ciocca, G., Mazzini, D. et al. Benchmarking algorithms for food localization and semantic segmentation. Int. J. Mach. Learn. & Cyber. (2020).
Y. Wang, F. Zhu, C. J. Boushey and E. J. Delp, Weakly supervised food image segmentation using class activation maps, 2017 IEEE International Conference on Image Processing (ICIP), Beijing, 2017, pp. 1277-1281, doi: 10.1109/ICIP.2017.8296487.
P. Pouladzadeh, S. Shirmohammadi and A. Yassine, Using graph cut segmentation for food calorie measurement, 2014 IEEE International Symposium on Medical Measurements and Applications (MeMeA), Lisboa, 2014, pp. 1-6, doi: 10.1109/MeMeA.2014.6860137.
P. Pouladzadeh, P. Kuhad, S. V. B. Peddi, A. Yassine and S. Shirmohammadi, Food calorie measurement using deep learning neural network, 2016 IEEE International Instrumentation and Measurement Technology Conference Proceedings, Taipei, 2016, pp. 1-6, doi: 10.1109/I2MTC.2016.7520547.
Sri Vijay Bharat Peddi, PallaviKuhad and AbdulsalamYassine, An intelligent cloud- based data processing broker for mobile e-health multimedia applications, Future Generation Computer Systems, Volume 66, January 2017, Pages 71-86
Pouladzadeh P., Yassine A., Shirmohammadi S. (2015) FooDD: Food Detection Dataset for Calorie Measurement Using Food Images. In: Murino V., Puppo E., Sona D., Cristani M., Sansone C. (eds) New Trends in Image Analysis and Processing -- ICIAP 2015 Workshops. ICIAP 2015.
P. Kuhad, A. Yassine and S. Shimohammadi, Using distance estimation and deep learning to simplify calibration in food calorie measurement, 2015 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA), Shenzhen, 2015, pp. 1-6, doi: 10.1109/CIVEMSA.2015.7158594.
M. T. Islam, B. M. N. KarimSiddique, S. Rahman and T. Jabid, Food Image Classification with Convolutional Neural Network, 2018 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), Bangkok, 2018, pp. 257- 262, doi: 10.1109/ICIIBMS.2018.8550005.
A. Metwalli, W. Shen and C. Q. Wu, Food Image Recognition Based on Densely Connected Convolutional Neural Networks, 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Fukuoka, Japan, 2020, pp. 027-032, doi: 10.1109/ICAIIC48513.2020.9065281.
Q. Yu, M. Anzawa, S. Amano, M. Ogawa and K. Aizawa, Food Image Recognition by Personalized Classifier, 2018 25th IEEE International Conference on Image Processing (ICIP), Athens, 2018, pp. 171-175, doi: 10.1109/ICIP.2018.8451422.
A. ?engür, Y. Akbulut and Ü. Budak, Food Image Classification with Deep Features, 2019 International Artificial Intelligence and Data Processing Symposium (IDAP), Malatya, Turkey, 2019, pp. 1-6, doi: 10.1109/IDAP.2019.8875946.
J. R. Rajayogi, G. Manjunath and G. Shobha, Indian Food Image Classification with Transfer Learning, 2019 4th International Conference on Computational Systems and Information Technology for Sustainable Solution (CSITSS), Bengaluru, India, 2019, pp. 1-4, doi: 10.1109/CSITSS47250.2019.9031051.
G. Ciocca, P. Napoletano and R. Schettini, Food Recognition: A New Dataset, Experiments, and Results, in IEEE Journal of Biomedical and Health Informatics, vol. 21, no. 3, pp. 588-598, May 2017, doi: 10.1109/JBHI.2016.2636441.
Giovanni Maria Farinell , Dario Allegra and, Marco Moltisanti, ?Retrieval and classification of food images? Computers in Biology and Medicine, Volume 77, 1 October 2016, Pages 23-39
Kagaya H., Aizawa K. (2015) Highly Accurate Food/Non-Food Image Classification Based on a Deep Convolutional Neural Network. In: Murino V., Puppo E., Sona D., Cristani M., Sansone C. (eds) New Trends in Image Analysis and Processing -- ICIAP 2015 Workshops. ICIAP 2015.
L. Herranz, S. Jiang and R. Xu, Modeling Restaurant Context for Food Recognition, in IEEE Transactions on Multimedia, vol. 19, no. 2, pp. 430-440, Feb. 2017, doi: 10.1109/TMM.2016.2614861.
GianluigiCiocca, Paolo Napoletano, and RaimondoSchettini, CNN-based features for retrieval and classification of food images, Computer Vision and Image Understanding, Volumes 176–177, November–December 2018, Pages 70-77
D. J. Attokaren, I. G. Fernandes, A. Sriram, Y. V. S. Murthy and S. G. Koolagudi, Food classification from images using convolutional neural networks, TENCON 2017 - 2017 IEEE Region 10 Conference, Penang, 2017, pp. 2801-2806, doi: 10.1109/TENCON.2017.8228338.
S. Horiguchi, S. Amano, M. Ogawa and K. Aizawa, Personalized Classifier for Food Image Recognition, in IEEE Transactions on Multimedia, vol. 20, no. 10, pp. 2836-2848, Oct. 2018, doi: 10.1109/TMM.2018.2814339.
M. T. Islam, B. M. N. KarimSiddique, S. Rahman and T. Jabid, Image Recognition with Deep Learning, 2018 International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS), Bangkok, 2018, pp. 106-110, doi: 10.1109/ICIIBMS.2018.8550021.
K. T. Islam, S. Wijewickrema, M. Pervez and S. O'Leary, An Exploration of Deep Transfer Learning for Food Image Classification, 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia, 2018, pp. 1-5, doi: 10.1109/DICTA.2018.8615812.
L. Jiang, B. Qiu, X. Liu, C. Huang and K. Lin, DeepFood: Food Image Analysis and Dietary Assessment via Deep Model, in IEEE Access, vol. 8, pp. 47477-47489, 2020, doi: 10.1109/ACCESS.2020.2973625.
M. A. Subhi and S. Md. Ali, A Deep Convolutional Neural Network for Food Detection and Recognition, 2018 IEEE-EMBS Conference on Biomedical Engineering and Sciences (IECBES), Sarawak, Malaysia, 2018, pp. 284-287, doi: 10.1109/IECBES.2018.8626720.
TaichiJoutou and KeijiYanai, A food image recognition system with Multiple Kernel Learning, 2009 16th IEEE International Conference on Image Processing (ICIP), Cairo, 2009, pp. 285-288, doi: 10.1109/ICIP.2009.5413400.
Alaria, S. K. "A.. Raj, V. Sharma, and V. Kumar.“Simulation and Analysis of Hand Gesture Recognition for Indian Sign Language Using CNN”." International Journal on Recent and Innovation Trends in Computing and Communication 10, no. 4 (2022): 10-14.
Dogiwal, Sanwta Ram, Y. S. Shishodia, Abhay Upadhyaya, Hanuman Ram, and Satish Kumar Alaria. "Image Preprocessing Methods in Image Recognition." International Journal of Computers and Distributed Systems 1, no. 3 (2012): 96-99.