Large Language Models for Ingredient Substitution in Food Recipes using Supervised Fine-tuning and Direct Preference Optimization

Thevin Senath, Kumuthu Athukorala, Ransika Costa, Surangika Ranathunga, Rishemjit Kaur · 2024

Read Large Language Models for Ingredient Substitution in Food Recipes using Supervised Fine-tuning and Direct Preference Optimization by Thevin Senath, Kumuthu Athukorala, Ransika Costa, Surangika Ranathunga, Rishemjit Kaur (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-04922.txt.

Introduction

Thevin Senath, Kumuthu Athukorala, Ransika Costa, Surangika Ranathunga, and Rishemjit Kaur use large language models to swap ingredients in recipes. Their paper from twenty twenty-four targets cooking tailored to taste, health, or what is on hand. Model use for this task had barely been tried. The team ran wide tests to pick the best model, prompt, and tuning plan.

Extra tricks were tried too: multi-task learning, two-stage tuning, and a preference-based polish step. Tests ran on the public Recipe1MSub corpus. A fine-tuned Mistral base model, polished by preference training, won. Its top-hit score beat the strong prior baseline for the corpus. The authors frame the gain as a step toward personal, creative cooking aids. Swaps stayed plausible within each dish's context.

Catalog record

CREATORThevin Senath, Kumuthu Athukorala, Ransika Costa, Surangika Ranathunga, Rishemjit Kaur
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE31 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-04922

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Large Language Models for Ingredient Substitution in Food Recipes using Supervised Fine-tuning and Direct Preference Optimization Thevin Senath, Kumuthu Athukorala, Ransika Costa, Surangika Ranathunga, Rishemjit Kaur Abstract In this paper, we address the challenge of recipe personalization through ingredient substitution. We make use of Large Language Models (LLMs) to build an ingredient substitution system designed to predict plausible substitute ingredients within a given recipe context. Given that the use of LLMs for this task has been barely done, we carry out an extensive set of experiments to determine the best LLM, prompt, and the fine-tuning setups. We further experiment with methods such as multi-task learning, two-stage fine-tuning, and Direct Preference Optimization (DPO). The experiments are conducted using the publicly available Recipe1MSub corpus. The best results are produced by the Mistral7-Base LLM after fine-tuning and DPO. This result outperforms the strong baseline available for the same corpus with a Hit@1 score of 22.04. Thus we believe that this research represents a significant step towards enabling personalized and creative culinary experiences by utilizing LLM-based ingredient substitution. keywords Natural Language Processing, Parameter-Efficient Fine-Tuning, Multi-Task Learning, Two-Stage Fine-Tuning, Direct Preference Optimization, recipe personaization, Mistral 1 Introduction Food recipes from diverse cultures and regions are spreading rapidly on the Internet, igniting curiosity to explore new culinary experiences. This exposure to various cuisines broadens our culinary horizons and encourages experimentation with different dishes at home. However, preparing new recipes often involves personalizing them to accommodate ingredient availability, cost, dietary restrictions, lifestyle choices, allergies, or personal taste preferences. This customization typically involves substituting certain ingredients with alternatives. Yet, finding suitable substitutes that maintain the intended flavor profile can be challenging, as the effectiveness of substitutions varies depending on the recipe and context. For instance, while oil may work well as a substitute for butter in frying chicken, it might not yield the same results in cake baking. Personalizing recipes through ingredient substitution has the potential to enhance culinary experiences by accommodating factors such as cost, availability, and dietary needs. Past research has experimented with various Natural Language Processing (NLP) techniques for this task, and several pre-trained Language Models (PLMs) based on Word2Vec Mikolov 2013 and BERT Devlin et al. 2019 have been proposed Pellegrini et al. 2021 ; Lawo et al. 2020 . Fatemi et al. 2023 ’s pioneering work Graph-based Ingredient Substitution Module (GISMo) uses recipe-specific context and generic ingredient relational information encoded as a graph simultaneously to predict suitable substitutions. Large Language Models (LLMs), combined with advanced prompting techniques, have emerged as the current de-facto solution for NLP tasks in various domains. However, their application in ingredient substitution remains under-explored, and we are aware of only one very recent work Rita et al. 2024 . Therefore, this paper aims to implement an LLM-based ingredient substitution solution that considers the recipe’s context for accurate ingredient replacement. Given that LLMs have only used once for the task of ingredient substitution before, we first carry out a set of experiments to determine the best publicly available LLM and the best prompt, using zero-shot and few-shot testing. Experiments are conducted using the publicly available Recipe1MSub Fatemi et al. 2023 benchmark dataset. Mistral7B-base model emerged as the best candidate for the task. Subsequently, we perform supervised fine-tuning (SFT) of this selected LLM. We experiment with multiple Parameter Efficient Fine-tuning (PEFT) techniques and identify QLoRA to be the best PEFT technique. We carry out further experiments using two-stage fine-tuning, multi-task fine-tuning, as well as Direct Preference Optimization (DPO) techniques. Our best model that employs both SFT and DPO outperforms the strong baseline of GISMO, with a Hit@1 score of 22.04. To summarize, the contributions of this paper are as follows: • Investigation into the best LLM prompting patterns for ingredient substitution and introduction of a new prompt that effectively makes use of the information in the recipe. • Extensive experiments to determine the best LLM as well as optimization strategies for the ingredient substitution task. • Presents a system that incorporate SFT and DPO, for the ingredient substitution task The rest of the paper is organized as follows. Section 2 discusses past work related to ingredient substitution and LLMs. Section 3 discusses our methodology. This includes a detailed discussion on the prompt development, LLM fine-tuning as well as performance optimization strategies. Section 4 presents the dataset and the experiment setup, and section 5 presents the results. Finally, section 6 concludes the paper with limitations of the research and a look into future research directions. 2 Related work 2.1 Ingredient Substitution Most of the existing approaches for ingredient substitution are based on textual recipe information and statistics Boscarino et al. 2014 ; Achananuparp and Weber 2016 . In recent years, researchers have started to explore NLP techniques to address this ingredient substitution task using PLMs such as BERT, R-BERT Wu and He 2019 and Word2Vec. Pellegrini et al. 2021 introduced two PLMs, namely food2vec and foodBERT, to obtain ingredient embeddings from recipe instructions and user comments. These learned embeddings were applied to the context of ingredient substitution in dietary use cases. Lawo et al. 2020 introduced ingredient2Vec, a Word2Vec-based model, to develop a neural network model for suggesting vegan substitutes for omnivorous ingredients. A recent approach to address this task is the Graph-based Ingredient Substitution Module (GISMo) Fatemi et al. 2023 , which uses recipe-specific context and generic ingredient relational information encoded as a graph simultaneously to predict suitable substitutions. GISMo uses recent advancements in graph neural networks (GNNs) to capture interactions between ingredients within learned ingredient embeddings. These embeddings are then refined with contextual information from a given recipe, enabling the prediction of plausible ingredient substitutions. Additionally, the authors have introduced a benchmark known as Recipe1MSubs, featuring substitution pairs linked to the Recipe1M dataset Salvador et al. 2017 , with standardized splits, evaluation metrics, and baselines. Some research has used LLMs and Vision-Language pre-trained models for recipe generation and retrieval Menon et al. 2024 ; Wahed et al. 2024 ; Mohbat and Zaki 2024 ; Kamatchi S et al. 2024 . Parallel to our work, Rita et al. 2024 experimented with LLMs for the ingredient substitution task. However, they have only employed SFT, and there has been no study on identifying the most effective prompt pattern or the recipe context. 2.2 Large Language Models Introduction Pre-trained Language Models (PLMs) derived by training the Transformer neural architecture Vaswani 2017 with large corpora have revolutionized the field of NLP. These PLMs can be either encoder-based, decoder-based or encoder-decoder based. Out of these, decoder-based PLMs have gained prominence with the success of commercial products such as ChatGPT. Recent decoder-based models have been trained using much larger corpora compared to the early-day PLMs, thus earning them the term LLMs. Llama Touvron et al. , Mistral Jiang et al. 2023 and Gemma GemmaTeam et al. 2024 are some of the commonly used publicly available LLMs that have shown comparable performance to their commercial counterparts. These LLMs are trained using large amounts of raw text crawled from the web, using the next word prediction self-supervised learning objective. This training process is termed pre-training , and the resulting models are referred to as base models. In general, we can represent a base model consisting of M parameters where

Read the complete text