FDA AI Search: Making FDA-Authorized AI Devices Searchable

Arun Kavishwar, William Lotter · 2026

Read FDA AI Search: Making FDA-Authorized AI Devices Searchable by Arun Kavishwar, William Lotter (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-00006.txt.

Introduction

Arun Kavishwar and William Lotter build a search site for AI medical devices cleared by the United States Food and Drug Administration. More than 1200 such devices hold clearance, yet the agency files give thin facts and summaries that cannot be searched. Their tool pulls traits from those summaries with a language model, then matches a clinician query by meaning rather than by key words alone. Checks against key word search favor the new method, and the authors hope it will help both care teams and device makers.

Catalog record

CREATORArun Kavishwar, William Lotter
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE17487
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2602.00006

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

FDA AI Search: Making FDA-Authorized AI Devices Searchable Arun Kavishwar, William Lotter Abstract Over 1,200 AI-enabled medical devices have received marketing authorization from the U.S. FDA, yet identifying devices suited to specific clinical needs remains challenging because the FDA’s databases contain only limited metadata and non-searchable summary PDFs. To address this gap, we developed FDA AI Search , a website that enables semantic querying of FDA-authorized AI-enabled devices. The backend includes an embedding-based retrieval system, where LLM-extracted features from authorization summaries are compared to user queries to find relevant matches. We present quantitative and qualitative evaluation that support the effectiveness of the retrieval algorithm compared to keyword-based methods. As FDA-authorized AI devices become increasingly prevalent and their use cases expand, we envision that the tool will assist healthcare providers in identifying devices aligned with their clinical needs and support developers in formulating novel AI applications. † † workshop: Machine Learning for Health (ML4H) 2025 keywords AI-enabled Medical Devices, FDA Marketing Authorization, Semantic Search, LLMs Data and Code Availability The data used is sourced from the FDA’s database of AI-enabled devices 1 1 1 https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-enabled-medical-devices and accompanying authorization summary documents. The database version used corresponds to an update on July 10th, 2025. Code is available at https://github.com/lotterlab/fda_ai_search . Institutional Review Board (IRB) This research does not require IRB approval. 1 Introduction Over 1,200 AI-enabled medical devices have received marketing authorization from the U.S. Food and Drug Administration (FDA). Realizing their collective potential requires connecting the specific needs of healthcare providers with devices designed to address them. This remains difficult, as the FDA’s database of AI-enabled devices only contains limited metadata (e.g., company name, review panel) and links to non-searchable PDF summaries. While recent efforts have categorized devices along dimensions such as input data type ( 10 ) or regulatory path ( 8 ) or focus on specific use cases ( 3 ; 5 ; 6 ; 7 ) , a more flexible and user-friendly solution is needed. To address this gap, we developed FDA AI Search , a website that enables semantic querying of FDA-authorized AI-enabled devices. Using the public summary statements that accompany FDA marketing authorization, we generate text embeddings for each device and retrieve relevant matches to user queries based on embedding similarity. In addition to introducing the tool and its underlying methodology, we present quantitative and qualitative evaluations compared to traditional keyword-based search methods. 2 Methods The backend of the website consists of an embedding-based retrieval system. Embeddings were generated for each of the 1,247 AI-enabled devices listed by the FDA as of July 10th, 2025 (Figure 1 ). These embeddings were generated from two sources: 1) basic metadata from the FDA’s AI-enabled device database and 2) the public summary accompanying FDA authorization, as described below. Figure 1 : Embedding generation process. a) FDA databases are scraped to retrieve metadata and the summary PDF for each AI device. b) Gemini-2.5-flash is prompted to generate a set of text features from the summary PDF and metadata. c) Each feature is embedded using MedEmbed for semantic search. 2.1 Data Curation The FDA’s list of AI-enabled devices provides basic tabular information for each marketing authorization along with a link to the corresponding entry in the FDA’s 510(k), De Novo, or PMA databases, depending on the regulatory pathway. Each database entry includes a link to the authorization summary PDF, which we retrieved for every device. In rare cases ( \sim 1%), the entire submission (with proprietary information redacted) was also released under the Freedom of Information Act, in which case we used that document rather than the summary alone. 2.2 Feature Extraction Rather than simply creating an embedding for each device in a single shot, embeddings were created in a structured fashion to focus on information likely relevant to user queries. For each device PDF, Gemini-2.5-flash ( 4 ) was initially prompted to extract the following five features: ‘summary’, ‘keywords’, ‘relevant questions’, ‘thesis’ (i.e., concise statement of purpose), and ‘key concepts’. The full prompt and further details of this process are provided in the Appendix. Additional features were then generated from the extracted features to help further align with potential queries. The first generated feature was a ‘search boost’ term consisting of the company name concatenated with the device name and extracted keywords, where the company and device names were extracted from the FDA’s AI database. The other generated features consisted of ‘query match’ terms wherein Gemini-2.5-flash was prompted to generate three distinct search queries that a clinician would use to find this device based on the originally extracted set of five features (see also Appendix). Of the 9 total features (5 original features and 4 generated features), 7 were ultimately chosen to be included in embedding generation: ‘keywords’, ‘relevant questions’, ‘thesis’, ‘search boost’, ‘query match 1’, ‘query match 2’, ‘query match 3’. The ‘summary’ and ‘key concepts’ features were excluded because they were found to be redundant with ‘thesis’ and ‘keywords’, respectively. 2.3 Generating Embeddings Embeddings for each of the 7 features for each device were generated using MedEmbed ( 2 ) , an NLP embedding model specifically trained for medical and clinical data. The small version of MedEmbed was used (MedEmbed-small-v0.1), as it was the only one that created vectors small enough to host with the designed frontend deployment (384 dimensions / vector). This version is also used when creating embeddings of user queries for the retrieval process. 2.4 Retrieval For query q , a device ( d ) score was calculated based on the weighted sum of the cosine similarity between the embedded query ( e_{q} ) and the feature embedding vectors e_{d}^{i} , along with a standard bag-of-words keyword scoring function, BM25 (Equation 1 ). BM25 is a common ranking function used by search engines ( 9 ) and was included to complement the AI-based semantic similarity. The BM25 input for a device was a concatenation of the keywords, relevant questions, thesis, key concepts, and search boost features. The embedding weights ( w_{i} ) in Eq. 1 were calculated via a Bayesian optimization search and the \lambda parameter controlling the trade-off between the embedding similarity and BM25 was calculated via a grid search, as detailed below. \displaystyle\operatorname{Score}(q,d) \displaystyle=\frac{\lambda}{\sum_{i=1}^{n=7}w_{i}}\sum_{i=1}^{n=7}w_{i}\,\mathrm{sim}\!\left(e_{q},e_{d}^{i}\right) \displaystyle\quad+(1-\lambda)\,\operatorname{BM25}(q,d) (1) 2.4.1 Optimizing Retrieval Weights For the embedding weights optimization, a simulated validation set was created by randomly sampling 50 devices and generating a hypothetical query for each device. This query was generated using Google’s Gemma-3 (specifically, gemma3n:e2b, a 2B parameter model designed to be computationally efficient). The model was prompted to create a clinically relevant search query based on the thesis and keywords of each device (see Appendix). The weights for the 7 features were optimized using Optuna ( 1 ) , a hyperparameter optimization software package. The objective function for optimization was the mean Hit Rate@K with K=5 across the 50 sampled devices (if the ground truth device was in the top K=5 returned results, the retrieval was considered a hit). The weights were optimized over a number of trials, where for each trial, Optuna samples a set of weights (constrained to the range of [0.01, 0.5]), the objective function score is recorded, and then new weights are sampled for the next trial based on the cumulative

Read the complete text