Harnessing Artificial Intelligence for Hypothesis Generation in Childhood Asthma: Insights from NHANES

Large language models (LLM) can understand human language and formulate responses interpretable by humans. LLMs perform well on a wide range of natural language processing tasks, including language understanding, text generation, sentiment analysis, and machine translation. Prominent examples of current LLM include OpenAI’s ChatGPT [1], Google DeepMind’s Gemini [2], Anthropic’s Claude [3], Meta’s LLaMA [4], and Mistral’s Mixtral [5]. LLM has been used by researchers in domain-specific scenarios. For example, a pilot study demonstrated the feasibility of generating clinical letters for patients with average reading ability using ChatGPT [6].

An important goal of epidemiologic research is to investigate protective or risk factors for diseases. Researchers typically start with a hypothesis that is tested using available data [7,8,9,10,11,12,13]. Formulating a proper hypothesis typically requires advanced knowledge, including synthesizing evidence, identifying research gaps, and conceptualizing the research question—a process that is time-consuming and demands a high level of expertise but may be facilitated by LLM.

LLM has potential uses in biomedical research but can generate false answers that may seem correct (“Artificial Hallucinations”) [14]. Such hallucinations are not permissible in research settings where accuracy and reliability are critical.

The root causes of artificial hallucinations stem from fundamental limitations of LLM, including a lack of access to up-to-date or proprietary data and limited reasoning capabilities, particularly in domains such as mathematical reasoning. Thus, LLMs often struggle to incorporate constantly updated knowledge or proprietary resources, such as specialized databases [15].

One approach to mitigating hallucinations is domain-specific training. ProtGPT2 has been trained to understand protein sequences, supporting protein design and engineering [16], while BioGPT, trained on PubMed abstracts, has demonstrated strong performance in text generation and information extraction tasks [17]. However, fine-tuning often reduces versatility, resulting in diminished performance on tasks outside the training domain and increasing the risk of model overfitting.

A more lightweight solution involves the use of engineered prompts. GeneGPT has demonstrated how in-context learning, combined with carefully crafted prompts, can enable LLM to interact with the National Center for Biotechnology Information (NCBI) Web APIs to answer genomics questions [18].

Strategies to mitigate hallucinations range from established techniques—such as prompt engineering, retrieval-augmented generation (RAG) [19,20], and chain-of-thought (CoT) [21] distillation—to developing novel architectures that combine algorithmic innovation with improved data quality. Researchers have also explored flexible, modular systems that integrate language models with symbolic tools (e.g., APIs), enhancing adaptability and reliability [15]. An example is PaperQA [22], an agent-based RAG framework specifically designed to process scientific literature at scale.

Building on these advances, we aimed to leverage LLM while mitigating the risk of hallucinations by integrating components of data query, statistical analysis, and evidence-based reasoning (RAG enhanced). To evaluate its effectiveness and credibility, we conducted a pilot study exploring factors potentially associated with asthma, the most common chronic respiratory disease of childhood, utilizing data from participants in the National Health and Nutrition Examination Survey (NHANES). While numerous studies have identified a wide range of factors linked to asthma (e.g., genetic and environmental), a comprehensive analysis of potential protective or risk factors could provide deeper insights into this disease.

NHANES, conducted by the National Center for Health Statistics of the Centers for Disease Control and Prevention, is a nationwide survey designed to evaluate the health and nutritional status of adults and children in the U.S. All data used in this analysis were freely available and obtained from the NHANES website (https://www.cdc.gov/nchs/nhanes/index.html, accessed on 14 April 2026). We included data for 10,359 distinct variables from 25,839 children (ages 6–17 years) who participated in the NHANES cycles from 2001 to 2020. Current asthma was defined by a positive answer to both of the following questions: “Has a doctor or other health professional ever told you that you have asthma?” and “Do you still have asthma?”. Participants who answered no to both questions were selected as controls.

The AI-Aided Hypothesis Generating & Testing Framework was implemented as a modular agent system using the LangChain [23] framework (Figure 1). Following the predefined state graph, the LLM-based agent could execute the workflow and call the tools or LLM models to fulfill the tasks for each step. The agent would use the result context information to reason and adjust its approach as needed until the task was finished.

The implemented tools included a database query tool, a statistical tool, a paper search tool, and a paper download tool. The LLM models were implemented in the Key Generator and Hypothesis Tester modules. The database query tool could access the NHANES database that was downloaded and stored beforehand. The statistical tool applied multivariable logistic regression to the structured data to get a list of variables significantly associated with current asthma from a statistical viewpoint (Figure 2: ① Database Query Tool Call and ② Statistical Tool Call).

First, for each candidate risk factor, the Database Query Tool was employed to extract data recordings from the NHANES dataset. To ensure the reliability of statistical inferences, we applied variable filtering criteria commonly used in pheWAS analysis [24]. We excluded variables with extreme distributions. In particular, we excluded: binary variables with <20 observations in any group, categorical variables with <20 observations in total across all groups (excluding the dominant group), any variable with <100 total recorded observations, and variables with multiple records per sample (to ensure one-to-one correspondence between observations and samples). Afterward, 3071 candidate variables with valid recordings were retained for multivariable logistic regression analysis, including 274 binary variables, 1208 categorical variables, and 1589 continuous variables.

Next, we applied a series of logistic regression models adjusted for age and gender to examine the associations of each variable with current childhood asthma. Considering the complex multistage survey design of NHANES, we followed the Weighting instruction (https://wwwn.cdc.gov/nchs/nhanes/
tutorials/Weighting.aspx, accessed on 14 April 2026) to select and query the corresponding sample weight variable from NHANES for each valid candidate risk factor. To further account for the complex survey design and sampling methods in NHANES—including stratification, clustering, and oversampling of specific subgroups—we constructed a multivariable regression model to incorporate the primary sampling unit variable (SDMVPSU), the stratification variable (SDMVSTRA), and the appropriate sample weight that were provided by NHNAES, ensuring they are representative of the non-institutionalized U.S. population. Significant associations were defined as those with false discovery rate-adjusted P (FDR-P) < 0.01. A total of 141 variables were significant. Human reviews were then conducted to identify and exclude non-informative variables and ensure the stability of regression results (Figure 2: ② Statistical Tool Call). 13 quality control variables were identified by the presence of “comment code” in their descriptions. Four variables were removed due to extreme odds ratio (OR) values, which indicated instability in the model estimates (likely caused by sparse data or quasi-complete separation). An additional 20 variables were removed due to OR = 1, suggesting no association.

Finally, a total of 100 variables, the agent generated tentative affirmative statements regarding their association with current asthma in the following format.

Asthma and Variable Name (Variable Description) are related.

For example, for the variable LBDEONO (peripheral blood eosinophils ≥1000 cells/µL), a claim would be generated as:

Asthma and LBDEONO are related.

Next, the LLM agent would call Key Generator (Figure 2: ③ Key Generator Call) to generate keywords to search in the PubMed database (Prompts in Figure S7).

For example, for the variable SPDBRONC (variable description: Best test FEV1/FVC ratio below Lower Limit of Normal and/or less than 70%; data file description: Spirometry—Pre and Post-Bronchodilator), the generated keywords were: [‘FEV1’, ‘FVC’, ‘Ratio’, ‘Bronchodilator’, ‘Spirometry’].

For DXXGYTOM (variable description: Gynoid total mass; data file description: Dual Energy X-ray Absorptiometry—Android/Gynoid Measurements), the generated keywords are: [‘Mass’, ‘X-ray’, ‘Absorptiometry’, ‘Energy’, ‘Android’, ‘Gynoid’].

For LBXSAL (variable description: Albumin, refrigerated serum (g/dL); data file description: Standard Biochemistry Profile), the generated keywords are: [‘Albumin’, ‘Biochemistry’, ‘Profile’].

To ensure the search was limited to asthma-related publications, the term asthma was combined with all other keywords using the AND operator. This guarantees that every retrieved article is about asthma. Within each group of related concepts, the keywords were combined with the OR operator to capture variations in terminology and maximize coverage. For example, the query term for SPDBRONC (variable description: Best test FEV1/FVC ratio below Lower Limit of Normal and/or less than 70%) was: Asthma AND (FEV1 OR FVC OR Ratio OR Bronchodilator OR Spirometry), Asthma AND (Mass OR X-ray OR Absorptiometry OR Energy OR Android OR Gynoid) for DXXGYTOM (variable description: Dual Energy X-ray Absorptiometry—Android/Gynoid), and Asthma AND (Albumin OR Biochemistry OR Profile) for LBXSAL (variable description: Albumin (g/dL)).

Such keywords were then used as parameters by the Paper Search Tool to search in the PubMed database (Figure 2: ④ Paper Search Tool Call), and the Paper Download Tool, implemented with open-source Python library “paperScraper”, would then download the PDF documents of related scientific papers into a local folder (Figure 2: ⑤ Paper Download Tool Call).

To implement the evidence-based validation step, we integrated the PaperQA library [22] as our Hypothesis Tester module (Figure 2: ⑥ Hypothesis Tester Call). This provided a robust, pre-validated framework for parsing PDFs, retrieving relevant passages via semantic search, and synthesizing summaries with relevance scores. The process, as detailed in [22], involved:

To assess the quality of the LLM-generated keywords, we conducted a survey among domain researchers. The survey presented 20 randomly selected variables and asked researchers to evaluate the corresponding generated keywords across three dimensions on the scale of 1–5 (1 = Strongly Disagree, 5 = Strongly Agree):

To validate the LLM-generated conclusions, we manually reviewed the variable definitions, LLM-provided summary, and cited evidence (Figure 2: ⑦ Conclusion Review). Variables were classified according to the following criteria:

To further evaluate the reliability of the manual classifications, we compared our results with those generated independently by three additional large language models (ChatGPT, DeepSeek, and Gemini) (These models were used exclusively in a validation step and were separated from the main workflow which employ only Llama 3 as the LLM in the hypothesis generation or testing process, as illustrated in Figure 1). Each model was provided with the same conclusion generated by the Hypothesis Tester module and asked to classify the variables into the same three categories (directly related, indirectly related, or not related to current asthma). We then compared their outputs with our manual review. Agreement rates were calculated for both a binary grouping (direct + indirect combined as “related” vs. “not related”) and the full three-class grouping.

This analysis included 25,839 children with (cases, n = 2928) and without (controls, n = 22,911) who participated in the 10 NHANES cycles from 2001 to 2020. Multivariable logistic regression was used to test for the association between 10,359 variables in study participants and current asthma. After applying quality control measures, 100 variables were identified as significantly associated with asthma, with a median sample size of 8035 (interquartile range [IQR] = 5168–16,628). Of these 100 variables, 51 were Questionnaire data, 29 were Examination data, 10 were Laboratory data, 9 were Demographics data, and 1 was Dietary data, encompassing all five data types in the NHANES dataset (Figure S1). Table 1 presents the top 10 variables associated with asthma, ranked by FDR-P value in descending order.

Figure 3 shows the category distribution of the selected variables. The top categories with the highest number of variables were “Body Measures”, “Respiratory Health”, “Demographic Variables and Sample Weights”, “Spirometry”, and “Hospital Utilization & Access to Care”. Overall, the variables were well distributed across health conditions, lifestyle and behavioral factors, and laboratory measurements.

After identifying significant variables, the LLM agent generated keywords for a PubMed database search. To evaluate the quality of the LLM-generated keywords, we conducted a survey with 9 domain researchers. Of these 9 researchers, 4 had more than 10 years of research experience in asthma and regularly used PubMed, Google Scholar, or Scopus. The survey presented researchers with 20 randomly selected variables and asked them to assess the corresponding generated keywords along three dimensions: coverage/recall (whether the keywords captured a broad and representative set of concepts), precision (whether the keywords were specific and relevant to the research scope), and redundancy (whether the list avoided unnecessary repetition).

On a 1–5 scale, the keywords received consistently high ratings, with average scores of 4.19 for coverage/recall, 4.15 for precision, and 4.35 for redundancy. These results indicate that the LLM agent produced keywords that were both comprehensive and relevant, with minimal overlap, supporting the reliability of the evidence-gathering process.

With the retrieved list of papers, the Paper Download Tool would try to download the PDF files of all the full-text studies it had access to. However, due to publisher restrictions and paywalls, full-text access could not be guaranteed for all retrieved articles.

Figure S2a shows a histogram of the number of downloaded and loadable studies, which ranged from 15 to 563, with a mean of 320.9 (SD = 106.22) and an interquartile range from 283.25 (25th percentile) to 393.5 (75th percentile).

The LLM’s ability to validate an association was contingent on the evidence that was available in the retrieved studies. As more publications became available, the model was more likely to encounter relevant data and consensus (or conflict) in the literature, leading to a more informed, and thus potentially more reliable, conclusion. Therefore, the sheer volume of retrievable references (measured as the number of loadable papers) served as a crucial factor in gauging the robustness of the LLM’s output.

From Figure S2b, we observe that over 99.0% of the variables had 10 or more full-text studies as references, 95.0% had more than 50, and more than 92.0% had over 100 full-text studies as references. The number of references provided a reasonable basis for the LLM’s conclusions.

For the 100 variables significantly associated with current asthma, the Hypothesis Tester module called the LLM model to generate answers based on the evidence it gathered. We manually reviewed the generated conclusions and classified 100 variables into three categories (Figure S3):

For the remaining 2 variables, the LLM failed to understand the exact meaning of the variables or their data file descriptions and provided irrelevant answers (Table S1).

Reproducibility testing with three additional LLMs demonstrated high consistency with our manual classifications. When using the binary grouping of “related” versus “not related”, agreement rates with the manual review were 98% (ChatGPT), 89% (DeepSeek), and 90% (Gemini), indicating strong reproducibility across models. When retaining all three categories, agreement rates were 88%, 81%, and 82%, respectively. These findings show that, similar to human reviewers, LLMs cannot always reach complete agreement, particularly when distinguishing between “Related” and “Indirect Related” relationships, as the boundary between these categories can often be vague.

Published studies supported the directly related or indirectly related variables. They are positive validations of the workflow, demonstrating that it can reliably recover known associations from observational data. In Figure 4, we show an example using one of the variables identified as related to current asthma, DBQ223A (What type of milk was it? Was it usually …) and extracted from the Diet Behavior & Nutrition DataFile. The final result was an LLM generated answer to the claim, organized in the format of Question, Answer, and References. In this example, LLM listed 5 references, with each one including a summary of the original text. Figure 3 and Figure S4 illustrate the distribution of variables across the categories found by the LLM to be Not Related, Indirectly Related, or Related to current asthma.

Variables classified as “not related” are those for which the LLM identified no supporting evidence in the provided context linking them to current asthma. This classification reflects the scope and limitations of the information used in the analysis and should not be interpreted as evidence of a definitive absence of association. Rather, it indicates that no documented mechanistic or epidemiological support was found within the analyzed sources. Importantly, some variables with statistically significant associations were classified as “not related” because statistical significance alone does not establish biological or clinical relevance. Such associations may arise from confounding, measurement artifacts, or downstream correlations without substantive supporting evidence. Distinguishing statistical associations from evidence-backed relevance is a central objective of the proposed workflow.

The most notable finding was that, in the absence of documented direct associations in the literature, the LLM systematically identified indirect relationships by integrating contextual information from the provided sources with its embedded domain knowledge. These inferred indirect relationships were not explicitly reported in the reviewed studies, yet they represent structured, evidence-informed connections grounded in existing mechanistic or epidemiological frameworks. As such, they constitute a validated output of the proposed workflow—demonstrating its ability to surface plausible, previously unarticulated links that can guide hypothesis generation and prioritization for subsequent empirical testing.

For the 2 variables with irrelevant answers from LLM (Table S1), a manual inspection of their variable and data file descriptions showed that their descriptions were very similar and lacked discriminative keywords, resulting in irrelevant conclusions. For example, the variable description for HUQ010 is “{First/Next} I have some general questions about {your/SP’s} health. Would you say {your/SP’s} health in general is …”. This demonstrates that the quality of an LLM’s conclusions heavily relies on the clarity and accuracy of variable descriptions.

This study demonstrates the use of LLM-based models for generating, validating, and rejecting hypotheses through a pilot case study using NHANES data to explore factors potentially associated with childhood asthma. Rather than replacing domain researchers, the proposed workflow restructures their involvement by shifting effort away from exhaustive variable screening and manual literature review toward targeted evaluation and validation. The LLM efficiently synthesizes evidence from the literature to identify variables with documented direct associations and to infer plausible indirect relationships when direct evidence is unavailable. These outputs provide structured, evidence-informed summaries that enable domain experts to focus on critical scientific judgment—assessing biological plausibility, identifying research gaps, and prioritizing variables for follow-up—thereby improving efficiency, transparency, and reproducibility in hypothesis-driven epidemiologic research.

The quality of the answers generated by LLM largely depended on whether the paper search tool module could find and download the most relevant papers on the topic. In this pilot study, we only used PubMed as our search engine, and not all the studies found could be downloaded for full text analysis, due to access limitations. However, >10 references could be downloaded and used as context information for LLM to draw conclusions for 99.0% of the variables of interest.

Searches using keywords composed solely of variable descriptions containing complex chemical compound names (e.g., “5-Hydroxymethyl-2-furancarboxylic”, “N-Nitrosoethylmethylamine (NMEA)”, and “Ethametsulfuron methyl”) often yielded no results in publication databases. Since the Data File Description for chemical compounds included higher-level categories (e.g., “Pesticides”, “Volatile Organic Compounds”, and “Perfluoroalkyl and Polyfluoroalkyl Substances”), we incorporated such categories into the prompts, obtaining markedly improved results, as the LLM successfully generated keywords that returned a substantial number of relevant publications.

While LLM can make logical inferences based on recognized facts, they struggle with vocabulary outside their knowledge base, making them ineffective at generating relevant keywords or answers for variables such as chemical compounds. Although this issue can be mitigated for most variables through proper prompt design, cases ending up with no loadable studies are inevitable. Without reliable evidence to synthesize and validate, the LLM is prone to AI hallucinations.

Understanding concepts and their relationships was valuable not only for generating relevant keywords but also for making reasonable inferences when generating answers or claims. These inferred relationships, combined with expert curation, could help validate generated outputs or inspire new hypotheses.

A relevant example is DRXTCRYP (Beta-cryptoxanthin (mcg)), a variable from the Dietary Interview (Figure S5). Although the LLM did not find direct evidence linking asthma to Beta-cryptoxanthin in the provided context, indirect evidence, such as “some carotenoids have been associated with respiratory morbidity and mortality” and “some studies suggest that certain carotenoids may be related to asthma diagnosis or prevention” was found. Recognizing Beta-cryptoxanthin as a kind of carotenoid, the LLM inferred that DRXTCRYP may be related to asthma.

Another example is LLM inference for MCQ120A (During the past 12 months, {have you/has SP} had … hay fever?) (Figure S6), a variable from the Medical Conditions database. Although LLM did not find direct evidence for the relationship between asthma and MCQ120A, it found that “atopic dermatitis, which may be related to hay fever, is one of the conditions for which dupilumab (IL-4Rα) is used. This suggests that there may be some overlap between asthma and other allergic conditions like hay fever” and “transmission of asthma susceptibility from mother to offspring through epigenetic changes and glucocorticoid signaling, which highlights the long-term effects of maternal asthma on fetal lung ILC2s and their responsiveness in adulthood. This may imply a link between maternal asthma and respiratory allergies like hay fever” [12]. From these, the LLM inferred that “there may be some overlap between asthma and other allergic conditions like hay fever due to shared mechanisms or susceptibility factors” based on the retrieved literature. This example demonstrates the capacity of LLM to make logical inferences based on underlying disease mechanisms.

Mapping lower-level concepts to higher-level concepts, or vice versa, proved highly effective in identifying relevant references. With refined prompt engineering or LLM accessible tools, general purpose LLMs enhanced with domain specific knowledge of vocabularies and concepts could achieve strong performance in domain-specific tasks.

Retrieval-Augmented Generation (RAG) using reliable publications improves the relevance and contextual grounding of LLM outputs but does not fully eliminate the risk of errors. Current LLM lack the ability to consistently recognize the boundaries of their own knowledge. This contributes to the persistence of hallucinations, underscoring the importance of integrating external validation mechanisms, domain-specific tools, or hybrid frameworks to ensure reliability in epidemiological applications [25].

We acknowledge additional study limitations. First, not all variables in NHANES were measured or available in every participant, leading to variable sample sizes across variables. Second, confounding is possible because we only adjusted our multivariable analyses for age and gender (two variables that influence both exposures and asthma outcomes). However, our goal in this pilot study was to identify associations while avoiding overadjustment. Third, a few variables were named differently across survey years (e.g., annual household income was recorded as “INDHHINC” prior to 2006 and as “INDHHIN2” thereafter). In this study, as the primary objective was to systematically screen a large number of candidate variables and identify those of potential interest, these variables were treated as distinct features rather than being combined, allowing each variable to be evaluated independently for its association with asthma while maintaining a highly efficient, largely automated workflow. Finally, we focused on a single dataset and a single disease (asthma), and thus our findings may not be generalizable to other populations or health outcomes.

In summary, this study introduced a structured, evidence-grounded, and reproducible workflow that integrates large-scale statistical screening with LLM-assisted literature retrieval, synthesis, and validation. While the individual components are not new, their coordinated use enables transparent differentiation between statistically significant associations and biologically supported relevance at scale. Our results highlight the potential use of LLMs in helping conduct efficient clinical research within a standardized, disease-agnostic framework. The proposed workflow can be readily applied to other disease contexts without structural modification, as its core components—statistical screening, evidence retrieval, synthesis, and validation—are not disease specific. Future work will focus on enhancing generalizability and efficiency by integrating domain-specific ontologies, expanding access to diverse databases, and developing specialized modules to better handle complex scientific terms. These advances will further reduce the burden on domain experts while preserving scientific rigor, enabling LLM-assisted workflows to drive innovation across a wide range of biomedical research domains.

The following supporting information can be found at: https://www.sciepublish.com/article/pii/982, Figure S1: Variable Data Type Distribution. Figure S2: (a) Histogram of number of loadable PDFs; (b) Complementary Cumulative Distribution of loadable PDF number. Figure S3: Different Answer Counts. Figure S4: Variable Category Distribution—Radar Plots. Figure S5: Example of LLM Generated Answer—Logical Inference 1. Figure S6: Key Generator Prompts. Figure S7: Prompts used to classify the LLM generated conclusions. Table S1: Variables with Non Discriminative Descriptions.

During the preparation of this manuscript, the author(s) used ChatGPT in order to check for grammatical errors and improve sentence clarity and flow. After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the published article.

The authors acknowledge support from the University of Pittsburgh Center for Research Computing (RRID:SCR_022735) through the computational resources provided.

J.L., W.C. and J.C.C. conceived this study. Y.-Y.H. participated in data collection and quality control. X.Y., K.M.G., F.J.R., J.L., W.C., C.Z. and J.C.C. participated in statistical analyses. J.L., W.C. and J.C.C. drafted the manuscript. All authors reviewed the manuscript, contributed substantial intellectual content, and approved the final version of the manuscript.

Not applicable.

Not applicable.

The data used in this study are publicly available from the National Health and Nutrition Examination Survey (NHANES). These datasets can be accessed free of charge through the official website of the Centers for Disease Control and Prevention (CDC) at https://www.cdc.gov/nchs/nhanes/ (accessed on 14 April 2026). No additional restrictions apply to the use of these data.

Dr. Liu was supported by a training grant T32 HL129949 from the U.S. National Institutes of Health (NIH). This work was supported by grants HL152475 and HL168539 from the NIH.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.