NHANES-GCP - Leveraging the Google Cloud Platform and BigQuery ML for reproducible machine learning

Q: What problem does NHANES-GCP address for researchers using NHANES data?

NHANES analyses require extensive, repetitive data management and cleaning before models can be trained, which costs research time and reduces reproducibility. NHANES-GCP automates these engineering steps on GCP to streamline preparation and improve consistency.

Q: How does NHANES-GCP automate infrastructure and data transformation?

It uses CDKTF for Infrastructure-as-Code and dbt resources to automate data engineering and management tasks within the Google Cloud Platform ecosystem. This standardizes how NHANES datasets are processed into clean tables.

Q: How is BigQuery ML used in NHANES-GCP?

BigQuery ML is integrated to enable model training and statistical analysis through an SQL-like interface. The workflow covers data selection and integration, training machine learning and statistical models, and generating results from a single query.

Aug 6, 2026

Upload by Finn

NHANES-GCP automates the data engineering workflow required to use NHANES data effectively, reducing repetitive cleaning and configuration overhead that can slow research and harm reproducibility. Built on Google Cloud Platform using CDKTF Infrastructure-as-Code and dbt, it provides clean, integrated tables ready for large-scale analysis. The approach demonstrates end-to-end modeling with BigQuery ML—data selection, integration, training, and result generation—via a single SQL-like interface, supporting analytics and fine-tuning of LLMs while remaining cost-effective.

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages7

Reading Time18 min

AccessFree to download

Abstract

Summary

Availability and implementation

Introduction

Background on NHANES use

Infrastructure and automation with NHANES-GCP

BigQuery ML integration

Reproducibility benefits

Use for LLM development and training

Implementation

System architecture and prerequisites

NHANES-GCP: Leveraging the Google Cloud Platform and BigQuery ML for reproducible machine learning with data from the National Health and Nutrition Examination Survey B. Ross Katz1 , Abdul Khan1 , James York-Winegar1 , and Alexander J. Titus2 , 3 , 4 1CorrDyn, Nashville, 37219 , USA 2 In Vivo Group, Los Angeles, 90292 , USA 3 Information Sciences Institute, University of Southern California, Los Angeles, 90292 , USA 4 Iovine and Young Academy, University of Southern California, Los Angeles, 90089 , USA Corresponding author: Alexander J. Titus ([publications@theinvivogroup.com](publications@theinvivogroup.com)) Abstract Summary NHANES, the National Health and Nutrition Examination Survey, is a program of studies led by the Centers for Disease Control and Prevention (CDC) designed to assess the health and nutritional status of adults and children in the United States ( U.S. ) . NHANES data is frequently used by biostatisticians and clinical scientists to study health trends across the U.S. , but every analysis requires extensive data management and cleaning before use and this repetitive data engineering collectively costs valuable research time and decreases the reproducibility of analyses. Here, we introduce NHANES-GCP, a Cloud Development Kit for Terraform (CDKTF) Infrastructure-as-Code ( IaC) and Data Build Tool (dbt) resources built on the Google Cloud Platform (GCP) that automates the data engineering and management aspects of working with NHANES data. With current GCP pricing, NHANES-GCP costs less than $2 to run and less than $15/yr of ongoing costs for hosting the NHANES data, all while providing researchers with clean data tables that can readily be integrated for large-scale analyses. We provide examples of leveraging BigQuery ML to carry out the process of selecting data, integrating data, training machine learning and statistical models, and generating results all from a single SQL-like query. NHANES-GCP is designed to enhance the reproducibility of analyses and create a well-engineered NHANES data resource for statistics, machine learning, and fine-tuning Large Language Models ( LLMs) . Availability and implementation NHANES-GCP is available at [https://github.com/In-Vivo-Group/NHANES-GCP](https://github.com/In-Vivo-Group/NHANES-GCP) Introduction The National Health and Nutrition Examination Survey ( NHANES) has been instrumental in shaping public health policies and research in the United States. It provides a comprehensive dataset reflecting the health and nutritional status of the U.S. population, covering a wide range of demographic groups. Despite its extensive utility, the effective use of NHANES data often requires significant data management and preprocessing ( 1) , which can be both time-consuming and a barrier to reproducibility (2) . These challenges necessitate innovative solutions to streamline the research process and enhance data usability. In response to this need, we introduce NHANES-GCP, a novel infrastructure developed on the Google Cloud Platform (GCP) . These Cloud Development Kit for Terraform (CDKTF) and Data Build Tool (dbt) resources automate the data engineering process for NHANES data, addressing the critical need for efficiency and reproducibility in research (3) . The operational cost-effectiveness of NHANES-GCP, with a setup cost less than the cost of a cup of coffee (<$2) and minimal ongoing fees (<$15/yr) , makes it a viable option for researchers and institutions of varying scales. A pivotal feature of NHANES-GCP is its integration with BigQuery ML, which simplifies data analysis through an SQL-like interface, allowing for seamless model training and statistical analysis (4) . This integration signifies a progressive step towards the amalgamation of traditional statistical approaches and modern machine learning techniques, enhancing the scope and depth of health data analysis. Crucially, NHANES-GCP addresses the challenge of reproducibility in scien

FAQs

What problem does NHANES-GCP address for researchers using NHANES data?

NHANES analyses require extensive, repetitive data management and cleaning before models can be trained, which costs research time and reduces reproducibility. NHANES-GCP automates these engineering steps on GCP to streamline preparation and improve consistency.

How does NHANES-GCP automate infrastructure and data transformation?

It uses CDKTF for Infrastructure-as-Code and dbt resources to automate data engineering and management tasks within the Google Cloud Platform ecosystem. This standardizes how NHANES datasets are processed into clean tables.

How is BigQuery ML used in NHANES-GCP?

BigQuery ML is integrated to enable model training and statistical analysis through an SQL-like interface. The workflow covers data selection and integration, training machine learning and statistical models, and generating results from a single query.

Aug 6, 2026

Upload by Finn