List Parquet Files in a Hugging Face Dataset

Query the Hugging Face Datasets API and return a tibble of files with a .parquet suffix. The function follows pagination links returned in the response Link header and returns path, size, and a resolved URL.

Usage

get_available_huggingface_files(organization, dataset)

Arguments

organization

Character(1). Hugging Face organization or user name.

dataset

Character(1). Dataset name under the organization.

Value

A tibble with columns: path (character), size (numeric), and url (character).

Details

Uses httr2 to perform HTTP requests. Requires internet access and the dataset to be publicly accessible or accessible with appropriate authentication.

See also

Other utility functions: create_summary_statistics(), list_supported_datasets(), list_supported_datasets_ff(), list_supported_datasets_ff_legacy(), list_supported_datasets_macro_predictors(), list_supported_datasets_other(), list_supported_datasets_pseudo(), list_supported_datasets_wrds(), list_supported_indexes(), list_supported_jkp_factors(), list_tidy_finance_chapters(), open_tidy_finance_website(), trim(), validate_dates(), winsorize()

Examples

if (FALSE) { # \dontrun{
get_available_huggingface_files("voigtstefan", "sp500")
} # }