Fast File Import with readr
When working with data in R, reading flat files like CSVs is one of the most common tasks. Fast file import with readr provides a reliable and performant way to import data, handling type detection, column parsing, and encoding issues so you can focus on analysis instead of data cleaning.
Why use readr?
The base R functions like read.csv() have served us well for decades, but they come with some baggage. They’re relatively slow for large files, make assumptions about your data that may not hold, and their defaults don’t always align with modern data workflows.
readr addresses these issues by being:
- Faster, Uses parallel processing and parses data more efficiently
- Smarter, Automatically detects delimiters and handles type conversion
- Tidier, Returns tibbles instead of data frames, with better printing
- More controllable, Explicit column type specification when you need it
Installing and loading readr
If you haven’t already, install readr from CRAN:
install.packages("readr")
library(readr)
Reading CSV files
The most common use case is reading comma-separated values. Use read_csv() for this:
# Read a CSV file
df <- read_csv("my_data.csv")
# Print the tibble to inspect
df
When you print a readr tibble, you get a compact summary showing the column names, types, and the first few rows:
# A tibble: 1,000 × 5
name age department salary
<chr> <dbl> <chr> <dbl>
1 Alice 32 Engineering 75000
2 Bob 28 Sales 62000
3 Carol 45 Marketing 89000
This output format makes it easy to see at a glance what data you’re working with, much cleaner than the default data.frame printing in base R.
Reading TSV and other delimited files
For tab-separated values, use read_tsv():
# Read a tab-separated file
df_tsv <- read_tsv("data.tsv")
For files with other delimiters, read_delim() handles any character as a separator:
# Read a semicolon-delimited file
df_semi <- read_delim("data.txt", delim = ";")
# Read a pipe-delimited file
df_pipe <- read_delim("data.txt", delim = "|")
Specifying column types
One of readr’s superpowers is automatic type detection, but sometimes you need explicit control. The col_types argument lets you specify exactly what each column should be:
df <- read_csv("data.csv",
col_types = cols(
id = col_integer(),
name = col_character(),
value = col_double(),
flag = col_logical()
))
The available column types are:
col_integer(), Integer numberscol_double(), Floating point numberscol_character(), Stringscol_logical(), TRUE/FALSE valuescol_factor(), Categorical factorscol_date(),col_datetime(),col_time(), Date/time typescol_skip(), Don’t import this column
Handling missing values
readr automatically converts common missing value representations:
# These are all treated as NA:
# - Empty strings: ""
# - "NA" (the string)
# - "N/A", "n/a"
# - ".", "#NUM!", etc.
df <- read_csv("data.csv", na = c("", "NA", "N/A"))
The na argument lets you customize what strings should be treated as missing values.
Reading files from uRLs
readr can read directly from URLs, which is handy for accessing online datasets:
# Read a CSV directly from the web
df <- read_csv("https://example.com/data.csv")
# Read from GitHub raw URLs
df <- read_csv("https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2022/2022-01-11.csv")
Performance tips
When working with large files, these tips will speed things up:
Skip rows and limit columns
# Skip the first 10 rows (like header comments)
df <- read_csv("data.csv", skip = 10)
# Only read first 1000 rows
df <- read_csv("data.csv", n_max = 1000)
# Select specific columns
df <- read_csv("data.csv", col_select = c(id, name, value))
Specify column types upfront
Specifying types avoids the overhead of type inference:
# Faster: types known upfront
df <- read_csv("large_file.csv",
col_types = cols(.default = col_double()))
Use show_col_types = FALSE
Suppress the column type printing for cleaner output in scripts:
df <- read_csv("data.csv", show_col_types = FALSE)
Comparing with base R
Here’s a quick comparison between readr and base R functions:
| Feature | readr | Base R |
|---|---|---|
| Returns | Tibble | Data frame |
| Speed | Faster | Slower |
| Type detection | Automatic | Automatic |
| String factors | No (by default) | Yes (by default) |
| Row names | None | Optional |
The main practical difference you’ll notice is that readr never creates row names, and it doesn’t automatically convert strings to factors, both behaviors that modern R programmers generally prefer.
Column type specification
read_csv() guesses column types from the first 1000 rows. Explicitly specify types with col_types: col_types = cols(id = col_integer(), date = col_date(), amount = col_double()). col_types = cols(.default = col_character()) reads all columns as character, useful when guessing fails. col_skip() skips a column during import.
Large file strategies
read_csv_chunked("file.csv", callback = DataFrameCallback$new(fn), chunk_size = 10000) processes a large file in chunks without loading it all into memory. fn receives each chunk as a data frame. For files that fit in memory but load slowly, data.table::fread() is 5-10x faster than read_csv(). Arrow’s read_csv_arrow() is similarly fast and returns an Arrow Table that DuckDB can query.
Column specification
readr infers column types by scanning the first 1000 rows. For large files with unusual values, the guess may be wrong. spec_csv("file.csv") prints the inferred column specification, examine it before reading the full file.
read_csv("file.csv", col_types = cols(id = col_integer(), date = col_date(format = "%m/%d/%Y"), score = col_double())) specifies types explicitly. col_skip() drops a column. cols(.default = col_character()) reads all unspecified columns as character, a conservative approach for messy data.
col_factor(levels = c("low", "medium", "high"), ordered = TRUE) reads a column as an ordered factor. col_date("%Y-%m-%d") parses dates in ISO format. Custom date formats: %m/%d/%Y for US format, %d.%m.%Y for European dot-separated format.
problems(df) returns a data frame of parsing failures, rows and columns where the inferred type did not match the actual value. Examine these to understand data quality issues before proceeding with analysis.
Locale and encoding
read_csv("file.csv", locale = locale(encoding = "latin1")) reads non-UTF-8 files. Common encodings: "latin1" (ISO-8859-1) for Western European files, "CP1252" for Windows-generated files, "UTF-16" for some Excel exports.
The locale argument also controls decimal marks and grouping marks: locale(decimal_mark = ",", grouping_mark = ".") handles European number format where 1.234,56 means one thousand two hundred thirty-four point fifty-six.
readr::guess_encoding(path) suggests likely encodings by reading a sample of the file. stringr::str_conv(text, "UTF-8") converts from a detected encoding.
Large files and performance
read_csv_chunked("file.csv", callback = DataFrameCallback$new(function(chunk, pos) { process(chunk) }), chunk_size = 100000) processes a CSV in 100k-row chunks without loading the full file into memory. Combine with a callback that accumulates results or writes to a database.
vroom::vroom("file.csv") is faster than readr::read_csv() for very large files because it uses lazy evaluation — it reads only the columns you actually use. vroom is included in the tidyverse and shares readr’s column specification interface.
For repeated analysis on the same data, read once and save as Parquet: arrow::write_parquet(df, "data.parquet"). Subsequent reads with arrow::read_parquet() are 5-10x faster than CSV re-parsing.
Multiple files
purrr::map_dfr(file_paths, read_csv) reads multiple CSVs and row-binds them into one data frame. If files come from different periods, add a filename column: purrr::map_dfr(files, ~ read_csv(.x) %>% mutate(source = .x)).
list.files(path, pattern = "*.csv", full.names = TRUE) collects file paths. fs::dir_ls(path, glob = "*.csv") is the tidyverse alternative with more options. purrr::map_dfr(fs::dir_ls("data/"), read_csv, .id = "file") reads all CSVs and tracks the source file name.
When all files have the same schema, dplyr::bind_rows() handles minor type differences (integer vs double columns) more gracefully than rbind(). When schemas differ, read each file separately, reconcile schemas, then bind.
The readr approach to import
readr is designed around the principle that import should be predictable and explicit. The type inference works by sampling the first 1000 rows and guessing the column type from those values. When the guess is wrong — a column with numeric values in the first 1000 rows but mixed types later — you get a column type that changes mid-read, usually with a warning. Specifying column types explicitly with col_types prevents these mid-read type changes.
The column specification printed after reading a file is readr’s way of communicating what it inferred. Copy that specification and pass it back to read_csv as the col_types argument to convert from inference to declaration. This makes subsequent reads deterministic: the column types are fixed regardless of what the first 1000 rows contain. For files that are read repeatedly as part of a pipeline, fixing the types is worth the one-time setup cost.
Locale and number parsing
readr’s locale object controls locale-specific parsing: the decimal mark, the grouping mark for thousands separators, the date format, and the encoding. Different regions use different conventions — a number that is “1,234.56” in North America is “1.234,56” in Germany. The locale argument to read_csv applies the locale globally; the col_locale argument to a column type applies it to one column.
Setting the locale correctly matters for files from international sources. A CSV exported from German Excel uses commas as decimal marks and semicolons as delimiters. read_csv2 handles the semicolon delimiter; you still need to set locale = locale(decimal_mark = ”,”, grouping_mark = ”.”) for numeric parsing. Getting this wrong produces character columns where numeric columns were expected, causing downstream type errors.
Multiple file import patterns
Reading multiple files with the same structure and combining them into one data frame is a common pattern for partitioned data — one file per day, one file per region. Passing a vector of file paths to read_csv reads all files and binds them by row, adding a column indicating the source file. The id argument to read_csv names this source column.
For reading many files where some may not exist or may have parsing errors, map over the file list with purrr::possibly or safely to handle individual file failures without stopping the whole import. A file missing from an expected daily sequence is a data quality issue worth knowing about; using safely lets the rest of the import complete while capturing the failure for review.
Summary
The readr package is your go-to for importing flat files in R. Whether you’re loading customer data, survey results, or any tabular dataset, readr makes the process painless and predictable.
Key takeaways:
- Use
read_csv()for CSVs,read_tsv()for TSVs, andread_delim()for custom delimiters - Specify
col_typeswhen you need explicit control over column interpretation - Use
nato handle missing value representations in your data - Take advantage of
skip,n_max, andcol_selectfor large files - Enjoy the tibble output and sensible defaults
With readr in your toolkit, you’ll spend less time fighting with file imports and more time doing actual analysis. The next time you need to bring data into R, reach for readr first, your future self will thank you.
Next steps
Now that you understand fast file import with readr, explore these related topics to deepen your knowledge and apply these techniques in more complex scenarios.