Building a REST Client in R

March 17, 2026 · 5 min read · Updated March 17, 2026 · advanced

rest api httr2 http r

In the previous tutorials of this series, you learned how to make HTTP requests with httr2 and work with APIs. This tutorial goes deeper into building a production-ready REST client — a reusable abstraction that handles authentication, retries, pagination, and error handling gracefully.

By the end, you’ll have a client that’s robust enough for real-world data pipelines.

Why Build a REST Client?

When you’re just getting started with an API, direct function calls work fine:

resp <- request("https://api.example.com/data") |> 
  req_perform()

But as your usage grows, you’ll encounter challenges:

Authentication tokens expire and need refreshing
Rate limits require exponential backoff
Large datasets come in pages
Network failures happen — your code should handle them
Different endpoints need different configurations

A well-structured REST client encapsulates all this complexity behind a clean interface. Instead of repeating authentication logic and error handling in every function call, you build it once and reuse it.

Designing Your Client

Let’s build a client for a hypothetical JSON API. The patterns apply to any REST API.

Step 1: Create a Client Function

The core of your client is a function that initializes a request with defaults:

library(httr2)

create_client <- function(base_url, api_key = NULL) {
  req <- request(base_url)
  
  if (!is.null(api_key)) {
    req <- req |> req_headers("Authorization" = paste("Bearer", api_key))
  }
  
  req
}

This creates a request template you can modify for each call. You start with a base configuration and add specifics as needed.

Step 2: Add Error Handling

HTTR2 makes error handling elegant with req_error():

safe_request <- function(req) {
  req |>
    req_error(is_error = ~ TRUE) |>
    req_perform() |>
    resp_check_status()
}

The is_error predicate returns TRUE for any 4xx or 5xx response, turning them into R errors with meaningful messages. By default, HTTR2 only throws errors for 5xx server errors; this makes it treat client errors (like 404 or 401) the same way.

You can also customize error handling for specific status codes:

handle_not_found <- function(req) {
  req |>
    req_error(status_code = ~ .x == 404, body = ~ "Resource not found") |>
    req_perform()
}

Step 3: Implement Automatic Retries

Network failures happen. Use req_retry() for resilience:

robust_request <- function(req, max_retries = 3) {
  req |>
    req_retry(
      max_tries = max_retries,
      backoff = ~ exp(.x) * 0.5,  # Exponential backoff
      is_transient = ~ resp_status(.x) >= 500
    ) |>
    req_perform()
}

The backoff formula starts at 0.5 seconds and doubles with each retry: 0.5s, 1s, 2s, 4s. The is_transient function tells HTTR2 which responses should trigger a retry — here, any 5xx server error.

You can also retry on rate limiting (429) with a longer backoff:

rate_limited_request <- function(req) {
  req |>
    req_retry(
      max_tries = 5,
      backoff = ~ if (resp_status(.x) == 429) 60 else exp(.x) * 0.5,
      is_transient = ~ resp_status(.x) >= 500 || resp_status(.x) == 429
    ) |>
    req_perform()
}

Step 4: Handle Pagination

Many APIs return paginated results. Here’s a pattern for collecting all pages:

fetch_all_pages <- function(client, endpoint) {
  all_results <- list()
  page <- 1
  has_more <- TRUE
  
  while (has_more) {
    resp <- client |>
      req_url_path(endpoint) |>
      req_url_query(page = page, per_page = 100) |>
      robust_request()
    
    data <- resp_body_json(resp)
    all_results <- c(all_results, data$items)
    
    has_more <- !is.null(data$next_page)
    page <- page + 1
  }
  
  all_results
}

Different APIs use different pagination schemes. Common patterns include:

Offset-based: ?page=2&per_page=50
Cursor-based: ?cursor=abc123
Link headers: Check Link header for next relation

Adapt the pattern to match your API’s response format.

Step 5: Token Refreshing

OAuth tokens expire. Build automatic refresh into your client:

create_oauth_client <- function(base_url, client_id, client_secret) {
  # Initial token fetch
  token_resp <- request(base_url) |>
    req_url_path("oauth/token") |>
    req_method("POST") |>
    req_body_form(
      grant_type = "client_credentials",
      client_id = client_id,
      client_secret = client_secret
    ) |>
    req_perform() |>
    resp_body_json()
  
  token <- token_resp$access_token
  expires_at <- Sys.time() + token_resp$expires_in
  
  # Return a function that handles automatic refresh
  function(endpoint, ...) {
    if (Sys.time() > expires_at) {
      # Token expired — refresh it
      token_resp <- request(base_url) |>
        req_url_path("oauth/token") |>
        req_method("POST") |>
        req_body_form(
          grant_type = "client_credentials",
          client_id = client_id,
          client_secret = client_secret
        ) |>
        req_perform() |>
        resp_body_json()
      
      token <<- token_resp$access_token
      expires_at <<- Sys.time() + token_resp$expires_in
    }
    
    request(base_url) |>
      req_url_path(endpoint) |>
      req_headers("Authorization" = paste("Bearer", token)) |>
      robust_request()
  }
}

The <<- operator updates the token and expiry time in the parent environment. Each call checks if the token is still valid before making a request.

Adding Timeouts

Production code should set timeouts to avoid hanging requests:

timed_request <- function(req, timeout = 30) {
  req |>
    req_timeout(timeout) |>
    req_perform()
}

Combine this with your retry logic for a robust pipeline:

production_request <- function(req) {
  req |>
    req_timeout(30) |>
    req_retry(max_tries = 3, backoff = ~ exp(.x) * 0.5) |>
    req_error(is_error = ~ TRUE) |>
    req_perform()
}

Putting It All Together

Here’s a complete example combining all patterns:

library(httr2)
library(purrr)

# Initialize client
api_call <- create_oauth_client(
  "https://api.example.com",
  Sys.getenv("CLIENT_ID"),
  Sys.getenv("CLIENT_SECRET")
)

# Fetch paginated data with automatic retries
fetch_users <- function() {
  fetch_all_pages(api_call, "v1/users")
}

# Fetch a single resource
get_user <- function(user_id) {
  api_call(paste0("v1/users/", user_id)) |>
    resp_body_json()
}

# Get data
users <- fetch_users()
user <- get_user(12345)

Best Practices

Store credentials in environment variables, not in your code. Use Sys.getenv() to retrieve them at runtime.
Log failures with timestamps for debugging. Wrap requests in tryCatch() to capture details:

try_fetch <- function(req) {
  tryCatch(
    robust_request(req),
    error = function(e) {
      message("Request failed: ", e$message)
      NULL
    }
  )
}

Set timeouts with req_timeout() to avoid hanging requests on slow or unresponsive APIs.
Test with mocked responses using the httptest2 package. Mock the API responses during testing to avoid hitting rate limits and ensure consistent test results.
Version your client as the API changes. Keep client code in a separate package or module to manage breaking changes cleanly.

Common Pitfalls

Forgetting to handle 404 — Missing resources shouldn’t crash your pipeline
No retry on 429 — Rate limits are transient; retry after the suggested delay
Hardcoding URLs — Use environment variables for base URLs to support staging and production
Ignoring response encoding — Some APIs return gzipped responses; HTTR2 handles this automatically