Data Manipulation with dplyr

March 7, 2026 · 11 min read ·Updated May 29, 2026 ·beginner

dplyrdata-manipulationtidyverserbeginner

Data manipulation with dplyr is the foundation of most tidyverse workflows. dplyr provides a consistent set of verbs — functions that solve the most common data-wrangling challenges — replacing dozens of lines of base R with readable, composable pipelines. In this tutorial, you’ll learn the six essential dplyr verbs that will transform how you work with data in R.

Installing and loading dplyr

Before we begin, make sure dplyr is installed and loaded:

# Install tidyverse (includes dplyr)
install.packages("tidyverse")

# Load dplyr specifically
library(dplyr)

For this tutorial, we’ll use a sample dataset. Let’s create a simple data frame representing a small retail business:

# Create sample sales data
sales <- data.frame(
  product = c("Laptop", "Mouse", "Keyboard", "Monitor", "Laptop", "Mouse", "Keyboard"),
  category = c("Electronics", "Accessories", "Accessories", "Electronics", "Electronics", 
               "Accessories", "Accessories"),
  region = c("North", "South", "North", "East", "West", "South", "North"),
  units_sold = c(5, 20, 15, 8, 3, 25, 12),
  price = c(999, 29, 79, 349, 999, 29, 79)
)

# Calculate revenue
sales$revenue <- sales$units_sold * sales$price

print(sales)

The pipe operator: your new best friend

Before diving into the verbs, you need to know about the pipe operator (%>%). It takes the output of one function and passes it as input to the next. Think of it as “then”:

# Without pipe: nested approach
result <- arrange(mutate(sales, revenue = units_sold * price), region)

# With pipe: readable chain
result <- sales %>%
  mutate(revenue = units_sold * price) %>%
  arrange(region)

Both produce the same result, but the piped version is much easier to read. R 4.1+ also supports the native pipe |>, which works identically.

filter(): selecting rows

The filter() function selects rows based on conditions. It’s similar to SQL’s WHERE clause:

# Filter rows where region is "North"
north_sales <- sales %>%
  filter(region == "North")

print(north_sales)

You can combine multiple conditions using logical operators:

# Electronics products in the North region with more than 5 units sold
high_electronics <- sales %>%
  filter(category == "Electronics", 
         region == "North", 
         units_sold > 5)

print(high_electronics)

Common comparison operators: == (equals), != (not equals), >, <, >=, <= Common logical operators: & (and), | (or), ! (not)

select(): choosing columns

The select() function chooses which columns to keep (or exclude):

# Select specific columns
product_revenue <- sales %>%
  select(product, revenue)

print(product_revenue)

select() supports pattern-matching helpers that save typing when columns follow naming conventions. starts_with() and ends_with() match prefixes and suffixes, contains() matches any substring, and matches() accepts a regular expression. To exclude columns, prefix the column name or helper with -. For renaming while keeping all columns, rename() follows the pattern new_name = old_name — more explicit than base R’s names(df)[i] <- "new" assignment.

# Keep columns matching a pattern
sales %>% select(starts_with("re"))
sales %>% select(contains("u"))

# Exclude a column
sales %>% select(-region)

# Rename without dropping anything
sales %>% rename(units = units_sold)

mutate(): creating new columns

Use mutate() to create new columns based on existing ones:

# Add a discount column and a discounted price
sales_with_discount <- sales %>%
  mutate(discount = ifelse(revenue > 500, 0.15, 0),
         final_price = revenue * (1 - discount))

print(sales_with_discount)

Multiple columns can be created in a single mutate() call, and later columns in the same call can reference columns created earlier — tax can use revenue computed on the line above. This in-order evaluation within one mutate() avoids the need for chained mutate() calls when new columns depend on each other.

multiplied <- sales %>%
  mutate(
    revenue = units_sold * price,
    tax = revenue * 0.08,
    revenue_after_tax = revenue + tax
  )

arrange(): sorting rows

arrange() sorts rows by one or more columns, ascending by default. Wrap a column in desc() to reverse the order. When sorting by multiple columns, ties in the first column are broken by the second, and so on — exactly like a spreadsheet sort with multiple sort keys.

# Ascending by revenue
sales %>% arrange(revenue)

# Descending with desc()
sales %>% arrange(desc(revenue))

Sort by multiple columns by listing them:

# First by category (alphabetically), then by revenue (descending)
sorted_multi <- sales %>%
  arrange(category, desc(revenue))

print(sorted_multi)

group_by() and summarise(): aggregation

These two verbs work together to create summaries by group:

# Group by category and summarize
category_summary <- sales %>%
  group_by(category) %>%
  summarise(
    total_revenue = sum(revenue),
    avg_units = mean(units_sold),
    n_transactions = n()
  )

print(category_summary)

The summarise() function creates new columns using aggregation functions like sum(), mean(), n(), min(), max(), sd(), and more:

# Detailed summary by region
region_summary <- sales %>%
  group_by(region) %>%
  summarise(
    total_revenue = sum(revenue),
    total_units = sum(units_sold),
    avg_price = mean(price),
    num_products = n_distinct(product)
  )

print(region_summary)

count() and tally(): quick counting

For simple counts, these verbs are incredibly useful:

# Count transactions by category
sales %>%
  count(category)

# Equivalent to:
sales %>%
  group_by(category) %>%
  tally()

Add a weight variable for weighted counts:

# Count total units sold by category
sales %>%
  count(category, wt = units_sold)

Chaining it all together

The real power of dplyr comes from chaining these verbs together. Here’s a complete example:

final_report <- sales %>%
  # Filter: focus on Electronics
  filter(category == "Electronics") %>%
  # Add new columns
  mutate(discount = 0.1,
         after_discount = revenue * (1 - discount)) %>%
  # Group by region
  group_by(region) %>%
  # Summarise each group
  summarise(
    total_revenue = sum(after_discount),
    total_units = sum(units_sold),
    avg_order = mean(after_discount)
  ) %>%
  # Sort by revenue descending
  arrange(desc(total_revenue))

print(final_report)

The six core verbs

filter() keeps rows matching a condition. select() keeps or drops columns. mutate() adds or modifies columns. arrange() sorts rows. summarise() collapses rows to summary statistics (usually with group_by()). group_by() adds grouping metadata that makes subsequent verbs operate per group. These six verbs handle the vast majority of data manipulation tasks.

Filtering rows

filter() accepts multiple conditions — commas separate AND conditions; | is OR. between(x, 1, 10) is shorthand for x >= 1 & x <= 10. near(x, target, tol = 0.01) handles floating-point equality. is.na(x) finds missing values; !is.na(x) excludes them. filter(if_any(where(is.numeric), ~ .x > 100)) keeps rows where any numeric column exceeds 100.

Selecting columns

select() uses tidy-select helpers: starts_with("q"), ends_with("_id"), contains("date"), matches("^v[0-9]"). everything() selects all remaining columns — useful for column reordering: select(id, name, everything()) moves id and name first. Negative selection: select(-col) drops a column. select(where(is.numeric)) keeps only numeric columns.

Grouped aggregation

group_by(category) |> summarise(mean = mean(value), n = n()) computes group means and counts. Multiple grouping: group_by(year, category). After summarise(), one level of grouping is dropped — use ungroup() to remove all remaining grouping. count(category) is shorthand for group_by(category) |> summarise(n = n()) |> ungroup().

Mutate with across

mutate(across(where(is.numeric), ~ . / 1000)) divides all numeric columns by 1000. mutate(across(c(x, y, z), list(scaled = scale, log = log))) applies multiple functions, creating columns named x_scaled, x_log, etc. The .names argument controls naming: .names = "{.col}_{.fn}" is the default.

Core verb semantics

dplyr’s five main verbs each solve a distinct transformation problem. Understanding which verb to use is the primary skill.

filter() selects rows meeting conditions. Combine conditions with & (and), | (or), ! (not). filter(df, x > 0 & y < 10) keeps rows where both are true. filter(df, category %in% c("A", "B")) keeps rows in either category. filter(df, !is.na(value)) removes missing values.

select() chooses columns by name, position, or type. select(df, name, value) keeps named columns. select(df, -to_drop) drops a column. select(df, where(is.numeric)) keeps numeric columns. select(df, starts_with("score")) keeps columns with that prefix. Reorder with select(df, important_col, everything()).

mutate() adds or modifies columns. New columns can reference other new columns in the same mutate() call, in order from left to right. mutate(df, rate = count / total, pct = rate * 100) creates rate first, then uses it for pct.

summarise() collapses rows to summary values. Without group_by(), it returns one row. With group_by(), it returns one row per group. The .groups argument controls whether groups are retained: .groups = "drop" removes all groups from the result.

arrange() sorts rows. arrange(df, desc(value)) sorts descending. arrange(df, group, desc(value)) sorts by group ascending, then value descending within each group.

Advanced filtering

slice_max(df, n = 10, order_by = score) selects the top 10 rows by score. slice_min() selects the bottom. slice_sample(df, n = 100) samples 100 random rows. slice_head(df, n = 5) takes the first 5 rows. These slice functions work within groups when the data is grouped.

filter(df, rank(desc(value)) <= 3) keeps the top 3 values per group when grouped. This is more explicit than slice_max() and easier to adapt for custom ranking logic.

between(x, left, right) is a shorthand for x >= left & x <= right. near(x, y, tol = 1e-9) compares floating-point values within a tolerance — more reliable than x == y for computed values.

Joining tables

left_join(x, y, by = "key") keeps all rows from x, adding matching columns from y. Unmatched rows from x get NA for y’s columns. inner_join() keeps only rows that match in both tables. full_join() keeps all rows from both. anti_join(x, y) keeps rows from x that have no match in y — useful for finding unmatched records.

by = c("x_col" = "y_col") joins on columns with different names. by = c("a", "b") joins on multiple columns (all must match). suffix = c(".x", ".y") controls the suffix added to ambiguous column names in the result.

After joins, check the result row count. left_join() can produce more rows than x if there are multiple matches in y. nrow(result) == nrow(x) is a useful sanity check when the join should be one-to-one.

Window functions in mutate

Window functions compute values relative to other rows. mutate(group_by(df, id), lag_value = lag(value)) computes the previous row’s value within each group. lead(value) gives the next row. cumsum(value) computes running totals. cummean(value) computes running averages.

rank(value), dense_rank(value), row_number(), percent_rank(), ntile(value, n) rank values. row_number() gives unique ranks even for ties; dense_rank() skips no ranks; rank() averages tied positions; ntile(value, 4) divides into quartiles.

These window functions are the dplyr equivalents of SQL window functions with OVER (PARTITION BY ...) clauses, and dbplyr translates them correctly for database backends.

Why dplyr became the standard

Before dplyr, data manipulation in R meant learning base R’s subsetting syntax: square brackets, dollar-sign access, which(), and apply family functions with their varying argument orders and return types. These tools work but require holding a lot of syntax in memory and produce code that is hard to read because the operations are implicit in the indexing syntax rather than expressed as named operations.

dplyr introduced named verbs with consistent interfaces: filter for rows, select for columns, mutate for transformations, summarize for aggregation, arrange for ordering. The pipe connects them into readable sequences. The result is code that narrates what it does. Code that says “filter to rows where sales are positive, then group by region, then summarize the total” communicates intent immediately. Equivalent base R code requires parsing indexing expressions to understand the same thing.

Tidy evaluation in dplyr

dplyr uses data masking, which allows column names to be used as if they were variables in the current environment. Inside filter and mutate, typing a column name refers to that column in the data frame, not to an object with that name in the workspace. This makes interactive code more concise but creates a challenge for programming: how do you write a function where the column name is a variable?

The embrace operator in double braces handles the common case: wrap the function argument in double braces inside the dplyr call, and the argument is treated as a data-masked expression. For selecting columns from a character vector, use the any_of or all_of helpers inside select. Understanding these two patterns covers most dplyr programming needs without requiring deep knowledge of rlang.

Grouped mutate vs summarize

Summarize reduces each group to one row. Grouped mutate adds columns computed within each group but keeps all rows. The two operations serve different purposes: summarize for aggregation, grouped mutate for within-group transformations like centering (subtracting the group mean), ranking within groups, or computing group-level quantities that apply to each member row.

After group_by, operations that would normally work on the whole data frame instead work within groups. This behavior persists through the pipeline until ungroup() is called. Forgetting to ungroup is a common source of bugs where a subsequent operation applies within groups when it should apply to the whole data frame. Calling ungroup() explicitly after grouped operations and adding it to code review checklists prevents this class of error.

Summary

You’ve learned the six essential dplyr verbs:

Verb	Purpose
`filter()`	Select rows based on conditions
`select()`	Choose columns to keep
`mutate()`	Create new columns
`arrange()`	Sort rows
`group_by()`	Define groups for aggregation
`summarise()`	Create summary statistics

These verbs form the foundation of data manipulation in R. Once you master them, you’ll find yourself writing cleaner, more readable data analysis code. In the next tutorial, we’ll explore tidyr for reshaping data between wide and long formats.

Next steps

Now that you understand data manipulation with dplyr, explore these related topics to deepen your knowledge and apply these techniques in more complex scenarios.