by()

by(data, INDICES, FUN, ..., simplify = TRUE)

by() is a base R function that applies FUN separately to each subset of a data frame split by one or more factors. The official documentation calls it “an object-oriented wrapper for tapply applied to data frames”, which is the cleanest way to think about it: when tapply() only sees a vector, by() hands your function the entire per-group data frame, so any column is in reach.

Reach for by() when each call should operate on the rows belonging to a group as a small data frame. Common use cases include fitting a model per group, computing a correlation across two columns, or summarising a multi-column block. The function lives in the base package, so it is always available without library() calls, though examples built on warpbreaks need require(stats) because the dataset does.

Signature and arguments

by(data, INDICES, FUN, ..., simplify = TRUE)

Argument	Type	Description
data	data frame, matrix, or coercible object	Normally a data frame. Matrices are coerced; other objects are coerced and FUN is applied separately to each column.
INDICES	factor, list of factors, or one-sided formula	Length must equal nrow(data). The data-frame method accepts a formula like ~ tension that is looked up inside data.
FUN	function	Applied to each subset. Receives the per-group data frame as its first argument.
...	any	Extra arguments forwarded to FUN on every call.
simplify	logical, default TRUE	If FALSE, the result is always a list. If TRUE, the result is an array when all FUN returns have the same length, otherwise a list.

The return value is an object of class "by", which is a list (with a dim attribute when array simplification kicks in). Drop the class with unclass(res) or coerce to a data frame with array2DF(res).

Example: summary statistics per group

The simplest case is a one-column summary, and here by() overlaps with tapply(). The difference is mostly cosmetic at this level: tapply() is the tidier spelling when you only have a single vector to summarise, while by() is the one that scales up once FUN needs the whole data frame.

df <- data.frame(
  group = factor(rep(c("A", "B", "C"), each = 5)),
  score = c(8, 7, 9, 6, 8, 5, 6, 7, 4, 6, 9, 10, 8, 9, 10)
)

by(df$score, df$group, mean)
# A  B  C
# 7.6 5.6 9.2

Example: function that needs multiple columns

Pass the whole data frame and FUN receives the per-group subset as its first argument. This is the use case tapply() cannot cover, because tapply() always hands your function a plain vector pulled from a single column. With by(), the function closes over the rest of the data frame for free.

by(df, df$group, function(sub) max(sub$score) - min(sub$score))
# A B C
# 3 3 2

The data-frame method is what makes this work. sub is a real data frame, not a numeric vector, so any column on the right-hand side stays available. Anything you can do in an lapply() over a split() call, you can do inside the function here.

Example: a linear model per group

The canonical ?by example fits a separate lm() per tension level and pulls the coefficients back into a matrix. This is the pattern to reach for when each group deserves its own statistical model (different intercepts, different slopes, different diagnostics) and you want all the model objects in one place for later inspection or comparison.

mods <- by(warpbreaks, warpbreaks$tension,
           function(x) lm(breaks ~ wool, data = x))

sapply(mods, coef)
#                L        M        H
# (Intercept) 31.03704 25.25926 18.14815
# woolB        5.37037  1.18519  2.40741

mods itself is a by object holding three fitted lm instances, which you can index, summarise, or convert to a data frame for later analysis. Each element is a full model object, so summary(mods[["L"]]) and friends work as expected.

Wrap the call with array2DF() to land directly on a tidy data frame, which is usually what you want for downstream plotting or joining back to the original data. The whole expression stays a one-liner once you have the formula indexing pattern in your head, because by() does the looping and array2DF() does the reshaping.

by(warpbreaks, ~ tension, with, coef(lm(breaks ~ wool))) |>
  array2DF(simplify = TRUE)

The one-sided formula ~ tension is shorthand for warpbreaks$tension, and the data-frame method handles the lookup for you. with is a function used as FUN, and the trailing expression is evaluated inside each subset via non-standard evaluation, which is why the model formula can refer to breaks and wool directly without naming the data.

Example: forwarding extra arguments

Anything you pass after FUN reaches it on every call. To compute per-group quantiles without writing a wrapper function, just pass the extra arguments through the dots. This is the same forwarding mechanism lapply(), sapply(), and mapply() all use, so the pattern should feel familiar.

by(df$score, df$group, quantile,
   probs = c(0.1, 0.5, 0.9), simplify = FALSE)
# $A
#  10%  50%  90%
#  6.4   8.0  8.8
# $B
#  10%  50%  90%
#  4.4   6.0  6.8
# $C
#  10%  50%  90%
#  8.2   9.0  9.8

simplify = FALSE guarantees a list even when groups return the same shape, which keeps the printout clean. Without it, the result would still simplify to an array because quantile() returns the same six-element vector in every group, but the list form is easier to read and easier to feed to do.call(rbind, ...) if you want a single data frame.

Example: a two-way split

INDICES is not limited to a single factor. Pass a list of two (or more) factors and by() returns a multi-dimensional result, where each cell corresponds to one combination of group levels. This is the same shape you’d get from tapply(), so the printout looks like a small contingency table of group means, useful for spotting interactions between two categorical variables without reshaping the data yourself.

by(warpbreaks$breaks,
   list(warpbreaks$wool, warpbreaks$tension),
   mean)
#   : A
#   : L    M    H
# : A 44.55556 24.00000 24.55556
# : B 28.22222 28.77778 17.77778

The first list element maps to the columns of the result and the second maps to the rows, matching how tapply() orders the dimensions. Swap the order of the factors in the list and the result transposes.

Working with the result

The object you get back is a by object, not a plain list, which is worth knowing because it changes how it prints and how you can poke at it. The custom print method is what produces the labelled output in the examples above, with one block per group. If you want the raw list, drop the class.

res <- by(df$score, df$group, summary)
unclass(res)
# $A
#    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
#    6.00     7.00     8.00     7.60     8.00     9.00
#
# $B
#    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
#    4.00     5.00     6.00     5.60     6.00     7.00
#
# $C
#    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
#    8.00     9.00     9.00     9.20    10.00    10.00

$ and [[ subsetting work as expected because the result is still a list under the class. To land on a data frame, use array2DF(res) on R 4.x or as.data.frame(res) on older releases. For a quick visual check, plot(res) dispatches a method that produces a lattice-style panel per group, which is occasionally exactly what you want without any further work.

Common pitfalls

• Don’t reach for by() on a plain vector. The default method coerces the object and then applies FUN column-wise within each group, which is rarely what you want. For vectors, use tapply().
• simplify = TRUE is the default, but it can surprise you when FUN returns differently-shaped objects. Default to simplify = FALSE for custom functions until you know the shapes match.
• The result is class "by", not a plain list. Use unclass() to drop the class and array2DF() (or as.data.frame() on older R) to coerce to a data frame.
• The data-frame method accepts a formula in INDICES; the default method does not. Pass the factor explicitly when data is a matrix.
• Order of output elements follows factor level order, not first-seen row order. Mind this when joining results back onto the original data.
• An empty group shows up as NA when simplify = TRUE and as integer(0) (or list()) when simplify = FALSE. Either is a real value, not a bug.

How by() compares to neighbours

tapply() is the function by() wraps, so use it on a vector when you want a matrix back. aggregate() returns a data frame and is the closest base-R spell for grouped summaries across multiple variables. split() combined with lapply() gives you total control over the per-group call. In the tidyverse, dplyr::group_by() |> summarise() covers most of what people use by() for, and dplyr::group_modify() is the closest direct analogue when you want the function to see the full per-group data frame. The trade-off is mostly about ergonomics: by() is concise and ships with base R, while the tidyverse versions return a tibble and compose with the rest of the dplyr pipeline.

See also

• /reference/base-functions/tapply/
• /reference/base-functions/apply/
• /reference/base-functions/lapply/