duplicated()

duplicated(x, incomparables = FALSE, MARGIN = 1, fromLast = FALSE, ...)

Returns logical vector· Updated May 28, 2026· Base Functions

vectorsubsettingbasededuplication

duplicated() returns a logical vector indicating which elements (or rows) are duplicates. This function is fundamental for data cleaning and analysis.

Syntax

duplicated(x, incomparables = FALSE, MARGIN = 1, fromLast = FALSE, ...)

Parameters

Parameter	Type	Default	Description
`x`	vector/data.frame/array	,	Input object to process
`incomparables`	vector	FALSE	Values to treat as incomparable
`MARGIN`	integer	1	For arrays: 1 for rows, 2 for columns
`fromLast`	logical	FALSE	If TRUE, count from last occurrence
`...`	arguments	,	Additional arguments passed to methods

Examples

Basic usage with vectors

x <- c(1, 2, 2, 3, 3, 3, 4)
duplicated(x)
# [1] FALSE FALSE  TRUE FALSE  TRUE  TRUE FALSE

Working with character vectors

The same logic applies to character vectors — duplicated() tracks string equality by value, not by position. Combining duplicated() with logical negation via ! gives you a clean way to extract only the first occurrence of each unique string, which is a common pattern for deduplicating name lists or identifier columns.

names <- c("Alice", "Bob", "Charlie", "Alice", "Diana", "Bob")
duplicated(names)
# [1] FALSE FALSE FALSE  TRUE FALSE  TRUE

# Get only unique names
names[!duplicated(names)]
# [1] "Alice"   "Bob"     "Charlie" "Diana"

With data frames

duplicated() also works on data frames, comparing entire rows. A row is flagged as a duplicate only when every column value matches a previous row. This row-wise comparison makes it straightforward to identify repeated observations in tabular data, which is essential for cleaning survey responses, log entries, or joined datasets.

df <- data.frame(
  id = c(1, 2, 2, 3, 4, 1),
  name = c("A", "B", "B", "C", "D", "A")
)

duplicated(df)
# [1] FALSE FALSE  TRUE FALSE FALSE  TRUE

Using fromLast argument

The fromLast argument reverses the scan direction. Instead of marking later duplicates, it marks earlier occurrences as duplicates and keeps the last one as the original. This is useful when you prefer to retain the most recent record rather than the first — common in time-series data where later entries are more up-to-date.

# Check from last occurrence instead of first
x <- c(1, 2, 2, 3)
duplicated(x, fromLast = TRUE)
# [1]  TRUE FALSE FALSE FALSE

Common patterns

Remove duplicates from data frame

The most practical use of duplicated() is filtering data frames to remove repeated rows. Subsetting with !duplicated(df) keeps only the first occurrence of each unique row. This base R approach is equivalent to dplyr::distinct() and works without any package dependencies, making it a portable pattern for data cleaning scripts.

# Base R approach - keep first occurrence
df_unique <- df[!duplicated(df), ]

# Using dplyr
library(dplyr)
df_unique <- distinct(df)

Find duplicate indices

Beyond filtering, you often need to locate where duplicates occur. which(duplicated(x)) returns the positional indices of duplicate elements, letting you inspect or modify specific entries. For a quick existence check, anyDuplicated() returns the index of the first duplicate without scanning the entire vector, which is faster than any(duplicated(x)) for large inputs.

# Find first duplicate index
which(duplicated(x))
# [1] 3 4

# Find all duplicates (including repeated)
anyDuplicated(x)
# [1] 3

Filter to keep non-duplicated rows

A concise one-liner for deduplication: df[!duplicated(df), ] drops every row that has appeared earlier. This pattern is compact enough to use inline in pipelines without defining intermediate variables. It preserves the original row order by keeping the first occurrence, which is the behavior most data cleaning workflows expect.

# Keep only rows that haven't been seen before
df[!duplicated(df), ]

duplicated() in practice

duplicated() returns a logical vector marking which elements are duplicates of an earlier element. The first occurrence of each value is marked FALSE; subsequent occurrences are marked TRUE. Use x[!duplicated(x)] to keep only the first occurrence of each value, this is equivalent to unique(x) for atomic vectors.

For data frames, duplicated(df) operates row-wise: a row is considered a duplicate if all its columns match a previous row. df[!duplicated(df), ] removes duplicate rows, keeping the first occurrence. df[!duplicated(df[, c("id", "date")]), ] deduplicates based on specific columns only.

The fromLast = TRUE argument marks the last occurrence as not-duplicate and all earlier ones as duplicates, the opposite of the default. df[!duplicated(df, fromLast = TRUE), ] keeps the last occurrence of each duplicate group. Combining both forms: duplicated(x) | duplicated(x, fromLast = TRUE) marks all elements that appear more than once, including both the duplicates and the originals.

duplicated() is faster than table() for simply checking whether duplicates exist: any(duplicated(x)) short-circuits on the first duplicate found. For counting duplicates per value, table(x)[table(x) > 1] shows values that appear more than once.

duplicated() marks the second and later occurrences of a value as TRUE, leaving the first occurrence as FALSE. The inverse, !duplicated(x), gives a logical index of first occurrences. With fromLast = TRUE, it marks the first and middle occurrences as TRUE instead. Combining both: duplicated(x) | duplicated(x, fromLast = TRUE) flags all occurrences of duplicate values, including the first. For data frames, it compares entire rows.

duplicated() marks the second and later occurrences of a value as TRUE, leaving the first as FALSE. Combine with !duplicated(x) to extract unique first occurrences, which is identical in behavior to unique(x) but returns a logical index rather than the values.

Setting fromLast = TRUE reverses the scan direction, marking the first and middle occurrences as TRUE and keeping the last occurrence as FALSE. Combining both directions: duplicated(x) | duplicated(x, fromLast = TRUE) flags every element that appears more than once, including all copies.

For data frames, duplicated(df) compares entire rows, all column values must match for a row to be considered a duplicate. duplicated(df[, c("id", "date")]) restricts the duplicate check to a subset of columns, useful when you want to identify duplicate keys but allow different values in other fields.

The incomparables argument lets you specify values that are never treated as duplicates of each other — duplicated(x, incomparables = NA) treats each NA as unique rather than marking subsequent NAs as duplicates.