Blog
Text tools

Remove duplicate lines from a list

Paste your list with one item per line. Every line that repeats a line already seen above it is removed, keeping only the first occurrence — the surviving lines stay in their original order, nothing is sorted.

Your data never leaves this device

Result

Duplicates removed
Unique lines kept

This page has no server side at all. The table you paste is parsed by JavaScript inside your own browser, the chart is drawn on a canvas element on your machine, and the export file is assembled locally. Nothing is uploaded, stored or written to any log. You can disconnect from the network after the page has loaded and everything will still work — which is the simplest way to verify the claim yourself.

Facts and limits of this method

Which copy survives The first occurrence of each line; every later repeat is deleted
Order Preserved — the tool never reorders lines, only removes some
Case handling Case-sensitive by default: "Apple" and "apple" count as different lines
Whitespace Leading/trailing spaces are ignored by default when comparing
Empty lines Treated as a value like any other — only the first blank line is kept
Sorting None. To sort and drop duplicates in one pass, use String Sorter's dedupe option

When it misleads you

How it is calculated

The text is split into lines. Each line is checked against every line seen so far during this same pass, in order from top to bottom.

A comparison key is built from each line: trimmed of surrounding whitespace when that option is on, and lowercased when case-insensitive is on. Lines with the same key are considered duplicates of each other.

The first line to produce a given key is kept and added to the output; every later line producing the same key is skipped and counted toward "duplicates removed".

Because the scan is a single left-to-right pass with a lookup of previously seen keys, the result always contains exactly one line per distinct key — never zero, never more than one.

Questions and answers

Which duplicate is kept — first or last?

The first one. If "apple" appears on line 2 and again on line 7, the copy on line 2 survives and line 7 is removed.

Does it also sort the list?

No, order is left exactly as you typed it. If you want a sorted, duplicate-free list, use String Sorter and turn on its "remove duplicates" option.

Are "Apple" and "apple" treated as duplicates?

Only if you turn on case-insensitive comparison. By default they're kept as two separate lines.

What about a line that's just spaces?

With whitespace trimming on (the default), a line of only spaces and a genuinely empty line compare equal, so only the first of them is kept.

Is my list sent anywhere?

No. Everything runs in your browser; nothing is uploaded.

Related tools

All tools