Remove duplicate lines
Remove repeated lines from a list and keep the first of each. The text is processed in your browser and nothing is uploaded, so you can clean e-mail lists, customer names and keyword lists safely.
How it works
-
Paste your text
Type or paste the text in the left box, or open a .txt file. No text to hand? Click Example.
-
Instant result
The result appears as you type. Change the options and see straight away what changes.
-
Copy or download
Copy the result with one click, save it as a file or use it as input for a next step.
Example
apple, pear, Apple, banana, pear and apple become apple, pear, Apple and banana: two duplicates are removed (the second apple and pear with a trailing space). With Ignore case, Apple goes as well and three are removed.
Options
- Ignore case:
Appleandappleare the same. - Ignore spaces at the start and end: on by default.
- Keep empty lines: on by default, so paragraphs stay apart.
Why not sort first?
Sorting with removal of duplicates also works, but it changes the order. This tool keeps the lines in the order you gave them, which matters for ranked lists and for lines that follow a logic, such as steps.
Where it helps
- E-mail and phone lists merged from several sources.
- Keyword and tag lists for SEO or shops.
- Log lines and IDs where the same entry repeats.
To compare two lists and see what differs, use compare text. To strip the gaps afterwards, use remove empty lines.
Frequently asked questions
Which of the duplicates is kept?
The first one, and the order of the lines stays as it was. A list of `apple`, `pear`, `Apple` and `apple` loses the last `apple` and keeps `Apple`, unless you tick *Ignore case*.
How do I treat Apple and apple as the same line?
Tick *Ignore case (Apple = apple)*. Then only the first of them is kept.
What about spaces at the end of a line?
By default *Ignore spaces at the start and end* is on, so `pear` and `pear ` count as the same. Untick it for an exact comparison.
Are empty lines removed?
Not by default, because they often separate groups. Untick *Keep empty lines* to treat blank lines as duplicates and keep only one.