Wizard Toolbox Tool map

Remove Duplicate Lines

Paste a list, one item on each line, and every repeated line goes. The first of each stays, in its place.

0 lines · 0 characters

Lines

Change case

Spaces

Find and replace

What counts as a duplicate

Two lines are duplicates when they match exactly, letter for letter. The first one stays where it was and the later copies go, so the order of your list does not change, and the page says how many lines it removed. Apple and apple are different lines, and so are apple and apple with a space after it.

Catching near-duplicates

Press Trim each line first to drop spaces at the start and end of every line, and lowercase first if capitals do not matter, as with email addresses. Then press Remove duplicates. Remove empty lines, next to it, clears out the blank lines a pasted list often has.

Questions

How long a list can it take?

Tens of thousands of lines take well under a second; the limit is what your browser is happy to paste into a text box.

Can I see which lines were duplicates?

Not as a list. The page only says how many it removed. Press Undo to get the original back.

Can it sort the list too?

Yes. Press Sort A–Z after removing duplicates. Sorting first would also put copies next to each other, but it changes the order of the list.

Worked out in this page, on your device · nothing is sent anywhere