Need a Mac App for Finding Duplicate Lines in a Text File

[new2macs]

Hello and thanks in advance.

I just switched to Mac -- and love it! -- but I'm trying to find some basic applications to do things I could do in Windows.

I run my club's large mailing list, and I need an application that can find duplicate lines in one file. (I want to save these duplicate lines, not automatically delete them.)

Does anyone know any good Mac apps that can do this?

Thanks very much,

New 2 Macs

[MacHeadCase]

Welcome to MacApper forum, new2macs!

Perhaps this app could be of use to you, take a look at Apimac's Clean Text (http://www.apimac.com/clean_text/). Hopefully you can ask the app to highlight the duplicate lines and not delete them.

Or maybe Bare Bones Software's TextWrangler (http://www.barebones.com/products/textwrangler/) could do the trick when you use the Find tool. Unless this would be something for TextWrangler's big brother, BBEdit...

[MacHeadCase]

If you decide to go the BBEdit route, take a look at this blog entry, BBEdit AppleScript: Improved Duplicate Lines and Select Lines (http://www.entropy.ch/blog/Mac+OS+X/2008/05/25/BBEdit-AppleScript-Improved-Duplicate-Lines-and-Select-Lines.html).

Not sure if this would work in TextWrangler.

Hope this helps. :)

[Kilted]

http://www.smileonmymac.com/TextExpander/index.html

TextExpander from "Smile on my mac" is good too :)

[mathogre]

Here's the simple way, assuming the file is in plain text. Open a terminal and type:

sort <filename> | uniq -c | sort -k 1,1rg

where filename is the complete name of the file (without the brackets), including its path (directory). This will only pick out exact matches, so you'd need to add some flexibility to your search to allow for various differences on "duplicate" lines.