I’ve focused a lot over the last few months on helping readers to speed up and optimize Apple’s Macs — everything from adding RAM to recovering hard drive space and upgrading old hard drives to faster SSDs. Today’s How-To is focused on something very specific but with a lot of optimization potential: trimming down your Mac’s photo library.
Particularly after installing OS X 10.10.3 with Apple’s new Photos app, you might be surprised to learn that you’ve lost a lot of hard drive space, and that there are suddenly tons of duplicate photos on your Mac. After installing OS X 10.10.3, the new Photos app converted my 90GB Aperture library into a 126GB Photos library, and left both on my hard drive. That’s an incredible amount of wasted space attributable to duplicates, so it’s no surprise that a $1 utility called Duplicate Photos Fixer Pro has recently become the #1 paid Mac App Store app, while a superior alternative called PhotoSweeper ($10) is in the top 50. I’ve used both apps, as well as many others, and can help you choose the one that’s best for your needs…
Gauging The Size Of Your Mac’s Problem
If you’re not sure just how much space your photo collection is consuming on your Mac, there are two ways to figure it out. The simplest technique is to open a Finder window and right-click the Pictures folder under Favorites, choosing Get Info. A window will pop up with a number, which in my Mac’s case was over 236GB between a number of different photo libraries — Apple’s Photo Booth, Aperture, iPhoto, and Photos, just to name a few. If you do the math relative to your hard drive’s size, you’ll understand how much photo libraries are weighing down your computer: they were consuming around 1/4 of my iMac’s 1TB of drive space.
I personally like to see disk information presented visually rather than mathematically. GrandPerspective, the free disk space mapper I recommended last week, quickly shows just how out of control the photo libraries have become: my OS X Photos and Aperture libraries are the two huge blocks on the left, consuming 1/3 of all used space on the drive. If your libraries are this big, you have a lot of potential to recover hard drive space.
But even if your photo library is smaller, there’s a very good chance that there are duplicate images inside, swelling what could be a manageable collection into something needlessly bigger. The duplicate eliminating tools below turn what could be days of agonizing hunting-and-pecking into a mostly automated hour or two of deleting unneeded images.
Exercise Extreme Caution
Before going any further in this How-To, read this carefully:
Be very careful (yes, that’s bold, underline, and italics all at the same time) before deleting any of your “duplicate” photos, particularly in bulk. One of these libraries may be the only record you have of precious memories, and making the choice to just dump a giant collection of images can be calamitous. That’s why lost photo recovery tools are almost as popular as duplicate removers.
Exercising caution when using duplicate photo tools can be challenging. Each app makes certain assumptions (with your guidance) as to what should be called a “duplicate.” You can make those assumptions strict, or loosen them to catch images that are extremely similar. The looser your rules are, and the less you manually manage the list of duplicates before hitting the “Delete All” button, the greater the chance you’ll accidentally delete something worthwhile. It’s tempting to dump as much as possible when you’re presented with the opportunity to reclaim gigabytes of lost space by just hitting a button, but be sure you’ve given everything a look-over, first.
The same warning applies if you’re thinking of deleting an entire previous photo library. If you’ve already decided to switch from iPhoto or Aperture to OS X Photos or Adobe’s Lightroom 6, you could trash your old iPhoto/Aperture photo library and use Photos or Lightroom’s new library for everything going forward. My advice: move your old library in its entirety over to a high-quality external hard drive and keep it there until you are absolutely certain that Photos or Lightroom imported everything correctly. That way, you have the option of going back and grabbing lost images if needed, but aren’t cluttering up your main hard drive with an old photo collection that’s almost entirely duplicates.
PhotoSweeper
At $10, Gwinno Software’s PhotoSweeper isn’t the cheapest of the duplicate photo removal tools out there, but in my experience, it’s the best. With the recent release of version 2.1, PhotoSweeper (unlike cheaper rivals) can find duplicates within OS X Photos libraries, as well as iPhoto, Aperture, and Lightroom libraries, plus images that are sitting in random photos on your hard drives. It has a reasonably straightforward but powerful user interface, which begins by giving you a Media Browser showing any Photos, iPhoto, Aperture and Lightroom libraries on your machine, letting you decide which to drag over into the main window for processing. If you only want to look at a portion of a given library, you can, or you can bring multiple libraries and folders together for comparison.PhotoSweeper will take some time (varying based on the speed of your Mac and size of your photo collection) to canvas everything, then will present you with a “What to Find?” window. This is one of PhotoSweeper’s best features, as you can pick from four main options: “Duplicate Files” (shown above), “Similar Photos: Basic,” “Similar Photos, Advanced,” or “Series of Shots.” Each one will process your photo library differently. Duplicate Files looks strictly for pure duplicates, and has only one option: “Compare photos with the same name only.” Leave it unchecked and PhotoSweeper will look at each photo’s checksum, presenting you only with 100% identical files; check it and you’ll only see 100% identical files with the same name.
While Duplicate Files is basically foolproof, Similar Photos loosens the rules. Under Basic Settings, Similar Photos will let you use fuzzy logic — image analysis on a scale from “partial” to “exact,” plus time comparisons and your choice of either same size or same aspect ratio — to find images that are close to one another. This is an ideal tool to use if your library somehow contains full-sized original images (the ones you probably want to keep forever), slightly modified versions, and/or smaller thumbnails that were created by another app. Using the “same aspect ratio” setting, PhotoSweeper can look for images that are the same overall but in larger and smaller versions. The Advanced Settings tab lets you calibrate the comparison engine, looking for matchups either of thumbnails or histograms, as well as color and detail levels in the thumbnails.
Series of Shots is the last option, included to handle split-second bursts of photos like the ones taken by recent iPhones, or looser groupings of images taken at nearly the same time. This option shows you clusters of images taken close together — you set how close — and lets you pick the one you prefer without looking for a 100% exact match. It’s handy to get rid of not-quite-as-good “near-duplicates” that might be taking up a ton of space in your library.
Once you’ve selected your sorting scheme, PhotoSweeper will go through everything and create “groups” of photos that it has identified as being similar. Each group will contain at least two photos, so you can either trust PhotoSweeper to automatically mark the ones it thinks should be deleted, or manually mark them yourself. PhotoSweeper has a list of 20 Auto Mark rules hidden within its Preferences menu, so you can check the rules you want to use, put them in the right order, and run the marking process. For instance, if you simply want the app to auto-mark the file with smaller size, DPI, or dimensions, you can do any of those; you could also auto-mark files with lower star ratings, fewer keywords, or other metadata criteria.
Auto marking saves you from having to manually choose the image to delete from each group yourself, but should always be manually audited before moving on. That manual audit — checking as many images as you want to make sure you’re not tossing out good files — is your most critical step. When you hit the Trash Marked button, all green images will stay, and all red ones will disappear. You may need to let PhotoSweeper automatically open the photo library’s app (as with OS X Photos) so it can complete the deletion process.
Using PhotoSweeper in the right way can safely free up a tremendous amount of space on your Mac; you can run it several times using different “What to Find?” options to eliminate different types of duplicates. Even though you may get tired during the manual audit process — and might just opt to trust that PhotoSweeper is picking the right duplicates to eliminate — bear in mind the cautionary details above so that you don’t lose any important files.
Duplicate Photos Fixer Pro
Although it’s currently at the top of the Mac App Store charts due to its temporarily low $1 price, Systweak Software’s normally $19 Duplicate Photos Fixer Pro has some serious limitations compared with PhotoSweeper. It won’t import OS X Photos, Aperture, or Lightroom libraries — only “iPhoto 9 or newer” and individual photos. You can choose from two settings: “Exact Match,” or “Similar Match,” and you’ll see an ad every time you don’t find duplicates.
Duplicate Photos Fixer Pro has far fewer settings to tweak its matching engine. It generates and compares only thumbnails, not histograms, and can use time intervals, GPS proximity, and a sliding scale “matching level” to sort your photos. At its weakest matching level, you’ll see “duplicates” that are clearly substantially different shots; at the strongest matching level, images will need to be pixel-perfect clones to be IDed. Even after doing the initial comparison, you can adjust the Matching Level slider and see just how many shots would be considered duplicates at each of the slider’s levels.
Like PhotoSweeper, you can auto-mark images using rules, though there are only six rules here, and they’re not as useful. Hitting the Trash Marked button lets you purge duplicates after a quick warning, with a reminder that you can find any accidentally deleted images in the trash can.
Having used PhotoSweeper quite extensively with Aperture and OS X Photos libraries, it’s hard for me to recommend Duplicate Photos Fixer Pro given that it’s so much more limited — fewer types of photo libraries are supported, the duplicate finding and auto marking features are less powerful, and there are other little annoyances in the user interface. At its normal $19 price point, there would be zero reason to prefer it over PhotoSweeper. But if you’re only using iPhoto or loose photo folders, want a budget solution, and don’t mind doing searches that mightn’t net as many actual duplicates as PhotoSweeper finds, this could be worth a try.
One More Option: Duplicate Detective
Although I wouldn’t generally recommend it for photographs, there’s another handy app for getting rid of duplicate files: Fiplab’s $3 Duplicate Detective. This app can be used across numerous types of files, quickly finding exact matches and letting you purge them. You simply point Duplicate Detective at a folder (or hard drive) full of files, start the scan, then pick the files to delete using simple (oldest/newest) criteria.
Why include Duplicate Detective in this article? Depending on how you’ve organized your photo libraries, you may have stored your iPhone, iPad, or iPod home videos along with your photos, or separately on your hard drive. These video files (particularly slow-motion and 1080p HD videos) can be gigantic, and if you want to hunt for duplicates of them or any other non-photo file, Duplicate Detective can help.
More Great Ways To Improve Your Mac
To make the most of your Mac (or pretty much any other Apple device), I’ve written quite a few How-To and Best of guides, as well as reviews of worthwhile accessories. Read more of my guides and reviews for 9to5Mac here (and don’t forget to click on Older Posts at the bottom of the page to see everything)!
FTC: We use income earning auto affiliate links. More.
Great post, PhotoSweeper is exactly what I need. I’d been planning to look to see if such a thing existed to fit my needs as soon as I get a week to go through a decade of photo libraries I’ve let fall into disrepair.
+1 for Photosweeper. I recently used it to clean up my sister-in-law’s Photos library (100K photos). Even at $10, this application provided excellent value. I tried a number of other duplicate detectors before stumbling across this one. They all pale in comparison. PhotoSweeper was so much faster and had excellent refinement controls.
Yes, hefty applause from me, as well. 👏🏼 I did pure dupes and similar files, checking thrpugh the results just to be sure. I just got back almost 20GB of space. Yes, it was that bad.
Isn’t it nice to get back 20GB of space?
Migration to Photos doesn’t duplicate your files, just leaves a reference to them in the iPhoto or Aperture library – the size of the libraries is misleading – https://support.apple.com/en-gb/HT204476
It’s hard to know what’s really going on with my iMac’s photo library for two reasons: first, my libraries are referenced (as discussed a little more below), and second, I could understand if both Finder and GrandPerspective are reporting the relative sizes of the libraries inaccurately, but the total amount of free space on my drive appears to be consistent with duplicated rather than overlapping libraries.
I thought it referenced it too. But then I moved the older Aperture library to another external drive, unmounted it, and the Photos library was still there… I don’t know how it works really, and what type of decision will Photos make to keep a referenced library, or make a duplicate of it.
And I must have had a duplicate, otherwise disconnecting the drive would have resulted is seeing no pics in Photos (full size, I mean).
And if you have a referenced Photos library, you will have to live with 2 “libraries” on your Mac forever?
What is the logic behind Referencing vs Duplicating? available disk space? (I had 700GB free, when upgrading to Photos, my Aperture library was about 90GB and the newly created Photos library a tad more than that).
The new Photos library uses “hard links” to the files on disk. In this case, iPhoto, Aperture and Photos all have libraries pointing to the same files. Delete one or two of those libraries, and the third library still points to the files on disk. There’s no duplication happening, just links.
The visual nature of the Finder may lead you to think that the photos live “inside” those libraries, but that’s just a representation. The file system looks at how many links are associated with each file, and when that reaches zero is the file actually deleted from disk.
Are you SURE you have duplicate photos, or are you just witnessing the symptoms of the use of hard links?? Photos uses hard links so that your photos are NOT duplicated. You might see a 80GB iPhone library and a 100GB Photo library, but this does NOT mean you are using 180GB worth of space!!!! You might only be using 110GB of space, because of the overlap of hard links. Be careful!!!
What are hard links? Just two hard references to the same file on the disk. If a file has two hard links, and you delete one, the file remains intact and NO space is freed. Delete the last hard link, and the file is removed.
The problem I’m personally seeing (of two separate large libraries) may be a symptom of the way Photos handled migration from a referenced (and notably much larger) Aperture library. In this particular case, there are unfortunately two separate library files that are occupying large blocks of space on my iMac.
The question that remains is, if I have an iPhoto Library, an Aperture library, and a Photos for Mac Library, how will PhotoSweeper handle the hard links? Will it recognize them as both referencing the same file, or will it think of them as two separate files and try to delete one of them?
I lost thousands of photos during the transition from iPhoto to Photos app in Yosemite. I must have had some corruption in the iPhoto DB, as over the last year I’d have sets of photos show up with a generic thumbnail and when opened it’d be blank. BUT, I could still find them in the iPhoto library on the HD. Thought it’d be taken care of during the upgrade to Photos but now those photos can’t even be found in the Photos library on the HD…so I think I’m screwed.
Anyone else have this issue, and can think of a possible resolution?
Right click on your iPhoto library and Show Package Contents. Browse the contents for a folder such as Masters. Either copy the entire collection of Masters onto your desktop and re-add everything to Photos, running PhotoSweeper afterwards to delete duplicates, or locate the specific images within the folders you find inside and copy them into your Photos library. Hope that helps!
You didn’t have a backup? Time Machine?
I can heartily recommend PhotoSweeper as well. I have used the Photos app-compatible version of it since its beta was kindly made available to me by its creators and the recently released final version works without a glitch. It is easy-to-use and (in my experience) reliable. Well worth the $10.
I would recommend people that are importing their photos into the new Photos app to create a Smart Album that shows all the referenced images first and resolve any issues with referenced files before continuing with removing duplicates.
I’ve tried Photo Sweeper but I found it came back with too many false positives. I bit the bullet and paid for PowerPhotos (which also lets you manage photo libraries, fwiw) and I’ve found its configurable parameters better suited.
Tuning the advanced settings to look for more detail reduces the number of false positives. :)
I’m concerned there might be a fundamental misunderstanding in the premise of your article. You might have had a 90gb iPhoto library and created a 126 gb photos library. But that doesn’t mean you’ve used another 126 gb of your storage space–or even any at all. The second photo library uses hard links; the photos are NOT duplicated. I’ve nothing against deleting “duplicates” but what really matters to your anecdotal analysis is how much space remained before you created the photos library, and how much after. Then, backup the (old) iPhoto’s library to an external drive and then, again, see how much space remains.
See above.
I looked at the new Photos app in Yosemite to see what it does with libraries from iPhoto and Aperture. The look is more like iPhoto but I felt I had less control over my organization of the individual photos themselves. Where I can place them within a group which I can do with Aperture but can’t with iPhoto unless I change a name or date or some other metadata that is listed in the menu for sorting. In Aperture I use a referenced library on an external drive so I can duplicate that drive for safety. Also, Aperture’s duplicate photos has other levels of the same photo including thumbs and duplicated images used for display when I want to view a larger version of the thumbnail. These could be the source for many duplicates that others are finding with these apps so viewing the data carefully for dimensions and files size is important. In iPhoto I have all my photos within the app library system. The reason for any permanent upgrade would be that it offers better organization and more adjustment controls. I thought “Photos” was going to have this. I was disappointed on both fronts. The adjustments on the right of the screen had such short lengths that using a slider approach was very coarse even if I could use the 1 click at a time while holding another key down. This defeats the purpose of easy use. It is still easier to find duplicates within Aperture with the simple find using specifics that are already included and combining them. Then by looking at the metadata on the left as I select the photos I can easily see which are duplicates or ones that are adjusted that I want to keep for a specific purpose. I can’t do that with “Photos”. Maybe, I have been using iPhoto and Aperture for such a long time ( since they first came out ) , I know them very well and can easily do whatever I need to do. But, even with that knowledge working with “Photos” is more UI unfriendly to get the refined result I am used to for adjustments or organizing. I am also afraid that it would be too easy to duplicate photos, at least in the beginning if I were to start using “Photos” with the cloud. My iPhone sharing would automatically be creating real duplicate files on 3 other devices and then I would have to individually shut down that feature so I wouldn’t go crazy if weeding out the duplicates on all those other streaming devices were to begin. Their new “convenient” feature for sharing photos has too many settings with all the other desktops and laptops that I might be using. So keeping it simple with 1 phone and 1 main central place and 1 main application is so much easier to archive and make safety duplicates using Carbon Copy Cloner so I know where all my photos are. The bottom line is that I am not going to upgrade to Yosemite for “Photos” and will keep Mountain Lion for Aperture and iPhoto when I need that app. I can still stream from my iPhone directly into Aperture through the cloud but nothing is stored on the cloud. Usually, I will just transfer the photos as I do with my Nikon directly to my library. Upgrading is not always the best way to be more efficient even if it has all those extra features that are supposed to make is easier in this new Cloud world we live in. Sometimes it can make things go very wrong. So be sure to back up everything if you decide to go the upgrade route in case you want to go back.
I just wanted to second your thinking on keeping mountain lion so you can keep Aperture and IPhoto and not going past 10.8.5…..I did it for the exact reasons you did….but I am finding apps are written for 10.6 and later and I am being left behind….so I guess I am ready to decide between photoshop and Lightroom….which will also let me use photo editing on both mac and PC. I don’t know how apple can just make these sweeping no optional changes like this.
I didn’t realize that I had gotten a comment about my comment till tonight. I bounce around a lot and some sites don’t let me know that there’s a reply there. So, let me start my saying that I am still using OSX 10.8.5 on my Mac Pro 2010 model. There’s lots of storage with 4 drives & I didn’t want to keep upgrading libraries for Aperture ( still using 3.2.4 ). I tried 3.3 next but there were some changes in adjustments (lost older PS style ) to a more general look ( same style in the Photos app where Apple wanted to keep it simpler for the general public ). It had softer shadows using the Shadows/Highlight adjustment. I still couldn’t get to the same look on a photo with 3.3 even trying other separate adjustments together. I also agree that there should be more backward compatibility when apps have small updates…the library should work the same way with only background changes made for efficiency, etc. I didn’t need any of the changes that were being made for the OS upgrades or for most applications. If it ain’t broke, don’t fix it. I do have a Mac Mini that has Yosemite which I do my tests on and have upgraded Final Cut Pro X to the newest version which works on it. Photoshop is still CS6-no need for the monthly Cloud version updates and payments. I like the one time purchase, not subscriptions. PS & Aperture still do their jobs well. Only Safari gives me some problems when using Facebook ( some freezes ) & some websites tell me that the site may not work as well with my current version-6.2.8 [ can’t go above that for 10.8.5 ]. If I do have problems, I use Firefox. I have not upgraded to the new Mac Pro with no immediate need at this time, although I do have a couple of the newer backup drives with Thunderbolt v2. These are connected to the Mac Mini when I use FCPro X. Any Qs, contact me at stevelacey2@me.com
This might be a stupid question, but are duplicate images removed by PhotoSweeper also removed from iCloud? Meaning that the duplicate images disappear from all by devices?
They are moved to an album called Trash (PhotoSweep), then you go into Photos App and delete the photos
Another nice How-to would be to show how to reduce size of older videos made with cameras of a few years ago shot in Motion-JPEG or another ineffcient (compared to today’s) CODECs.
A somewhat automated way to treat these type of videos, which also land into photo libraries, would be great and save great amout of space.
Thank you. This is exactly what I needed.
Excellent article…my wife has been haranguing me for 2-3 weeks to deal with our exploding photo library, currently filled with many dupes. I had no idea where to start. I appreciate the deliverance!
Good article, but we can’t let Apple off the hook for this.
Disappointed with Photos so far. Not worth the upgrade.
My photos used to be in order by title, and now that appears to have been killed.
Organising them by date is now the Apple default, which for me is useless.
really good. thnx
GREAT REVIEW !!!. CleanSweeper was exactly was I was looking for. Any ides of an App that GeoTag’s our photos AND works together with Photos App, just as CleanSweeper does?
i already have Duplicate Annihilator for iPhoto. wondering if $5 is worth the upgrade to Duplicate Annihilator for Photos or if the two reviewed here are better.
anyone have any combative experience to guide me?
Thanks for always sharing us the useful reviews. PhotoSweeper seems a pretty good tool to specially delete those photo duplicates. But I want to recommend another powerful tool-MacClean, it can not only detect and remove duplicates photos, but also all of the duplicate files in Mac. Besides this kind of function, it even offers a comprehensive clean solution to free up Mac OS X space and speed up Mac, and will always make your MacBook run like new. And what’s more, it’s totally free.I highly recommend it to those who have the same demand.
Thanks for always sharing us the useful reviews. PhotoSweeper seems a pretty good tool to specially delete those photo duplicates. But I want to recommend another powerful tool-MacClean, it can not only detect and remove duplicates photos, but also all of the duplicate files in Mac. Besides this kind of function, it even offers a comprehensive clean solution to free up Mac OS X space and speed up Mac, and will always make your MacBook run like new. And what’s more, it’s totally free.I highly recommend it to those who have the same demand. For more info: http://www.imobie.com/macclean/
Excellent review and information. I’m disappointed in PhotoSweeper, however, because it does not seem to actually work with the Photos app. It created an album with all the duplicates in it and told me to trash that album. I did, but the duplicates remained. The Photos app only allows photos to be deleted from the “All Photos” album or from “Photos” at the top of the sidebar. Deleting that album did nothing. Did I miss something?
PhotoSweeper’s support helped me out. You need to open the “Trash (photosweeper)” album, select the photos, and delete them from there. They will then move to your recently deleted.
Great Post.
I would also make this suggestion.
In addition to making a copy of my precious photos and putting them on a hard disk,
For $5.99, you can have automatic upload to the cloud via CrashPlan. I say this for a number
of reasons, but one in particular.
Whenever there’s coverage of the aftermath of a bad tornado, CNN shows people returning to
what’s left of their houses,
when the reporter asks what they’re looking for, invariably the people say, “Photos.”
So I’d like to have my photos on a hard disk AND offsite in the cloud.
Disclaimer: I have no association of any kind with CrashPlan. And instead of that suggestion, you can certainly copy your
photos to Box when they’re giving away 50 gigs, which I grabbed immediately. Their software syncs your files automatically.
Similar Disclaimer: I have nothing whatsoever to do with Box.
Best,
Evan
You might want to be very careful with “Duplicate Photos Fixer Pro” as it has nothing of Pro, its just the same “Photos Duplicate Cleaner” with another name. The exact interface and all. One of the criterias of the Auto Mark feature is that it will assume that the oldest picture is the one that should be deleted if there are duplicates of it. From my point of view this is wrong, as always you would like to keep the oldest version if the quality is the same, as it is the most accurate reference of the date you actually took the picture.
Their support is bad, only phone support based in India (the software is developed by a company sister or within a company called “The Phone Support”, they do not offer the possibility to send emails or contact form with print screens to show them what the problem is. I tried calling, after 3 escalations all within their same office, I honestly gave up.
For me, I will be checking the PhotoSweeper app as I don’t trust the Duplicate Photos Fixer “Pro”.
Hi, thanks so much for this post! It was a delight to find 70 GB of duplicates on my harddrive with the help of PhotoSweeper. But (and I realise you’re not a helpdesk, so ignore if this is inappropriate here) after deleting the photo’s (and off course deleting them from the Trash-cans of iPhoto and Photos) I still have the same 4 GB available on my harddrive. While PhotoSweeper is saying that there are no more duplicates. Where did I go wrong..?
Thanks. I used the newest version of duplicate photo fixer pro and removed over 1500 duplicates which I’m happy with. But now my icloud storage is still reaching it’s maximum size based on my 50GB plan. I can of course increase my storage plan, but that seems illogical given that icould has triplicates of my photos from the original upload. So, how do I remove the doubles in icould now that I’ve removed them from my photos on my imac? I’m so confused!