CESTIS: The fire can't hurt you, can it? You're no more bound to that mortal form than I am. You're pure spirit, and the scars are just for show.
But I command you to be bound. Hear me now. Feel your spirit shrink until that soiled flesh measures the length and breadth of you. It is you.
And it's flammable.
--Lucifer: Children and Monsters, p. 73.
I can't be the only person who reads this and thinks, oh, that's what it's like to realize you don't have proper backups, right?
Disk failure doesn't always look like you'd expect it to. Sometimes it takes the form of slower and slower reads, which only really become noticeable when the operating system times out, and when running CHKDSK overnight is still stuck at six percent in the morning. Through all of this, smartctl was reporting that the overall health assessment had PASSED, though attributes indicating a failing drive were nonzero.
I haven't done this sort of work in... years, I think. I usually don't build my own machines any more, certainly not from components, and I don't even know what the usual specs are. But I do know that while you use dd (an ancient Unix tool) to move data from place to place normally, you use ddrescue, a much subtler and more powerful tool, to do so in extreme circumstances.
The rescue toolkits have also gotten better. Instead of burning CDs or, worse, fiddling with SYSLINUX and wondering why your stick won't boot, you use something like multibootusb to set up SystemRescueCd (for Linux tools), Hiren's BootCD and/or FalconFour (for Windows tools), Tails (for unforeseen circumstances), and whatever else I can think of, and because large USB sticks are cheap, you have room left over to store movies or whatever else you want to keep on there.
I ordered a replacement drive off of Amazon with same-day delivery, plugged it into an external drive enclosure, booted SystemRescueCd off of the stick, ran ddrescue -f /dev/sda /dev/sdb recovery.log, waited nine or ten hours, swapped the drives, ran CHKDSK on startup, and we were back up and running in the morning. About twenty-five hours from start to finish.
$ ddrescuelog -t recovery.log current pos: 139233 MB, current status: retrying domain size: 1000 GB, in 1 area(s) rescued: 1000 GB, in 189 area(s) ( 99.99%) non-tried: 0 B, in 0 area(s) ( 0%) errsize: 1396 kB, errors: 188 ( 0.00%) non-trimmed: 0 B, in 0 area(s) ( 0%) non-split: 0 B, in 0 area(s) ( 0%) bad-sector: 1396 kB, in 188 area(s) ( 0.00%)
This is recovery at the logical-disk level, not the filesystem level. I don't know if the unrecoverable sections even covered any files. CHKDSK seemed to catch some complaints about metadata. I wish NTFS had some kind of internal checksumming to definitively determine health or corruption at this point. But in general, this was just unbelievably successful. This sort of problem--boot disks, recovery tools, forensic programs--has been solved.
Thanks, free software community! I donated 20 EUR to Antonio Diaz Diaz, author and maintainer of ddrescue, because his work, and that of his colleagues, saved me at least that much in termsof angst.
Data is one of the most valuable assets in a business, making data loss occurrences a seemingly unbearable nightmare. Why do most business executives consider these instances of data loss to be a worst-case scenario? In short, when data is lost, it translates to the following:
Lost revenue
Lost time
Lost working files
Lost contacts
Lost client and vendor information
After data loss, there comes the arduous task of recovery (unless you’re planning to throw your hands up in defeat and accept your inevitable doom). What is data recovery? By definition, it’s any effort exerted to salvage data that you can’t recover from your server or computer.
Ask any IT professional, and they’ll tell you that data recovery isn’t a walk in the park, even for the tech experts. While cloud-servers are a useful resource when backing up data, there’s some sensitive data that you wouldn’t want to host with third-party servers. Due to these exceptions, you risk losing valuable data that’s difficult to recover.
How? There are many reasons you could lose data. Some of these factors are avoidable, but others are simply out of our control.
Read the full article
DVD ripping part 2 -- ddrescue, "dvdbackup --mirror" and "dvdbackup --feature"
So I've got things working a bit better now. There have still been one or two that haven't ripped using one of these three methods, but this covers 99% of my DVDs that I want to play with MythTV:
When this works you might get one or two error sectors, but most of the disc will rip without error. These errors are intentionally added as part of the copy-protection. This gives a DVD iso image which mythvideo is happy to play.
Method 2 : dvdbackup --mirror
This produces a VIDEO_TS folder which, again, mythvideo is happy to play. Sometimes this doesn't work, or produces a damaged rip which is several times the size of the real DVD. If it's bigger than about 10GB then this probably won't play, so try method 3:
Method 3 : dvdbackup --feature
This produces a VIDEO_TS folder, but often mythvideo won't play it. mplayer is generally happy to play this.
All in all, still a bit of a pain but at least I can play most of the DVDs I own now!
I spent a week trying to recover photos and documents from a friend's dying hard drive ( whilst coding TellDJ of course ;) ). Most of that time was simply spent waiting for various tools to do their jobs, but ultimately I managed to get the stuff off for a grand total of £10 (or free if you don't need the enclosure, but it's still significantly less than a quoted £600). This post cuts out all the various things I tried (which tooks days sometimes) and gets to the thing that worked for me. I figured I'd write a blog post about how I did it in the hope that it helps save someone else's time and data.
Disclaimer: I take no responsibility for what happens to your drive, data, or anything related. I'm just explaining the steps I took to perform the task and the things that worked for me. Use the information below at your own risk and if the data is important enough, then you should consider paying for a professional data recovery service to get the data back. Basically, YMMV :)
My friend's hard drive was from a Macbook Pro (MBP 500gb). I have a Macbook (320gb) to connect and mount the drive to and I also have a Windows 7 PC with a bigger hard drive (1 TB). The information below should also work with a FAT32/NTFS formatted drive, but since the MBP hard drive is HFS+ formatted in this case, I needed a MBP to be able to read the files off it. Both MBPs (damaged hard drive one and my working one) were running Snow Leopard.
Setup
My first idea was to put the HDD into an enclosure and hope that my working MBP could read the data off it after booting into OSX. I purchased a 2.5" SATA External HDD enclosure and installed the dying HDD into it. I connected the enclosure via USB to my MBP. However, the MBP couldn't mount the device (nothing appeared in Finder) and Disk Utility couldn't repair it (although it could see that it was a 500gb hard drive). I wasn't sure how much of the 500gb was actually used and so, to be on the safe side, I needed to ensure that I had 500gb free. This is where my desktop comes in and I managed to transfer some files to an external drive so that I could have 500gb free (in the end, it turned out I only needed 60gb since that's how much was in use on the broken hard drive).
Software I Used
ddrescue (This is different from dd_rescue and will be used to 'clone' the harddrive)
PhotoRec (will be used to recover documents)
I did try the trial for Data Rescue II and left it running for a few days, but I didn't get any data back and just quit the process since it was still running. Of course, that's not to say it won't work for you, but it depends on how far gone your hard drive is.
ddrescue is used to copy and recover blocks from the HDD into an image. Now, there are various ways you can approach this and I suggest you read the manual for all the options. You can directly save the contents to another hard drive, or you can save it into a DMG file which is what I did.
Both, ddrescue and PhotoRec can be used on Windows and Linux so even if all the steps in this article are not relevant to you, the key parts may well be.
Connecting the Mac to the PC
If your Mac has a bigger hard drive than the damaged one, then you don't need to save to a PC and can skip this step. I initially just connected via the wireless network but a short drop in connectivity caused ddrescue to throw an error (because it couldn't find the destination address to save to). For reliablity, a crossover cable connected my Mac to my PC. I have a shared folder on my PC and accessed it via Terminal on my Mac:
pcpassword = password of the account you're trying to log in with (if there is one)
localip = local ip of the machine (this doesn't necessarily start with 192.168...) You can find your PC's local ip address by pressing 'start' and typing 'cmd', followed by enter. Then type in 'ipconfig' in the command prompt and you'll see the ip address of the network adapter you're using. Note that the 'KP' at the end refers to the directory being shared on the machine.
/Users/keyboardP/Recovery is the folder on your MBP that links to the shared directory on the PC. So, for example, if you create a new text document within that folder, it's saved on the desktop PC because that folder is simply an alias for the shared directory and isn't actually storing that data on your MBP.
After you've entered the above, press enter and if all went well, there shouldn't be any messages. You can now test this connection by opening Finder and navigating to /Users/keyboardP/Recovery (or whatever you typed). You should see the PC's shared stuff in this directory.
I've heard about this many times, but the performance of copying data between Mac and Windows 7 is terrible and I experience the same thing. However, I followed this trick and it did improve the performance slightly. That 'slightly' shouldn't be underestimated in this project as it adds up to save quite a bit of time.
Running ddrescue
Now that the PC is connected, I'm guaranteed to have enough space to copy the entire contents of the damaged HDD. I plugged in my hard drive (which doesn't mount) but appeared when I typed diskutil list in Terminal. I could tell which one it was by looking at the hard drive space (500gb as opposed to the MBP I was using which had 320gb). I then downloaded, installed and ran ddrescue via terminal modifying the commands mentioned here.
At the time I peformed this, 1.16-rc1.tar.gz was the latest stable version. You can see all the versions here.
First we need to install ddrescue, so extract the downloaded file and navigate to in Terminal. Once in the directory, type ./configure and press return. Assuming no erorrs, type make and ddrescue should now be built and installed.
Whilst still in the ddrescue directory, you can now perform the command to copy the blocks of the HDD to your working machine.
Now, the -n parameter version is optional (as opposed to the -r version I'll be explaining) and is used if you want to quickly copy across any error-free sections. This will be quicker since it ignores any damaged areas. I did perform this step, but I've heard other people skip this step and just attempt the full recovery which attempts to fix and copy the corrupt areas as well. You can read about all the command line switches here. /dev/disk1 is the name of the damaged HDD when I connected it via the USB enclosure. If you've forgotten the name from above, you can see it by typing diskutil list. nImage.dmg is the name of the file we're going to copy the across to. Note that the directory is local to the MBP but is actually the alias for the desktop which we created above. Therefore, the dmg file is actually being saved on the desktop since it has a bigger hard drive space than the broken HDD. nLog is the name of the log I want to save to. The log is very important if you're interested in being able to stop and start the process as it allowed ddrescue to start from where it left off. If you don't have a log and you stop ddrescue, then ddrescue will have to start from the beginning (which can result in potentially days of effort). Basically, you should keep a log IMO.
If you decide to use the above version first, then you can let the process run. Once completed, you'll have a dmg file and you can skip to the 'File Recovery' section below. If the files you want aren't recovered, they're likely in the damaged section and you can perform ddrescue again but with the -r parameter below. Alternatively, if you just want to use the -r version and skip the -n version, you can do that as well.
This will attempt to fix any errors it finds and will retry 3 times when it comes across a bad pass. You can change the 3 to however many times you want, but I used 3 when I did it.
Let it do its thing which can take between hours to days depending on how damaged the drive is.
In my case, ddrescue got to around 60gb at which point it was barely increasing. I pressed CTRL+C to interrupt and created a copy of the dmg file (just in case I messed it up somehow). Also, make sure you don't delete the log file by accident otherwise you'll have to do it all again (and hope that your hard drive is still physically readable).
Now that we have the dmg, we can try and get the MBP to read it as a disk device (even if it still can't mount it). I copied the copied version of the DMG file to the Mac so that the recovery process ran faster since it didn't have to perform it over a network.
(notice I'm using rImageCopy which is the copied version, just in case I broke something :D)
Press enter and hopefully it will return /dev/disk2 (or something like that, instead of an error).
File Recovery
Open up (or install if you haven't done so), PhotoRec. Follow the great guide explained on the official PhotoRec's page. You can select what file types you want to recover and then let it do its thing. This stage didn't take too long, relatively speaking (maybe around 8 hours), but I now had the all-important photos and word documents my friend wanted.
On a slight tangent, if you're running Windows and your hard drive is fine, but you want to recover deleted files, I've successfully used PC Inspector File Recovery. The key thing with deleted files is to try and recover them ASAP. The more you use your machine, the higher the chance of the deleted file data being overwritten thus making it ever harder to recover the files. PhotoRec should do the job as well, but less technical users may prefer the visual based aspect of PC Inspector File Recovery.
That's pretty much it! I hope this helps save your data and your time (some tools took days to just read the hard drive, so I ended up quitting them). This is all from memory and my Terminal history, but having re-read it, there doesn't seem to be anything missing. Of course, the key thing with data recovery is speed so avoid using your HDD when you don't need it (which is why a clone is better than trying to recover directly form the HDD). If you're not sure of what you're doing and the data is very important, then a data recovery service may be a better approach than a DIY thing. You can also try other tools such as Data Rescue II or DiskWarrior. Good luck!
The formula for ddrescue was recently updated to version 1.14.
To install ddrescue on your Mac, open your Terminal application and run the following commands;
brew update brew install ddrescue
The updated formula is as follows:
1 2 3 4 5 6 7 8 9 10 11 12 13
require 'formula' class Ddrescue < Formula url 'http://ftp.gnu.org/gnu/ddrescue/ddrescue-1.14.tar.gz' homepage 'http://www.gnu.org/software/ddrescue/ddrescue.html' md5 'd6f6cc63df9ad352bc6e43b65c975af5' def install system "./configure", "--prefix=#{prefix}", "--disable-debug", "--disable-dependency-tracking", "CXX=#{ENV['CXX']}" system "make install" end end
Currently ddrescue has no dependencies in homebrew.
