Copying a Time Machine Volume

I decided recently to upgrade the storage in my Mac Pro. Currently it had 3 hard drives in it

• 320 GB – the one that came with the computer. This is little changed from when I got the computer as I immediately added a 500 GB and 750 GB to the machine as soon as I got it
• 750 GB – this is my startup volume, and has all the applications and documents on it. 90 GB free. It contained 1,803,033 files using 608.5 GB (653,331,116,032 Bytes).
• 1000 GB – this is my Time Machine volume, and has Time Machine backups going back 16 months to October 2008. 135 GB free. It contained 16,765,985 files using 787.6 GB (845,667,311,616 Bytes).

So for the upgrade – the plan was to add a 2 TB drive into the last empty drive bay in the Mac Pro to become the Time Machine volume, then clone the 750 GB drive to the 1000 GB drive, then use the 750 GB drive as a network backup volume perhaps.

So the first step was to attempt to move the Time Machine volume. This article http://www.macworld.com/article/146085/2010/02/migratebackups.html indicated it should be a fairly straight forward task. So I followed the instructions and set out. It took about 2 hours to copy all the data from the 750 GB drive to the new 2 TB drive, then I waited another 2 hours for the “Verifying” phase to complete, and just as it appeared success was at hand, a dreaded error message.

Well that wasn’t very helpful. Thinking that it was finished verifying I just accepted that thinking all would be good, but attempting to check the disk with Disk Utility gives

Even less impressed now. So I decided to run a Disk Repair on the 750 GB Time Machine volume – that took over 6 hours and found a few errors which it said were fixed. Then I tried the MacWorld procedure again, with at the end of another 4 plus hours the same frustrating result – an error message and a failure to be able to verify the drive.

Googling the error message led to a number of other sites with people complaining of similar problems. I tried the several suggested solutions, including initially making a partition on the new drive that was only a little larger than on the old drive, doing the Copy, and then enlarging the partition. This time the copy was successful, but enlarging the Partition wasn’t.

By now 3 days have passed me by.

So I tried Carbon Copy Cloner by Bombich Software, but when that was unable to manage the task I found that copying Time Machine volumes isn’t supported by Carbon Copy Cloner.

So I then tried SuperDuper! by Shirt Pocket Software. This set out to do the copying, but seemed agonisingly slow. After some 8 hours it had copied less than 6,000,000 of the files, so having read more sites that indicated that the MacWorld process, or very similar variations thereof really would work, I aborted the SuperDuper process and started another process using Disk Utility, but alas this too failed.

So it was going to have to be SuperDuper!. I set that running yesterday, and it finally completed today after an elapsed time of 17 hours 22 minutes.

According to Activity Monitor, at times the copy speed was above 100 MB/sec – presumably when copying large files, but for much of the time the copy speed was around 3-4 MB/sec as it copied the tens of thousands of symlinks that make a Time Machine volume work.

I don’t know what happened to the apparently missing 1,741,753 files (16,765,985 – 15,024,232), but once the copy was finished, Disk Utility reported that the drive had 16,269,744 files on it – a difference of a mere 496,241!!

When it was all done, and a “Repair Disk” from Disk Utility showed that all was normal, I opened the Time Machine preferences, selected the new 2 TB disk as the Time Machine Disk, and nervously waited for it to complete the first backup. This took a bit over 2 hours for Time Machine to scan the volume, Spotlight to get it indexed for Time Machine, and then the first backup done in several days to run, but HAPPILY when it was finished it recognised that the oldest backup was October 22, 2008, and reported that the next backup would be in about 45 minutes time. That backup ran successfully and completed in only a few minutes – it seems we’re back to normal, with a whole TerryByte of free space on the Time Machine volume now!!

So the next challenge is cloning the 750 GB drive to the 1 TB drive. Tomorrow is another day!!

Mac OS X ipfw Firewall

I had occasion recently to try and figure out how to use the Firewall built into Mac OS X to prevent a very bad mannered “bot” from hitting one of my sites – at times at the rate of 10 hits per second, and 2 seconds later another 8-10 hits!!! So I needed to be able to block certain IP numbers, or ranges of IP numbers. Mac OS X comes with the FreeBSD firewall programme called IPFW. This is a very powerful feature that can be accessed from Terminal.

Some Googling later I came up with several helpful sites that got me up and running with this.

First, using the Apache server logs identify the IP number, or range of IP numbers you want to block. I used TextWrangler to open the log file and do some preliminary editing, and then imported that into FileMaker Pro to get only the log lines applicable to PiplBot (BAD ROBOT!!!!). Over the course of about 5 hours it used 84 different IP numbers as it hit away at one of my sites over 19,000 times.

So once I had a list of these numbers, I was able to break them down into a number of shorter lists that had the first 2 or 3 octets of the IP number the same. With this done, this site http://www.mikero.com/misc/ipcalc/ provides a VERY handy calculator that will take the starting and ending IP numbers in a range, and convert it to a range in the CIDR notation (very technical explanation here) which takes a range of numbers like

which potentially covers 14 different numbers and converts it to 67.228.42.160/28 which represents 16 numbers without the need to list them all out. And simlarly the range from 208.43.23.227 to 208.43.33.238 covers 2,572 addresses, and is represented by 208.43.0.0/18 – a range of 16,384 addresses.

So, armed with this knowledge and ability, I’m now able to understand the instructions on this page http://www.dancatts.com/articles/dealing-with-bad-bots-at-the-firewall-level.php and this page http://www.ibiblio.org/macsupport/ipfw/ which in their simplest form are saying that you can use Terminal with this instruction

to add a block into the Firewall for the range of numbers 67.228.42.160/28. You can see the current status of your ipfw with this Terminal command

which will return a list in this form

Changes made by Terminal only last as long as your Macintosh is running – they are not saved to be used on a Restart unless you write a startup script to do this. This site http://www.ibiblio.org/macsupport/ipfw/ provide details on this, including a number of sample scripts, but frankly this was way over my head, so I turned to MacUpdate to see if there was an application that would do this via a GUI (Graphical User Interface). I found several, and settled on WaterRoof by Hany El Imam. This allows you to define the rules you want to implement, and then takes care of creating the script that will activate these rules each time your Macintosh is started up.

WaterRoof Screen Shot

This seems much easier to deal with

So for now PiplBot is banned, even though they seem to be honouring their statement that they would remove all of my sites from their list of sites to crawl.

I hope this helps someone else – I’ve written it partly to help me remember what I did, but also to help others.

Upgrading to Mac OS X 10.6 “Snow Leopard”

After waiting almost patiently most of yesterday (Friday 28 August) I received my copy of “Snow Leopard” about 4:45 PM.

Being the cautious type I knew that I would first upgrade the MacBook, and see what out of PHP, MySQL and Apache didn’t work afterward before embarking on upgrading my Mac Pro which does full time duty as a webserver for a number of domains that use PHP and MySQL.

The install of Mac OS X 10.6 Snow Leopard went about as easily as advertised, and an hour after starting the MacBook was back running again. There was a notice that I had some now unsupported applications, but I didn’t see a list of them. I tried to open Parallels and was told that it wouldn’t work with Mac OS X 10.6 – I had Parallels 3. But no worries – I have VMWare Fusion 2 and that works, so upgrade the Parallels Virtual Machine to work with VMWare Fusion and delete Parallels.

Read the rest of this entry »

FamilySearch Indexing on Macintosh

A few hours after I wrote the below…

FamilySearch Indexing support replied to my eMail(s) to them with this link

https://help.familysearch.org/publishing/202/104450_f.SAL_Public.html

which covers both Mac OS X 10.4 and Mac OS X 10.5.

========================

I noted with some excitement last evening that FamilySearch Indexing had a project underway with a New Zealand flavour to it – “New Zealand—Passenger Lists, 1871–1915″, so I thought this was the incentive I needed to get back to indexing – something I hadn’t done in quite a few months. But alas I couldn’t get the FamilySearchIdexing application to start up – no matter what I tried it generated an error “Unable to start the Application”.

Some Googling led me to this page https://help.familysearch.org/publishing/301/103757_f.SAL_Public.html which provided the hint as to where to look, but unfortunately those directions aren’t directly applicable to Mac OS X 10.5.7 which I’m running.

So this page lists what I did to fix this problem. NOTE – these instructions apply to Mac OS X 10.5.x

Go to your Macintosh HD ——> Applications ——> Utilities folder and find the icon “Java Preferences”. Double click that to open it, and then click on the Network tab

Read the rest of this entry »