Why Upgrade Now
Now that the first major patch release is out (OS X 10.5.1) and after having read numerous reports about the new features, OS improvements to stability and the user experience, and the recent bug fixes, I upgraded. The upgrade went smoothly, but it wasn’t without quirks.

Installing was an interesting experience
It took close to two hours for the installation procedure to complete. While the progress bar accurately reflected the installation progress timing, the installation timer was noticeably broken. It raced along like time for a relativistic observer. An upgrade install on my G4 PPC Powerbook and on a (nearly) brand new MacBook Pro quoted initial times-to-completion of over four hours and then converged to zero as the install proceeded. This was slightly annoying, but I guess it was better than having broken expectations of an installation time that continually gets adjusted up instead of down (Windows timers are notorious for this).

About halfway through the installation I had a flash of horror when I couldn’t recall if I was presented with a screen to choose an installation type. (No, I’m not an idiot, I didn’t realize that I was never prompted to choose an installation type until this point). The minimal installation progress screen displayed little more than an “Installing Leopard” header and a progress bar across the now famous ‘cosmic’ OS X purple backdrop. Apparently the engineers in Cupertino didn’t feel the need to provide any distinguishing text for fresh installs, archive installs, and upgrades in the install-progress UI panel. I found this frustrating and consider it a major oversight of the installation sequence. I can only imagine what Genius Bar employees will need to contend with when a user comes in claiming that Leopard ‘ate’ his computer and deleted everything.

What Worked
Almost everything. About two hours after the installation I had an upgrade Leopard install ready to go. It seems like may other people had the same results using the upgrade option. All my applications were intact and I experienced no major upgrade-related problems.

Quirks
Now it was time to upgrade to OS X 10.5.1. This was painful as this was when a number of short-lived quirks and annoyances came to bear.

After I installed Leopard my Spotlight index was cleared and needed to be entirely rebuilt. As you might imagine, my powerbook was not exactly responsive while it re-indexed every piece of data I had. This intense disk use slowed my machine to a crawl during the patch upgrade as Spotlight and Software Update competed for disk seek and IO operations.

While this was happening I thought it might be a good idea to to kick back and read some email. So, I fired up Mail. This was a big mistake. Like my Spotlight indices, my Mail indices (i.e. Smartlist internals) were also gone. I use Mail as a local backup of my entire GMail mailbox, so I have about 2GB worth of mail scattered about a few dozen Smart Folders and Archives. After I opened Mail it took the better part of 15 minutes to reconstruct these indices and reconstruct my Smart Folders. While this was happening I could hear my hard disk thrashing about, vigorously seeking from Softare Update install, to Mail indexing, to Spotlight read/write, and back, all while I sat and waited for the curiously colorful beach ball to stop spinning.

On the MacBook, the system didn’t hang unresponsively, it was just sluggish for a while.
After about an hour I was done. I ran through the list of instaled applications I had, firing them up one-by-one to make sure they still worked and - to my amazement - everything appeared to work. I was ready to continue to phase two — installing my dev toolchain: XCode, Fink, Python, etc. I actually just started this, so for details you will have to read on next time.

References

[1] Ars Technica OS X Leopard In-Depth Review, (long, but packed with information)
[2] Apple Official OS X Leopard 10.5.1 Patch Notes
[3] AppleInsider OS X Leopard 10.5.1 Discussion
[4] MacRumors OS X Leopard 10.5.1 Discussion
[5] LifeHacker Leopard Clean-Install vs. Archive-Install vs. Upgrade Survey
[6] One user’s take on the new Leopard theme

I keep an external hard drive attached to my powerbook, almost permanently. While working at my desk, this drive provides an expanse of disk space that seems endless when compared to the 60GB drive that my notebook contains. It serves as the home of a nightly backup and a repository of every application (for both pc and mac) that I have ever used or conceivably have a legitimate use for. I’ve found that keeping a copy of everything requires minimal effort, is reasonably cheap after considering the cost of disk space compared to the time and difficulty of re-downloading everything, and it’s just plain smart. As an added bonus, it allows you to completely rebuild or replicate your setup on the rare occasion that its needed (such as when a drive failure occurs).

Me — really OS X Disk Utility — being stupid

I had cleared a bunch of junk on my external disk, and emptied the trash. I then decided to change the filesystem on one partition of the external device from Fat32 to ext3. Up until this point, Fat32 was the easiest pc-mac interoperability solution available, and I didn’t know about the excellent and free ext3 drivers available for windows. I had recently acquired iLife 06 and wanted to back up my media, as usual, but I needed a filesystem that would support a 6GB DVD image file. Fat32 doesn’t allow this, as it has a 4GB file size ceiling.

For whatever reason — sucks to you Apple developers — while formatting the partition the trash still being emptied. Subsequent dodgy behavior ensued, the drive format failed and left the disk in an undefined state with a missing partition table, and all data on the drive appeared lost. The drive, superficially at least, was bricked. It wasn’t even mountable.

Kubuntu and Parted saved my life
Performing any sort of drive maintenance on a mac is downright painful. Disk Utility obviously doesn’t offer comprehensive support for anything, and it’s not well engineered. As we know, a sanity check wasn’t done to make sure that there weren’t any open files on the disk while it was being tampered with. Further still, there is a general lack of availability for standard tools like parted and gpart for OS X (including through back channels like fink).

Both gpart and parted have the ability to rebuild a lost or corrupt partition table by inferring the filesystem types from the drive contents. Naturally, there isn’t support for all partition types, but there definitely is support for de facto standard filesystems such as Fat16/32 and ext2/3 among others. This was perfect for my situation — I had one 200GB Fat32 slice, and one 50GB (almost, it was actually 32GB) ext3 partition. Keep this in mind the next time you think you’re slick and you install that super-fast foo-bar-baz filesystem on your cutting edge machine, as there might not be anyone to save you from yourself later on.

So I turned to linux. Kubuntu in particular, since it’s LiveCD was PPC-compatible and it came packaged with parted. I booted up, fired up an xterm, ran two commands (that’s all it took), and I was done:

sudo parted /dev/sda

rescue 0GB 200GB
// rescues partition spanning approximately 0GB - 200GB
rescue 200GB -1
// same thing, but from 200GB until end-of-disk

And that’s how my hard drive was saved and crisis was averted.

References

[1] GNU Parted

After a little research, I found that any Firewire-equipped Mac can be made into an HD-PVR for unencrypted content at the expense of an appropriate firewire cable. Apple even provides the necessary capture software that you need, provided that you know where to look.

The Project

Building a super-low-cost, HD-PVR.

Materials

1. A cable box with Firewire output
2. A Mac (with an available Firewire port)
3. A Firewire cable for connecting the cable box to your computer (with appropriate connection interfaces)
4. Some spare hard drive space (HD required ~100MB/min)
5. Apple Firewire SDK
6. MPlayer (or, alternatively, VLC)

In my case, I used a Scientific Atlanta 3250HD Cable Receiver, a 12 inch 1.33Ghz Aluminum Apple Powerbook, and a spare Belkin male-male firewire cable that I had laying around from a Mac-to-Mac data transfer I did at some point. The Apple Firewire SDK is downloadable (free registration required) from the Apple Development Kits Page. MPlayer and VLC are also free downloads.

In theory, any combination of 1-3 should work. If your cable set-top box lacks Firewire connectivity, ask for it. The FCC mandates that all cable services provide firewire output (on request if necessary) as of April 1, 2004 [2, 3].

Install the Prerequisites

In order to record and playback a signal from the cable box, we basically need 3 things: a signal/physical connectivity, some software for video capture, and some software for playback.

Before getting started, make sure you do the following:

1. Check that your cable box is operational (I hope that goes without saying).
2. Connect it to your Mac via Firewire.
3. Download and install Apple’s Firewire SDK
4. Download and install your choice of MPlayer or VLC

Now we’re ready for the fun.

Recording Content

Recording content couldn’t be easier. In fact, Apple’s Firewire SDK ships with all the capture software that we need. It contains a nifty little program called ‘AVCVideoCap’ that provides an interface for connecting to and capturing DVHS video streams via firewire. It’s located (on my machine, using version 23 of the Firewire SDK) at:

/Developer/FireWireSDK23/Applications/AVCVideoCap.app

Fire it up. If everything goes well you should see your cable tuner in the ‘attached devices’ list:

Capturing a video is as simple as choosing a filename:

and some basic options:

In the capture options window you can choose a specific channel to record. Unfortunately, I was unable to get this feature to work on my SA3250HD (I have read reports of it working on some other models though). In any event, AVCVideoCap will capture the current channel on the set-top-box:

Playback

The AVCVideoCap application captures the raw MPEG2 Transport Streams as .m2t files. Lucky for us, MPlayer and VLC readily support playback of this media, if the channel that was recorded is unencrypted. At a minimum, this applies to standard broadcast channels (i.e. those available outside the realm of cable tv) and will vary from service provider to service provider.

Since these files are streams, they lack timing information for random-access playback, that is, seek and skip functionality. Timing information can be added with a few mousclicks using another utility in the Apple Firewire SDK — VirtualDVHS. Open this application, load the .m2t file, and create a navigation file for the selected MPEG2 transport stream.

If you are fortunate enough to have an HDTV with a firewire input, you can use VirtualDVHS to stream the m2t transport stream directly to the TV [3].

Caveats, Notes and Other Discussion

Recording HD content is awesome. It looks amazingly crisp on my powerbook LCD screen. I’m glad I figured this out before the season premiere of 24.
Now it’s time for a few words of caution. It takes significant firepower to process streams of this capacity. At over 100MB/min, recording HD content averages about 7-8GB/hour for HD content. My powerbook suffers from screen-jitter during HD playback. It’s 2 years old — 1.33Ghz PPC, 1.25GB RAM, 32MB Video Card — and not designed for this kind of abuse. In addition, my meager 60GB hard disk proves much too small to record anything regularly — I’m glad that my server has ~500GB of spare disk capacity.

Just make sure that whatever disk you store your .m2t files on has large file support — Fat32 filesystems only support files up to 4GB. Thankfully, HFS+ formatted drives (standard on Macs) do not suffer from this limitation.

The final caveat that I wish to mention is that of encryption. Many channels will be capturable but you’ll lack the ability to play them back due toan inability to decrypt the stream. With some time I bet a suitable decryption application will spring up.

On a final uplifting note, it turns out that someone has already wrapped up all the recording details into a slick carbon app — iRecord.

Update

As pointed out by one Digg user (thanks teif), instructions for capturing content over Firewire on PCs is available at hdjunkie.com.

References

[1]http://www.avsforum.com/avs-vb/showthread.php?t=386740

[2]http://www.gizmodo.com/gadgets/tag/fcc-requires-firewire-on-all-cable-boxes-9313.php

[3]http://www.macosxhints.com/article.php?story=20040426151111599

[4]http://www.ammesset.com/software/irecord/

[5]http://hdjunkie.com/tutorials.html

If you’re feeling adventurous, you can read the official DB2 9.1 documentation and the many, many DB2 Manuals available on IBMs site.

Getting DB2
There are a number of verions of DB2 available from IBM. Unless you have usage requirements that demand advanced data replication, partitioning, and clusteriing, the freely distributed Express-C community version of DB2 should suffice. The only notable limitation imposed by this version is a 2CPU/4GB Ram hardware constraint on the physical host on which the software is installed. More information on the varying DB2 versions, their features, and their constraints can be found on the official IBM DB2 page. You can get DB2 Express-C at the IBM downloads page.

Installing DB2
The DB2 installation package is rpm-based, so we’ll need to install an rpm-compatible installer for Ubuntu. For this, we apt-get alien, an rpm package installer:

$ sudo apt-get install alien

Now we may db2_install installation script that is bundled with DB2. This needs to be done as root:

This should take a few minutes to install. At the time of this writing, the installation procedure had approximately ten steps and lasted about three to five minutes. Note that depending on the version of gcc that you have installed (if at all), you may need to install libstdc++5, as it is required for successful installation. This is easily obtained:

$ sudo apt-get install libstdc++5

Setting Up Groups/Users
Now that we have DB2 installed, we need to configure it for use. We start by creating the standard DB2 user groups: instance owners, fenced users, and administrators. (Note that these commands need to be run as root — the sudo is dropped here for notational simplicity and a # is added in its place for clarity).

# groupadd -g 999 db2iadm1
# groupadd -g 998 db2fadm1
# groupadd -g 997 dasadm1

Now we’ll create a default user for each group:

# useradd -u 1002 -g dasadm1 -m -d /home/dasusr1 dasusr1 -p password2
# useradd -u 1003 -g db2fadm1 -m -d /home/db2fenc1 db2fenc1 -p password3
# useradd -u 1004 -g db2iadm1 -m -d /home/db2inst1 db2inst1 -p password4

Creating the Adminstrative Database
Now we need to create a DB2 Administration Server to administer all instances of DB2 under the purview of this installation:

# /opt/ibm/db2/V9.1/instance/dascrt -u dasusr1

Note here that dasusr1 may be any user that is a member of the dasadm1 group we set up earlier.

DB2 Instance Semantics. Creating an Instance.
A DB2 instance is an environment that acts as a logical container for a collection of databases. It’s an abstraction that allows the creation and usage of multiple independent DB2 environments using the same physical resources.

We need an instance to hold all of our databases. We’l create a default instance using the db2inst1 user we just created as its owner:

# /opt/ibm/db2/V9.1/instance/db2icrt -u db2fenc1 db2inst1

As we noted, db2inst1 is the instance owner, while db2fenc1 credentials are used to execute fenced stored procedures and executables.

Starting and Stopping an Instance
A DB2 instance is an environment that acts as a logical container for a collection of In order to start and stop the DB2 instance (remember, we may have multiple instances for a given installation on a physical host), we log in as the owner and call db2start after inializing the DB2 environment:

$ su db2inst1
$. ~/sqllib/db2profile
$db2start

Stopping the environment is just as simple: we’d call db2stop instead of db2start.

You may enable (or disable) automatic DB2 instance start/stop on system start-up/shut-down by calling:

$ db2iauto -on db2inst1

where db2inst1 is the login name of the instance. Replacing the -on flag with -off disables this feature.

The Sample Database
If you wish to build the sample database bundled with DB2 in order to play around, call db2sampl while logged into a DB2 instance owner account:

$ su db2inst1
$ . ~/sqllib/db2profile
$ db2sampl

That’s It.
This concludes the bare-minimum needed to set up a working DB2 environment on Ubuntu. Note that this configuration does not allow for external connections over TCP/IP. We’d need to start a separate service

References
[1] Official DB2 9.1 documentation
[2] DB2 Manuals

Let’s get started. To be clear, we use the current stable SVN server and client packages availabe through Fink at the time of this writing, SVN 1.2.3-1012.

Prerequisites

• Fink
• Apache2 with mod_ssl installed

Installing the SVN Server

We need to install the SVN-server and SVN-client packages separately. While most users will only need the client, those wishing to run a local SVN server instance on their mac for local source revision control will need to install the server packages.

Installation of the server packages is easy. If you want to authenticate SVN clients over http using ssl, you will need mod_ssl on apache in addition to an ssl-enabled SVN server. Initiate the server installation by calling…

$ sudo fink install svn

of, for the ssl enabled package, call…

$ sudo fink install svn-ssl

If you’re unsure of whether or not you’ll need or use ssl, install the ssl-enabled server since it permits both modes of operation: non-ssl and ssl.
During the installation procedure you may be prompted with a few virtual dependency warnings:

fink needs help picking an alternative to satisfy a virtual dependency. The candidates:
(1) python: Interpreted, object-oriented language
(2) python-nox: Interpreted, object-oriented language

and

fink needs help picking an alternative to satisfy a virtual dependency. The candidates:
(1) db42-ssl: Berkeley DB embedded database - ssl
(2) db42: Berkeley DB embedded database - non crypto

In both cases choosing the default [1] is fine.

Depending on the breakdown of your currently installed packages, this installation may take a while since it requires python, ruby, and a slew of other tools. On my 1.33Ghz G4 Powerbook it took between 2 and 3 hours to complete compiling and installing roughly 40 packages. (I went out to lunch while all this took place).
For those of you who cannot wait, prefer binaries, or are general weenies, you can avoid compiling the code yourself by using the apt-get binary installer by calling

$ sudo apt-get install svn

or

$ sudo apt-get install svn-ssl

This will download and install the binaries, sparing you the compilation wait.

Installing the SVN Client

Setting up the client is similar to the procedure used to setup the server. Simply fire up a shell and call

$ sudo fink install svn-client

or

$ sudo fink install svn-client-ssl

making sure to choose the appropriate client for your server installation type. Note that the ssl-enabled client will play nice with both non-ssl- and ssl- enabled SVN hosts, while the standard SVN client will not.
You now have a fully functional SVN client and server. Now we just need to install some Apache addons so that we can serve SVN using Apache2.

Installing Apache2 Addons

First, we need to download and install the appropriate modules to serve SVN content over http with Apache2. This is simple enough:

$ sudo fink install libapache2-ssl-mod-svn

Done. Now we need to enable the module in our httpd.conf:

If you are using the Fink Apache2 installation, httpd.conf is located at

/sw/etc/apache2/httpd.conf

and you need to uncomment the following line (or add it if it does not already exist)

LoadModule dav_svn_module
/sw/lib/apache2/modules/mod_dav_svn.so

to the set of LoadModule statements near line 265.

If you are using the OS X original Apache2 installation (i.e. you did NOT install Apache2 using Fink), httpd.conf is located at

/etc/httpd/httpd.conf

and you need to add the following line (if it does not already exist)

LoadModule dav_svn_module modules/mod_dav_svn.so

to the set of LoadModule statements near line 240.

You need to restart Apache for these changes to take effect:

$ sudo apachectl restart

Now we’re all set to configure and set up a repository. We’ll do this in Part II, next time.

References

1. Installing Apache2 and SVN on OS X using Fink
2. Making the Jump to Subversion

Installing Apache

To install Apache2, simply call…

$ fink install apache2

or, if you prefer the SSL-enabled version, call…

$ fink install apache2-ssl

During the installation you may run into a few dependency disambiguation issues, but they can be superficially resolved. The first dependency issue you may see has to do with the server client-handling model:

fink needs help picking an alternative to satisfy a virtual dependency. The candidates:
(1) apache2-ssl-mpm-worker: Apache2 Server Binary
(2) apache2-ssl-mpm-perchild: Apache2 Server Binary
(3) apache2-ssl-mpm-prefork: Apache2 Server Binary
(4) apache2-ssl-mpm-leader: Apache2 Server Binary
(5) apache2-ssl-mpm-threadpool: Apache2 Server Binary

Unless you have a preference or a reason for doing otherwise, use the default setting [1].

The second dependency issue has to do with the underlying embedded Apache database:

fink needs help picking an alternative to satisfy a virtual dependency. The candidates:
(1) db44-aes: Berkeley DB embedded database - crypto
(2) db44: Berkeley DB embedded database - non crypto

Again, unless you have a preference the default [1] will work just fine. Personally, I prefer the default– I’d use the AES-encrypted database over the unencrypted one unless I had a reason not to want my settings/data encrypted.
Note, this installation will not replace the existing apache/apache2 server found on all base OS X systems. However, it will reconfigure the apache command line tools to use the version installed by Fink. The two Apache installations can peacefully coexist, and even simultaneously run, as long as they are not bound to the same port at the same time.

Once everything is finished compiling, you can check to make sure everything went smoothly by issuing…

$ sudo apachectl start

and the pointing your web browser of choice to

http://localhost/

If all goes well you chould see the Apache welcome page.

More information on the command line apache control utility can be found on the apachectl documentation page.

Enter Fink. Fink is a powerful source-based package management system for OS X written in Perl and maintained by the Fink Project. While there are a number of binary packages available through the project, the available packages are primarily distributed as source and usually require the standard GNU dev toolchain to compile (gcc, make, etc). Either way, it’s easy to use, makes everything you’ll probably ever need to develop software in an OS X dev environment immediately available, and is terminal-based in the good-old unix tradition. There is a GUI that offers basic package management capabilities, but who cares– in my experience I spend so much time at a shell that switching back to the gui costs more time than it saves.

Here’s what you’ll need to install Fink:

• Mac OS X Developer Tools (XCode 2.4 with gcc 4.0, X11)
• The Fink installable binary

Prerequisite: Installing OS X XCode Developer Tools

You can get these packages off of the OS X installer CD that came with your system. It might be dated, so you’re probably better off getting them directly from Apple via the Apple Developer Connection portal downloads section. This requires a valid ADC account and enough time to download a 900MB installable. Note that X11 is optional since it’s also available through Fink as a source package, but installing the Apple-bundled binary will save you loads of compile time (roughly a working day).

XCode uses the standard OS X package installer, so just click on through and wait until it’s done. By default the WebObjects API and CHUD profiling tools are not installed, so if you want these be sure to use the custom installation method. The installation of actual executables takes about 10 minutes, though the indexing of the documentation bundled with the tools takes anywhere from 20 minutes to an hour, depending on your machine. Sit back and relax until all this is done. It took 45 minutes on my 1.33Ghz G4 Powerbook.

Installing Fink

The Fink installable can be found at the Fink Project homepage in the downloads section. There are separate builds for Intel and PPC macs since, so make sure you get the right one.

Click through the standard package installer screens as usual. One of the last steps will set up your path to call the fink installed binaries (you’ll be prompted for the admin password when this happens). This is necessary since all the fink libraries and binaries are compiled into the /sw directory. On my machine Fink auto-configured the path only for X11 xterms, leaving all my Fink goodies unavailable from the standard terminal. If this happens you need to configure the path to use these binaries by manually add ’source /sw/bin/init.sh’ to your ~/.profile file.

That’s it Fink is now installed. All we need to do now is update it and configure its use. Fire up a shell and call…

$ fink selfupdate

to update Fink to the current stable release and update the package cache. You may be asked to set some configuration preferences related to mirrors and logging, but in most cases you can just accept the defaults. Once this is done call…

$ fink update-all

to update all the installed packages on your system using the fresh package list (if any new releases are available).

That’s all there is to installing Fink. Check back later for instructions on how to actually use Fink.