LTFS Podcasts

Podcast

In many ways the Linear Tape File System (LTFS) is my baby. I was the architect of the development work and a primary author of the LTFS Format Specification. I have spent a good deal of time advocating for LTFS including my posts about LTFS on this blog.

Last year I was offered an opportunity to bring more of the LTFS story to light. Jay Livens (Director, Product & Solutions Marketing, Iron Mountain) asked me if I would record some podcasts with him about the development of LTFS. Over the course of a few months we recorded 3 interviews with each session focused on a different aspect of the LTFS development; the Creation, the Technology, and the Future.

The podcasts that came out of these interviews were published today on the Iron Mountain website.

Posted in architecture, LTFS | Leave a comment

stealth startup ⥤ StackStorm

A couple of months ago I left my position at SanDisk to join a hush-hush stealth startup. The experience has alternated between feeling weird, and the most normal thing in the world. Weird when friends and former colleagues ask what I’m working on and I feel like a shifty agent from a three-letter agency. “We work on, you know making things better.” Normal because this is the valley and stealth is just part of the industry.

I’ve always felt that this was the right move for me and so far it’s been great.

Today we are announcing the company to the world…

StackStorm Logo

StackStorm is an emerging leader of the third wave of operations automation. Built from the ground up with DevOps in mind, StackStorm’s vision is a world of self-driving data centers that learn over time how to better operate themselves. StackStorm solutions leverage existing configuration management and monitoring solutions to deliver automation that safely ties together today’s loosely coupled, heterogeneous, ever changing cloud infrastructures.

ic-linkedin ic-facebook

Posted in michael | Leave a comment

Computer infection

Sitting at home tonight the home phone rang. A man with a thick accent informed me that he was calling from the Windows computer service department. “Sir, your computer is sending a lot of files to our remote system. This means there is a security problem with your Windows computer.”

Knowing this scam I decided to fish for a bit of a laugh and switched the phone to speakerphone. I made some concerned noises while the caller uttered gibberish about security file problems. At some point he claimed my computer was “infected”.

Right now, we have a lot of plastic sheeting around our home to guard against dust from some renovation work currently going on. That can be put to good use. :)

He tried to talk me into powering my computer on because it is safe and he will help me. At some point I wondered out loud if I should wrap my computer in plastic. I then hurried into the living room and made a lot of noise rustling the plastic sheeting. I claimed that I didn’t want to go near my computer since it is infected and I don’t want to get sick. I explained, “I’m wrapping my computer in plastic to protect my family against the infection”1. He tried to explain that the infection was an internet security infection. I listened a little and reduced the level of agitation in my voice.

Then I shouted suddenly: “No Gracie, stay away from the computer it’s infected…. uh the computer is Dangerous… Stay away.” … “How I can protect my children from the computer infection? I’m worried about my family’s health.”

At this point my darling wife was doubled over laughing uncontrollably but muffling her mirth in a pillow.

Again, the scammer attempted to back-peddle and explain that he needed access to my computer. But he clearly didn’t know how to handle me so hung up.

Total call time: ~5 minutes. Entertainment value: priceless.


  1. This was the time when the caller announced he would hang up. 

Posted in fun | Leave a comment

Bridging ExpressCard to PCIe for LTO

In the mid-1990′s I worked as a photoshop jockey for the Sydney Morning Herald and other Fairfax Media newspapers. At that time we would regularly send a photographer into the field with a large Pelican case that held a digital field kit.

The kit consisted of  a maxed out Powerbook 180 (Greyscale 9.8″ screen, 14MB RAM, 33MHz M68030 CPU, 56.7kbps modem) and a Nikon LS-1000 film scanner with associated cables and power supplies. The kit allowed a photographer to shoot film, process the film at a local mini-lab in the field, scan the images, and transfer them to the newspaper office. In practice this allowed comparatively rather rapid delivery of photos into the editorial process.

During my work on LTFS I messed around with a bunch of side projects on the periphery of LTFS. In one of these projects tried to devise a portable system that would support LTFS. My thinking was largely modeled on my earlier experience with the Powerbook-based field kit. I wanted to be able to give someone a portable box that could be used with a laptop and a digital camera that could be used as a videographer field unit. My idea was something that could be carried to any location, would be easy to set up, and allowed the user to dump content onto LTFS tapes and FedEx the tapes back to home base.

As a demonstration unit I assembled an enclosure that held a LTO5 drive, a SAS HBA, and a PCIe to ExpressCard bridge. This enclosure ended up having the footprint of a 15″ Powerbook and was a little taller than a LTO half-high drive. I showed the demo unit at several trade shows and other public events we did for LTFS.

LTO/LTFS portable drive at NAB 2010LTO/LTFS portable drive at NAB 2010

While refining this portable LTFS box I played with a few different ExpressCard->PCIe bridges and some early thunderbolt->PCIe bridges. A recent question on the Mac Enterprise mailing list asked:

“One route is to go with a TB to PCI chassis then SCSI card to LTO but what little I have found says this is not reliable. Ditto TB to SAS then to SAS LTO. Anyone have any experience with setups like this and with what software?

Tape is pretty much a given. This is a situation where people need to be able to go back in time and pick up a file or folder state from 3 months ago.”

My work building this LTFS portable drive provided me with somewhat relevant experience. My answer to the mailing list may be useful for a broader audience so I’ve included it below:

My experience using both thunderbolt->PCIe and ExpressCard->PCIe bridges has largely been successful. I used the bridges with both SAS and Fibre Channel HBAs to connect LTO4 and LTO5 drives to a few different OS X machines (and some Dell and Lenovo systems). In all cases I found the ExpressCard/Thunderbolt->PCIe bridges reliable.

The few problems I encountered were always traced back to the HBAs and/or LTO drive not supporting or recognizing various sleep state events. (Or OS X not reinitializing the bus after waking up from sleep which amounts to the same thing.)

Fortunately working around this sleep issue is trivial. Just disable system sleep. Allow the monitor to go to sleep if you like, but ensure that the OS X system does not sleep.

If you are going with LTO tape in the scenario you briefly describe, you will need to work out how you want to handle the file meta-data. (Filename, timestamps, folders, permissions, etc.) Traditional tape systems maintain separate parallel databases to hold the metadata. This works, but if the database gets corrupted then all of your tapes suddenly hold garbage.

A more recent approach that is possible with LTO5 and LTO6 is to use LTFS. LTFS is a POSIX-compliant filesystem that uses LTO tape as the primary filesystem datastore. This means the file meta-data is stored on the tape right alongside the file content. As a result, the tape is self-contained and can be mounted exactly like a USB drive can be mounted. Data can be added to the LTFS volume by dragging and dropping files. Restoring data is also a drag and drop operation.

Alternatively, since LTFS is POSIX compliant applications can seek to any byte offset within a file in LTFS. In comparison, traditional systems require a parallel database, or a packaging format like “tar”. Packaging formats typically require an extensive tape traverse during restore. Whereas LTFS supports a direct seek to the data being accessed. LTFS also provides file-system versioning baked into the filesystem. The file-system versioning might be of use in your scenario.

The implementation of LTFS for OS X, Linux, and Windows is free for single tape-drive operation.

Posted in data storage, demo, file-system, hardware, LTFS, open source, os x | Leave a comment

Atari 2600 hardware mod

This week, much of the gaming world focused the release of GTA V from Rockstar Games. I found some spare time to get an old Atari system cleaned up and working by installing an atari 2600 composite output mod.

After 20+ years in a closet our well loved Atari 2600 was a little worse for wear. The console has accumulated grime and the reset switch is no-longer springy. Besides cleaning off the dust and replacing a faulty reset switch I wanted to improve the video output.

The stock Atari 2600 produces an RF modulated video/audio signal as output. On old TVs the RF signal connects to the antenna-in jack. On modern TVs it is often easier to hook up a composite video input. Eliminating the RF modulation should also improve the picture quality.

A few different hardware hacks that produce composite out from an Atari are described on various websites. The output board I selected is one of the better composite output circuit designs.

Parts list

Original console:

While dismantling I found the remains of a previous home-made repair – an elastic band to add springiness to the reset switch. The original spring seems to have gone missing over the years. While the console was dismantled I replaced the reset switch with a new old-stock part.

After installation:
atari 2600 composite output board installed

The video mod board is in the lower right with the rainbow ribbon cable going from the motherboard (inside the thick aluminium RF shield box) to the video mod then a second ribbon cable running up the right side to the composite and s-video output jacks. A couple of pieces of kapton tape hold the video mod board securely in place.

New ports:

Results

Photograph of Pole Position game on Atari 2600 heavy-sixer console before hardware modification. (Photograph of LCD TV screen.)
Pole Position on Atari 2600 before modification

Photograph of Pole Position game on Atari 2600 heavy-sixer console after hardware modification.
Pole Position on atari 2600 composite output after installing video mod board

Overall the mod is a resounding success. RF noise is eliminated, image colors have better saturation, and the console can be connected to standard composite input jacks thereby eliminating the need to re-tune the TV. Installing the video mod involved bypassing the RF modulator and removing the original coax cable.

Basic testing with Pole Position and Pac-Mac during reassembly showed normal operation. I’ll update this post when I’ve tested with other games.

Posted in atari 2600, gaming, hardware | Leave a comment

Create a FreeBSD 9.0 machine image under Linux

This is a guide to building a virtual machine image for KVM that runs FreeBSD with virtio drivers installed.

Requirements:

Notes:

Commands and to be performed under Linux are shown in green.
Commands and file edits to be performed under the FreeBSD guest OS are shown in purple.

Create empty disk image (10GB):

  •  On your Linux host, run the command:
dd if=/dev/null of=freebsd-9_0.raw bs=1024 seek=10485760

Boot FreeBSD installer and install:

  • On your Linux host, run the command:
sudo kvm -vnc 0.0.0.0:54 -drive file=freebsd-9_0.raw,if=ide -net nic -net user -cdrom FreeBSD-9.0-RELEASE-amd64-disc1.iso
  • Connect to the guest OS using a VNC client pointed to port 5954. (5900 + 54 from command above.)
  • Step through the FreeBSD installer:
    • allocate the whole virtual drive to FreeBSD.
    • Configure networking to use IPv4 and DHCP.
  • Reboot the FreeBSD guest.

Install Virt IO drivers:

  • Download the binary package from http://people.freebsd.org/~kuriyama/virtio/
  • Install the package by running the command “pkg_add virtio-kmod-9-0.239473.tbz” in the guest OS.
  • Edit /boot/loader.conf to add the following lines:
virtio_load="YES"
virtio_pci_load="YES"
virtio_blk_load="YES"
if_vtnet_load="YES"
  • Edit /etc/fstab in the guest OS to change the device identifiers to “/dev/vtbd*“. (The following example shows the original device entries commented out followed by the virtio entries.)
# Device Mountpoint FStype Options Dump Pass#
#/dev/ada0p2 / ufs rw 1 1
#/dev/ada0p3 none swap sw 0 0
 /dev/vtbd0p2 / ufs rw 1 1
 /dev/vtbd0p3 none swap sw 0 0
  • Edit /etc/rc.conf in the guest OS to add the line shown in bold. (This line configures FreeBSD to use DHCP when bringing up the “vtnet0″ network interface.):
hostname="freebsd-90"
ifconfig_re0="DHCP"
 ifconfig_vtnet0="DHCP"
sshd_enable="YES"
# Set dumpdev to "AUTO" to enable crash dumps, "NO" to disable
dumpdev="NO"
  • Power off the guest.
  • Relaunch using virtio for the virtual hard drive and virtual network adapter using the following KVM command:
sudo kvm -vnc 0.0.0.0:54 -drive file=freebsd-9_0.raw,if=virtio -net nic,model=virtio -net user
  • Verify correct hard drive and network operation in the guest OS.
  • If the hard drive and network appear to be working correctly then power off the FreeBSD guest.

VirtIO in FreeBSD setup resources:

Posted in operating systems, virtualization | Leave a comment