unix

This could probably be done more elegantly...

In my crontab:

0 * * * * if [ `wget -q -O - http://www.weather.gov/xml/current_obs/KCMX.xml | xpath "/current_observation/temp_f/text()" 2>/dev/null | xargs printf "%1.0f"` -gt 70 ]; then /usr/local/bin/heyu fon B1; fi
10 * * * * /usr/local/bin/heyu foff B1

Basically, if it's greater than 70° F outside, turn on the X10 circuit for my upstairs fan and run for 10 minutes each hour. I give myself bonus points for including XML (which makes it an Enterprise Solution).

A (now) humorous and actually pretty interesting Computer Chronicle episode on Unix from 1985.

Frustrated by low-quality aerial photography on the free web-based mapping services (like Google Maps and TerraServer), I set out to find free sources of the USGS data itself. It comes from a government agency, so it should be simple for us taxpayers to find, right?

After much searching, I found some great resources at Michigan's Center for Geographic Information and the Michigan DNR Aerial Imagery Archive. A relatively complete listing of resources for other states can be found at lgalvin's site.

To use the Michigan CGI data, here's what I did:

1. Select a county from the County Geographic Extent page.
2. On the next page, select the aerial imagery you want to use. For Houghton county, I wanted the Digital Orthophoto Quads - 2005 series.
3. Select and download a quadrangle. These are typically compressed in standard zip format.
   
4. Unpack the file using the unzip command (this works even though the files are self-extracting Windows executables).
5. Download and install FWTools for Linux.
6. Convert the file to TIFF by running something like:
   gdal_translate LAURIUM_NW.ecw LAURIUM_NW.tiff

For the Michigan DNR data, I did this:

1. Select the 1998 series data from the Aerial Imagery Archive page.
2. Select a county of interest, and then a quadrangle.
3. Click on the map of the quadrangle to download the compressed image file.
4. Download and unpack MrSID Decode for Linux.
5. Convert the file to TIFF:
   mrsidgeodecode_linux -i laurium_nw.sid -o laurium_nw.tiff

Either way, you'll now have a high-resolution TIFF that can be used in any number of image manipulation programs. I found the CGI imagery to be the most detailed, and far better than web-based mapping services.

For my 1-Wire home automation project, I'm using an old Soekris net4501 that used to be my home firewall/router, running FreeBSD. It has a 486-class CPU and 64 MB RAM, which presented some challenges for this application, as I wanted to use OWFS (which requires FUSE, and thus a Linux kernel).

I looked at a few small Linux distributions, like DSL (rejected because it doesn't appear to have a non-ISOLINUX/SYSLINUX installation method), Pebble (too old and difficult to update), Pyramid (missing Perl, package management, and sane update/upgrade method), and a few others. I had hoped to use a distribution that was already optimized for flash memory (minimal writes) and very small (I was using a 256-MB Compact Flash card).

Then I found these great instructions for installing Debian on a USB flash drive. Also on this site is deb2flash, a Python script for automating much of the installation. I ended up using a slightly-hacked version of the script (mainly small bug fixes, and replacing the 686 kernel with one for 486).

First, I installed a minimal Debian 4.0 in a VMware VM, and added the parted and debootstrap packages. I pulled the CF card from the net4501, put it in a USB CF adapter, assigned the USB device to the VM, ran deb2flash, and answered the prompts. After quite a while, I had a functional Debian installation on my 256-MB CF card, with a whopping 5.7 MB free:

igor:~# uname -a                                                                                
Linux igor 2.6.22-2-486 #1 Thu Aug 30 23:45:36 UTC 2007 i486 GNU/Linux                          
igor:~# df -h /
Filesystem            Size  Used Avail Use% Mounted on                                          
/dev/hda1             238M  220M  5.7M  98% /

(I do have a 2-GB card ordered, but thought I'd start testing with the card I had.)

To make the installation functional on the net4501, I needed to make the following tweaks.

1. Mount the CF card and chroot.
mkdir /mnt/cf
mount /dev/sdb1 /mnt/cf
chroot /mnt/cf
mount -t proc none /proc

   
2. Rebuild initramfs with ide_generic support.
echo "ide_generic" >> /etc/initramfs-tools/modules
update-initramfs -t -u

   
3. Edit /etc/inittab to enable serial console on ttyS0.
T0:23:respawn:/sbin/getty -L ttyS0 9600 vt100

   
4. Edit /boot/grub/menu.lst for serial console support.
## use serial
serial --unit=0 --speed=9600 --word=8 --parity=no --stop=1

## use terminal
terminal --dumb --timeout=15 serial

   Also add console options to each kernel line.
sed -i '/^kernel/ s/$/ console=tty0 console=ttyS0,9600/' /boot/grub/menu.lst

   

At this point, I was able to put the CF card back in the Soekris and boot it with no problem. I'll have to wait for the larger CF card to start working with OWFS, but that, as they say, is another show.

One of the best features of RH300 is that it teaches defense in depth, without really calling it that. Early on, we learn about SELinux, tcp_wrappers, and Netfilter. From that point on, the services we learn about (such as Samba, NFS, FTP) don't conveniently ignore those security tools, but forces us to make them work in conjunction with them. This really helped me build familiarity with using SELinux and Netfilter (and tcp_wrappers) to protect services I was already familiar with (SMB and NFS).

Today's been an extremely fast-paced day, as we've fallen behind the previous two days. I expect my brain to be completely fried by the end of today's class.

Here's a list of resources I've found for building Red Hat Package Manager (RPM) packages.

• Red Hat's book, Maximum RPM
• the IBM Developer Works article, Packaging software with RPM (Part 1) (Part 2)
• Chapter 8 from Eric Foster-Johnson's book, Red Hat RPM Guide
• Fedora Project's Packaging Guidelines
  
• Creating RPMs (PDF) lecture and exercise from Guru Labs
• Better Living Through RPM, Part 2, from Red Hat Magazine
  

Today I created an RPM for a fairly oddly-installed software package (I do this frequently). If you're not familiar with RPM, one its features is the use of "pristine sources"?the software author's distributed files are used as-is and modified in a repeatable way by the RPM build process.

Anyway, the package in question is ENVI 4.3.1. The ENVI 4.3 distribution conforms to their traditional semi-bizarre installation scheme, but the 4.3.1 patch is more interesting. I wanted to install the 4.3.1 patch as a part of the RPM build process, and found that the patch package was a collection of archives within archives.

It occurred to me to unpack these nested archives with one long pipeline, instead of unpacking to a temporary location, unpacking again, etc. The traditional rpmbuild process seems to favor using gzip -dc and tar -xf - instead of tar -xzf - which, for one of the patch archives, would have looked like this:

gzip -dc envi431linux.tar.gz | tar -xf - envi_platform.tar -O | tar -xf - envi431.linux.tar.gz -O | gzip -dc | tar -xf -

Which makes me chuckle. I first uncompress the distributed file (envi431linux.tar.gz), unpack the named file (envi_platform.tar) from that archive, unpack the desired file (envi431.linux.tar.gz) within that archive, and then uncompress and unpack that last archive. This applies the patch archive without using a temporary directory.

Since I'm already using the probably-not-standard -O option with tar, I could just go the rest of the way and compact it to:

tar -xzf envi431linux.tar.gz envi_platform.tar -O | tar -xf - envi431.linux.tar.gz -O | tar -xzf -

"Pristine sources" and lame installers equals fun scripts!

Tonight I installed a new system disk in the main feep.org server, in response to these ominous warnings:

(da0:sym0:0:0:0): WRITE(10). CDB: 2a 0 0 30 1 1f 0 0 20 0
(da0:sym0:0:0:0): CAM Status: SCSI Status Error
(da0:sym0:0:0:0): SCSI Status: Check Condition
(da0:sym0:0:0:0): Deferred Error: HARDWARE FAILURE asc:19,0
(da0:sym0:0:0:0): Defect list error field replaceable unit: 2
(da0:sym0:0:0:0): Retrying Command (per Sense Data)

The spare disk was already in the server (it was being used for staging backups, but I moved that elsewhere for now), and was identical to the failing one. So all I really needed to do was duplicate the boot block, partition table, disk label, and filesystems. I was able to do most of this while the system was running normally, which kept downtime to a minimum. In the examples, da0 is the failing disk, and da1 is the new target disk.

First, initialize the disk by creating a DOS partition table (mine already had one):

fdisk -I /dev/da1

Install the boot block:

fdisk -b /boot/mbr

Duplicate the disklabel:

disklabel da0s1 | disklabel -R da1s1 -

Create file systems:

newfs /dev/da1s1a
newfs -U /dev/da1s1e
newfs -U /dev/da1s1f
newfs -U /dev/da1s1d

Copy data:

mount /dev/da1s1a /mnt
cd / && tar -cf - --one-file-system --exclude /mnt . \
   | ( cd /mnt && tar -xpf )
umount /mnt
mount /dev/da1s1e /mnt
cd /tmp && tar -cf - . | ( cd /mnt && tar -xpf )
umount /mnt
mount /dev/da1s1f /mnt
cd /usr && tar -cf - . | ( cd /mnt && tar -xpf )
umount /mnt
mount /dev/da1s1d /mnt
cd /var && tar -cf - . | ( cd /mnt && tar -xpf )
umount /mnt

Now I intended to boot into single-user mode to finish with a final rsync on a very quiet system, but I couldn't boot from da0 any longer; it seemed to be getting worse. So I booted from FreeBSD 5.5 Disc 1 and used the Live CD feature to finish up with a few rsyncs (using the rsync binary from da1, since the Live CD doesn't include it).

mkdir /da0 /da1
cd /da0 && mkdir root tmp usr var
cd /da1 && mkdir root tmp usr var
mount /dev/da0s1a /da0/root
mount /dev/da1s1a /da1/root
mount /dev/da0s1e /da0/tmp
mount /dev/da1s1e /da1/tmp
mount /dev/da0s1f /da0/usr
mount /dev/da1s1f /da1/usr
mount /dev/da0s1d /da0/var
mount /dev/da1s1d /da1/var
/da1/usr/local/rsync -avx /da0/root/ /da1/root
/da1/usr/local/rsync -avx /da0/tmp/ /da1/tmp
/da1/usr/local/rsync -avx /da0/usr/ /da1/usr
/da1/usr/local/rsync -avx /da0/var/ /da1/var
umount /dev/da0s1a
umount /dev/da1s1a
umount /dev/da0s1e
umount /dev/da1s1e
umount /dev/da0s1f
umount /dev/da1s1f
umount /dev/da0s1d
umount /dev/da1s1d

That done, I yanked da0, put da1 in its place, and a fresh disk in da1's place. Everything came up with no problems.