This is specific to the restore CD for the Qube 3, other versions not (yet!) tested. The goal here is to boot a VM, bootstrapped by the official Cobalt restore CD, so the Qube OS can be installed on it.
Since this method relies entirely on software from the recovery CD (for the image itself) or relatively simple Open Source (for building the bootstrap ISO), I'm reasonably confident this approach would work for the earlier MIPS-based Qubes too (though this has not been tested). If you're attempting that build, some digging through the content of the NFS root folder on the CD will likely be required. You may also need to import alternative network modules and tell Qemu to use a different emulated network card.
Prerequisites:
The restore CD I had available defaults to installing a pro version of the Qube, with RAID. For simplicity, we can run with two virtual disks. It is possible to vary the size of these if you want, but 10GB will be enough for experimentation.
wget http://people.cobaltqu.be/~theothertom/VMs/Qube3/qube3.iso #restore CD, from https://archive.org/details/qube3
wget http://people.cobaltqu.be/~theothertom/VMs/Qube3/bootloader.iso # Boot CD, with initrd built from restore CD.
#Create disks for target VM
qemu-img create disk1.img 10G
qemu-img create disk2.img 10G
#Run the restore VM
qemu-system-x86_64 -name guest=qube3_restore -machine pc,usb=off,vmport=off,dump-guest-core=off,hpet=off,acpi=on -netdev socket,id=qubenet,listen=:12345 -device tulip,netdev=qubenet -cdrom ./qube3.iso &
Now check the restore VM started correctly, following the on-screen instructions. You will need to accept the license agreement before it will start recovery services.
Once the recovery VM is ready, start the target VM. You will need to type "install" at the boot prompt to start the automatic installation process - by default it will try to boot from disk.
qemu-system-x86_64 -name guest=qube3_target -machine pc,usb=off,vmport=off,dump-guest-core=off,hpet=off,acpi=on -netdev socket,id=qubenet,connect=:12345 -device tulip,netdev=qubenet -drive file=disk1.img,if=ide,index=0,media=disk,format=raw -drive file=disk2.img,if=ide,index=2,media=disk,format=raw -drive file=bootloader.iso,if=ide,index=1,media=cdrom -boot order=d
Once the install is complete, you will need to move the target VM to another virtual network to get access to it. Will also need a client OS of suitable vintage - I had success with Windows 2000.
If you want to quickly check the install, you can re-run with NAT networking to make the VM available:
qemu-system-x86_64 -name guest=qube3_target -machine pc,usb=off,vmport=off,dump-guest-core=off,hpet=off,acpi=on -netdev user,id=qubenet,hostfwd=tcp::1080-:80,hostfwd=tcp::1444-:444,hostfwd=tcp::80-:80,hostfwd=tcp::444-:444 -device tulip,netdev=qubenet -drive file=disk1.img,if=ide,index=0,media=disk,format=raw -drive file=disk2.img,if=ide,index=2,media=disk,format=raw -drive file=output.iso,if=ide,index=1,media=cdrom -boot order=d
Note that the admin UI hardcodes port 444, and the command above uses 1444 to avoid needing root. Once the VM starts, you should be able to see a welcome page at http://localhost:1444
What follows is a walkthrough of the process I followed to build the bootstrap ISO image. Hopefully it will be a helpful starting point if you want to createa similar image for other Qube models, or other legacy Linux appliances with a similar boot process.
For ease, all of these steps are being conducted within VMs. You will need two: one for the restore CD, and a target. They need to be on an isolated (virtual) network, often called "VM only" or "internal". Make sure to avoid bridging the virtual network to your real network, as the restore CD will run a DHCP server, and that could upset other devices.
You will have a far harder time if you're using anything other than Linux as a host OS.
You will also have an easier time if you're using flexible virtulisation software (such as Qemu). In particular, things like lots of choice of emulated network cards will be valuable.
The intended behaviour of the restore CD is to run on a standard x86 PC, then have the Qube hardware bootstrap and run the restore of the OS over NFS. Much of this bootstrap process is controlled/set up by the ROM in the Qube, in particular, network and NFS drivers are contained within the ROM.
As a result of this design, running the restore on 3rd party (virtual!) hardware requires faking aspects of the ROM functionality.
Once booted, the restore CD runs DHCP and NFS services. These are used to run the restore, and are the only available interfaces/services.
The restore system itself makes efforts to confine you to an instance of "less" showing documentation about the restore process, though this does run as root, allowing some annoying investigation of the restore system. You can type !COMMAND to run shell commands for debugging, though trying to modify the running system is challenging as it is not possible to use advanced text editors such as ed or Vim (you may have some luck trying to blindly use sed, but it is likely simpler to just mount the CD on a normal OS).
In my experiments, I've not attempted to modify the ISO to make manipulating the system simpler in the hope that avoiding that approach will lead to a more generic solution to installing Qube OS on a VM.
Whilst debugging the scripts for the restore instance, I've found at times running commands like tcpdump helpful. However, be aware that you don't have a TTY, so you may need to reset the recovery CD VM in order to escape a rogue command as ctrl-C doesn't work.
I've also found it's possible to totally crash the restore CD while installing, though have not yet found a cause for this. Might be worth checking the VM hosting the restore environment if you see errors on your target system.
Since the ROM provides the drivers required to connect to the network, we need to build a bootstrap image. The simplest way to do this is by building a boot CD containing the kernel from the restore image, and a custom initrd containing required drivers and setup scripts.
The tools in the initrd need to be chosen carefully, as technical limitations of the restore kernel mean the largest initrd you can create is 4MB.
The CD I have been testing with can be directly mounted, and contains an NFS root folder. Other CDs may need to be run and mounted over NFS (likely NFS v2 - you may need an older OS to mount it).
You need to download Syslinux/ISOLinux (or install it from your package manager). This provides the ability to boot from CD/ISO. You will also need mkisofs - in Arch it's in the cdrkit package, in Debian it's in genisoimage.
#This is to make copy/paste easier later.
export CD_ROOT="$PWD/CD_root"
mkdir -p CD_root/isolinux
#These are the paths my package manager used - your distro may vary.
cp /usr/lib/syslinux/bios/isolinux.bin CD_root/isolinux/
cp /usr/lib/syslinux/bios/ldlinux.c32 CD_root/isolinux/ # only for newer versions
#The content of this file will be displayed on boot. You can add instructions here if you want.
echo 'Run "install" to run the install' > CD_root/isolinux/intro.txt
You now have a baseline ISO building environment,
When using a kernel that supports dynamic modules, an initial RAM disk (initrd) is generally required, to contain the optional drivers etc. that are needed to mount the root filesytem (which then contains drivers for the remaing hardware). Generally, Linux distros supply scripts to automatically create an appropriate initrd based on the required modules for your system. However, this tooling is not provided, so we need to create one manually.
The initrd needs to contain kernel modules and scripts sufficient to bring up the network and mount NFS. It will also need some default devices etc. in order to be a vaild initial root filesystem.
Once you've built an initrd, make sure you can use the utilities within it by chrooting into the mounted volume and trying to run them (many may be dynamic and depend on other files).
# This is a 4MB initrd - max allowed size
dd if=/dev/zero of=$CD_ROOT/initrd bs=1M count=4
#Create a filesystem with 128 byte inodes, 1024 byte blocks, no options, revision 1
mke2fs -I128 -b 1024 -O none -t ext2 -r 1 $CD_ROOT/initrd
Once the initrd filesystem has been built, mount the initrd and the restore CD.
sudo mkdir /mnt/restore
wget -O qube3.iso https://archive.org/download/qube3/qube3.iso
sudo mount -o loop qube3.iso /mnt/restore
sudo mkdir /mnt/target
sudo mount -o loop $CD_ROOT/initrd /mnt/target
When copying binaries from the restore CD, be aware that they are (generally) dynamically linked. You can get a hint about required libraries with ldd.
# ldd /mnt/cdrom/nfsroot-x86/bin/bash
libtermcap.so.2 => /lib/libtermcap.so.2 (0xf1f27000) libc.so.6 => /lib/libc.so.6 (0xf1e39000)
/lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0xf1f33000)
Some base folders are always required - note that you may need to update the kernel version if you have a different restore CD. You can check the NFS root if you want to see the available modules. I only needed block, fs, net and misc, but there are others if you need extra drivers.
sudo mkdir /mnt/target/{bin,dev,proc,lib,sbin}
sudo mkdir /mnt/target/next-root
sudo mkdir -p /mnt/target/lib/modules/2.2.5-15/{block,fs,net,misc}
I copied these modules, which allow mounting NFS using an emulated Tulip network card. If you want to use another card, you will need to pull in the appropriate driver. Note that loading more drivers is possible once NFS is up, so we don't need RAID drivers in the initrd for the installer.
sudo cp /mnt/restore/nfsroot-x86/lib/modules/2.2.5-15/net/tulip.o /mnt/target/lib/modules/2.2.5-15/net/
sudo cp /mnt/restore/nfsroot-x86/lib/modules/2.2.5-15/misc/pf.o /mnt/target/lib/modules/2.2.5-15/misc/
sudo cp /mnt/restore/nfsroot-x86/lib/modules/2.2.5-15/misc/sunrpc.o /mnt/target/lib/modules/2.2.5-15/misc/
sudo cp /mnt/restore/nfsroot-x86/lib/modules/2.2.5-15/fs/lockd.o /mnt/target/lib/modules/2.2.5-15/fs/
sudo cp /mnt/restore/nfsroot-x86/lib/modules/2.2.5-15/fs/lockd.o /mnt/target/lib/modules/2.2.5-15/fs/
sudo cp /mnt/restore/nfsroot-x86/lib/modules/2.2.5-15/fs/nfs.o /mnt/target/lib/modules/2.2.5-15/fs/
sudo cp /mnt/restore/nfsroot-x86/lib/modules/2.2.5-15/fs/nfsd.o /mnt/target/lib/modules/2.2.5-15/fs/
You will also need some standard devices:
sudo mknod -m 600 /mnt/target/dev/console c 5 1
sudo mknod -m 666 /mnt/target/dev/null c 1 3
sudo mknod -m 600 /mnt/target/dev/tty1 c 4 1
sudo mknod -m 600 /mnt/target/dev/tty2 c 4 1
sudo mknod -m 600 /mnt/target/dev/tty3 c 4 1
sudo mknod -m 600 /mnt/target/dev/tty4 c 4 1
sudo mknod -m 600 /mnt/target/dev/systty c 4 0
sudo mknod -m 600 /mnt/target/dev/ram c 1 1
We also need, an extreamly minimal userland. Note that the binaries need to be stripped, or there is no way they're fitting in 4MB. As with the kernel modules, you may find you need to mess with versions here, depending on your retore CD.
sudo strip /mnt/restore/nfsroot-x86/bin/bash -o /mnt/target/bin/bash
sudo strip /mnt/restore/nfsroot-x86/bin/mount -o /mnt/target/bin/mount
sudo strip /mnt/restore/nfsroot-x86/bin/umount -o /mnt/target/bin/umount
sudo strip /mnt/restore/nfsroot-x86/sbin/ifconfig -o /mnt/target/sbin/ifconfig
sudo strip /mnt/restore/nfsroot-x86/usr/sbin/rpcinfo -o /mnt/target/sbin/rpcinfo
sudo strip /mnt/restore/nfsroot-x86/usr/sbin/rpc.nfsd -o /mnt/target/sbin/rpc.nfsd
sudo strip /mnt/restore/nfsroot-x86/sbin/portmap -o /mnt/target/sbin/portmap
sudo mkdir -p /mnt/target/usr/sbin
sudo strip /mnt/restore/nfsroot-x86/usr/sbin/chroot -o /mnt/target/usr/sbin/chroot
sudo strip /mnt/restore/nfsroot-x86/lib/libtermcap.so.2.0.8 -o /mnt/target/lib/libtermcap.so.2.0.8
sudo strip /mnt/restore/nfsroot-x86/lib/libc-2.1.1.so -o /mnt/target/lib/libc-2.1.1.so
sudo strip /mnt/restore/nfsroot-x86/lib/ld-2.1.1.so -o /mnt/target/lib/ld-2.1.1.so
sudo ln -s libtermcap.so.2.0.8 /mnt/target/lib/libtermcap.so.2
sudo ln -s ld-2.1.1.so /mnt/target/lib/ld-linux.so.2
sudo ln -s libc-2.1.1.so /mnt/target/lib/libc.so.6
sudo cp /mnt/restore/nfsroot-x86/sbin/insmod.static /mnt/target/sbin/
Finally, we need a script to load drivers, mount filesystems, and initialise the target system. This script needs to be called /mnt/target/linuxrc:
#!/bin/bash
#The environment that runs this script is pretty limited - lots of debug logging really helps know where it fell over
echo Hello, world
echo "Mounting /proc..."
# mount needs -n, as mtab isnt writable
/bin/mount -n -t proc proc /proc
echo "Loading tulip driver..."
/sbin/insmod.static /lib/modules/2.2.5-15/net/tulip.o
echo "Load NFS modules..."
/sbin/insmod.static /lib/modules/2.2.5-15/misc/sunrpc.o
/sbin/insmod.static /lib/modules/2.2.5-15/fs/lockd.o
/sbin/insmod.static /lib/modules/2.2.5-15/fs/nfs.o
/sbin/insmod.static /lib/modules/2.2.5-15/fs/nfsd.o
#TODO: DHCP during boot
/sbin/ifconfig lo 127.0.0.1/8
/sbin/ifconfig eth0 10.0.0.5/8
#The (my?) restore CD listens on 10.0.0.254, so this address is widely hardcoded in these scripts.
echo "RPCinfo:" /sbin/rpcinfo -p 10.0.0.254
echo "Run portmap..."
/sbin/portmap
echo "Run nfsd..."
/sbin/rpc.nfsd
echo "Mount..."
mount -n 10.0.0.254:/nfsroot-x86 /next-root -orsize=8192,wsize=8192,nolock,ro,vers=2
echo "Load additional modules via NFS for RAID support..."
/sbin/insmod.static /next-root/lib/modules/2.2.5-15/block/raid0.o
/sbin/insmod.static /next-root/lib/modules/2.2.5-15/block/raid1.o
/sbin/insmod.static /next-root/lib/modules/2.2.5-15/block/raid5.o
echo "Remount /proc..."
/bin/mount -n -t proc proc /next-root/proc
/bin/umount /proc
echo "Format a RAM disk for /tmp..."
/next-root/usr/sbin/chroot /next-root /sbin/mkfs.ext2 /dev/ram2
echo "Mount a RAM disk for /tmp..."
/bin/mount -n /next-root/dev/ram2 /next-root/tmp
echo "Current partition table:"
/next-root/usr/sbin/chroot /next-root /sbin/sfdisk -d /dev/hda
PROJ_DIR=/bto-sync/production/4100WG_1.12 PRODUCT=4100WG BUILD=0.1 LANGUAGE=en VENDOR=Cobalt RELEASE=0.1 CODENAME="Angry Herring" TMP_DIR=/tmp PATH="/bin:/usr/bin:/sbin:/usr/sbin" MNT_DIR=/tmp/mntdir /next-root/usr/sbin/chroot /next-root /bin/bash -c 'cd /bto-sync/production/4100WG_1.12/installer/; ./build_release'
echo "Ignore the LCD..."
/next-root/bin/mkdir /real-root
/bin/mount -n /next-root/dev/md1 /real-root
# Find commands that interact with the LCD, and stub them. "LCD is not" is mentioned in errors using the LCD.
# Using two methods in the hope of making it easy to get everything.
/next-root/usr/sbin/chroot /real-root /bin/bash -c '/bin/ls /sbin/l?d* | while read i; do echo rm -f $i; /bin/rm -f $i; echo cp /bin/true $i; /bin/cp /bin/true $i; done'
/next-root/usr/sbin/chroot /real-root /bin/bash -c '/bin/grep -rl "LCD is not" /sbin/ | while read i; do echo rm -f $i; /bin/rm -f $i; echo cp /bin/true $i; /bin/cp /bin/true $i; done'
/next-root/bin/umount -n /next-root/dev/md1
echo "Sync..."
/next-root/usr/sbin/chroot /next-root /bin/sync
echo "Unmount all filesystems..."
/next-root/bin/umount -n -a
echo "Ended install/hacks - booting installed system..."
This script must be executable.
sudo chmod +x /mnt/target/linuxrc
During boot, failures have an annoying habit of being silent. As a quick test, you can chroot into your init rd and confirm that everything can at least run.
sudo chroot /mnt/target /bin/bash
Then run any binaries in your init scripts. If your chroot silently fails to start, it is likely that bash doesn't have all the libraries it requires available.
Once you've finished testing, make sure to unmount theinitrd:
sudo umount /mnt/target
The installer doesn't add a boot loader or initrd, so for ease we can bootstrap the system with the same ISO we used for the install, using a slight variation on the initrd.
First, create a copy of the initrd for the installer:
cp $CD_ROOT/initrd $CD_ROOT/bootrd
Then mount it as before:
sudo mount -o loop $CD_ROOT/bootrd /mnt/target
For the boot image, we may need the RAID drivers (depending on the model being emulated):
sudo cp /mnt/restore/nfsroot-x86/lib/modules/2.2.5-15/block/raid0.o /mnt/target/lib/modules/2.2.5-15/block/
sudo cp /mnt/restore/nfsroot-x86/lib/modules/2.2.5-15/block/raid1.o /mnt/target/lib/modules/2.2.5-15/block/
sudo cp /mnt/restore/nfsroot-x86/lib/modules/2.2.5-15/block/raid5.o /mnt/target/lib/modules/2.2.5-15/block/
You also need to replace the linuxrc script:
#!/bin/bash
echo Hello, world
echo "Mounting /proc..."
/bin/mount -n -t proc proc /proc
# In theory, the real system should be able to auto-load these from disk. This feels safer though
echo "Loading tulip..."
/sbin/insmod.static /lib/modules/2.2.5-15/net/tulip.o
echo "Load RAID modules..."
/sbin/insmod.static /lib/modules/2.2.5-15/block/raid0.o
/sbin/insmod.static /lib/modules/2.2.5-15/block/raid1.o
/sbin/insmod.static /lib/modules/2.2.5-15/block/raid5.o
echo "Switching root and booting..."
At this point, the script ends and control passes to the kernel which will mount the filesystem specified on the commad line and run init.
Make sure to unmount before continuing:
sudo umount /mnt/target
Now we have:
All that's left is copying the kernel from the recovery CD (make sure to copy the kernel, not a symlink):
cp -L /mnt/restore/nfsroot-x86/boot/vmlinuz $CD_ROOT/
And creating a config file for ISOlinux in $CD_ROOT/isolinux/isolinux.cfg:
display intro.txt
prompt 1
default boot_installed_system
#Timeout is in centiseconds
timeout 100
label boot_installed_system
kernel /vmlinuz
# You may need to vary the root=, depending on what you installed.
append initrd=/bootrd root=/dev/md1
label install
kernel /vmlinuz
append initrd=/initrd root=/dev/md1
When customising this file, be aware that filenames will get mangled while making the ISO. Best to stick to names in ASCII that have 8 characters or fewer.
You can now build the ISO with mkisofs:
mkisofs -o bootloader.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table CD_root
You can now follow the steps in "quickstart", but without downloading the bootloader.iso.