My Profile Photo

AndrewCz


Using liberty-minded opensource tools, and using them well


RESPIN From Scratch





Updated old version

To keep the old version up-to-date after I borked my previous install, I used virtualbox to do an install and update.

Prep

  1. Install to disk ~25G with respin-installer
  2. # apt-mark unhold linux-compiler-gcc* linux-headers* linux-libc-dev* linux-image*
  3. # apt update && apt upgrade && <kernel>
  4. # apt-mark hold linux-compiler-gcc* linux-headers* linux-libc-dev* linux-image*
  5. # rm -rf /lib/live/mount
  6. # respin dist
    • # watch -n .5 df -h
    • # watch -n .5 du -ch /home/respin/respin/ISOTEMP/live/filesystem.squashfs
  7. Export /home/respin/respin/RESPIN-MMM-YY.iso
    • Shared folder name=jimmy
    • # mkdir ~/jimmy
    • # mount -t 9p -o trans=virtio jimmy ~/jimmy
    • # mv /home/respin/respin/RESPIN.iso ~/jimmy/RESPIN-MMM-YY.iso

VM Setup

I should set this up using Vagrant, but I’m still not at the point where I feel the need to learn that, as I’ll only be setting these up once. So I pulled down the netinstalls of Debian Jessie, and will get those up and running. I used macvtaps on the NICs so that I could ssh into them from my control machine.

I also set up the partition type with a mix of LVM and primary partitions. I have one 3GB primary partition for /boot, and one primary partition that serves as my LVM base - that one being about 17GB. In that I put ~8GB to be / and ~10GB to be /home. I should be safe with that.

And even if I do fuck up, I can always take snapshots and revert.

Later, I made another VM, and just gave it the single / partition and gave it 60G. This way I don’t have to worry about running out of space.

Take it to testing?

This is as simple as changing its /etc/apt/sources.list entries that specify jessie to testing. After that, an apt update && apt upgrade does the trick.

Live-build setup

Easy:

# apt install live-build

First run

Following the manual (linked below) I created a build directory, and built a vanilla image.

# cd ~
# mkdir live-build && cd live-build
# lb config
# lb build

Then I tested it by transferring it to my machine, where I ran it using qemu. OK, so it works, but how to customize it?

Customization

ISOLINUX

I thought I’d start out with the bootloader, as that’s the first thing that pops up when I test it, and is required to work before anything else. From my days working with the respin installer, I had the suite of isolinux files ready to go. First I ran lb clean --purge and rm -rf .build to get rid of everything. Then I made a new directory ./config/bootloaders/isolinux and put all of my isolinux files into there. After that, I issued another lb build to generate a new testable iso just to make sure everything went according to plan.

Bootsplash with Plymouth

Plymouth is a bootsplash animation engine that will show pictures instead of the terminal output during boot. It’s really the only good one of it’s kind. I have a theme that I’ve used with my RESPIN ISOs for a while - Tux. I have to make sure that theme is present and selected for plymouth to use.

Files

There’s a skel-like directory where the entire root filesystem can be mimicked in order to place various files in place. So for this, I need a directory usr/share/plymouth/themes. That structure goes inside of the directory live-build-root/config/includes.chroot. I’ll get to why the chroot later.

Hook

There’s also a command that I need to issue to activate tux as the selected plymouth theme. /usr/sbin/plymouth-set-default-theme tux will do the trick, however, I have to call it while building the image. In order to do this, I need it to execute in the chroot before building the ISO.

This I can do with a hook. They reside in live-build-root/config/hooks. It is just a simple shell script that in fact calls exactly that command I outlined earlier, and exits. This sets up plymouth to run during the live boot. There was already plenty of numbered hooks there, up to 0450, so I numbered mine 0800-config-plymouth-theme.hook.chroot.

Getting back to the files, you can now see why I had to specify includes.chroot. If I put them under live-build-root/config/includes they would not be placed in the filesystem until too late, and plymouth would fail. Well, did fail, as I learned by my own mistake here. I’m pretty sure that hooks follow the same in that if there is no .chroot on the end, then it would wait until too late to be executed - but there are some commands that I want to run on boot, so we’ll see. I have a feeling this isn’t my last foray into hooks.

Package-list

I obviously also needed plymouth itself installed. In live-build-root/config/package-lists I created a file named respin.list with the following:

plymouth
plymouth-themes

The second is needed for the Tux theme to work. Otherwise, that’s fairly straightforward, and how I expect to install the majority of the programs that will be featured on the live ISO.

Time and CPU Cycles

It was about this time that I started to detest how long it took for an image to build. It’s almost 20 minutes on average with two cores. So, because virt-manager allows for expansion of hardware inbetween boots, I increased the formerly 2 CPU core VM to 4 cores. The RAM usage was minimal and the I/O I couldn’t do anything about.

User- and hostname

I wanted to make sure the username and the hostname were correct. Typically that’s just appending a bit of into to the boot parameters, but there’s a specific file to do that in with live-build scripts. I had to look at live-build-root/config/binary to find the place to append the parameters. After the components keyword in the LB_BOOTAPPEND_LIVE variable I put username=ghost and hostname=the_worst. The password will be a bit trickier and come later.

Look at how Kali does Passwords for a clue about what’s coming up

ZSH

Since the user ghost is created at boot time (by the boot parameters above) any call to chsh would have to come after the user is created. So I need to put a boot hook to be ran at…well, boot. So a new directory needed to be created: live-build-root/config/includes.chroot/lib/live/config/ and I put the shell script in that directory 0801-config-change-shell-to-zsh.hook.chroot.

It works, but since I didn’t put any of my dotfiles in there, login prompts me for the first-time zsh configuration script.

Dotfiles

Like the ones I put up on github.

extract

I have an extract alias in my dotfiles that automatically extracts various types of compressed files based on the filetype. This requires that I have various compression programs installed, including 7zip, xz, unrar, unar, etc.

LightDM

There is a live-config parameter that specifies to autologin LightDM. However, it’s easier to set up the conf file under the live directory that gets run every live boot. I mean, I’ll have to consider what happens when one would install it, but that’s not what I’m doing here right now. Regardless, here’s live-build-root/config/includes.chroot/etc/lightdm/lightdm.conf:

[SeatDefaults]
autologin-user=ghost
autologin-user-timeout=0
greeter-hide-users=true
#session-startup-script=/usr/local/bin/update-n-upgrade.sh

The session-startup-script is for the terminal popup at the beginning of the live session.

Password

To emulate Tails, I’m going to change the password of ghost every boot. Unlike tails, I will not let the user choose the password. I’m going to generate a random password composed of two words from /usr/share/dict/words that have no special characters, no numbers, and are less than six letters apiece (but no more than 12, I don’t want to tax ones brain power too much). That’s one hell of an if statement. That password will be given to the user during the session-setup-script, and act as the sudo password for the duration of the session.

I also have to make sure that only root can execute the script, but that it gets run at the beginning of the session. Also, I’d love to have the startup script be able to regenerate the password if the user doesn’t like the one generated for them. But that script should only be able to be called by root - which means that the session startup would have to be run by root and displayed inside of the X session of ghost. To do this, I’ll have to call it from the session-setup-script as that runs with root permissions (according to the example lightdm.conf file).

Well, I fucked up

Not too bad, and I’m not even sure what I did, but it’s to a point where I keep getting the same error message over and over again - even with a raw lb config && lb build command. Luckily two things saved me.

One, I’d been working on this in a git repo, so I can roll back to before the errors started occurring. Also, I was able to just reset the VM to stock. I had to install my staples as well as the live-build package again, but it seems to be working fine now. For those in the future, the message was:

xorriso : NOTE : -as mkisofs: Ignored option '-cache-inodes'
xorriso : WARNING : -volid text problematic as automatic mount point name
xorriso : WARNING : -volid text is too long for Joliet (28 > 16)
xorriso : WARNING : -volid text does not comply to ISO 9660 / ECMA 119 rules
Added to ISO image: directory '/'='/binary'
xorriso : UPDATE : 1518 files added in 1 seconds
xorriso : UPDATE : 1518 files added in 1 seconds
xorriso : NOTE : Copying to System Area: 432 bytes from file '/usr/lib/ISOLINUX/isohdpfx.bin'
xorriso : FAILURE : Cannot find in ISO image: -boot_image ... bin_path='/isolinux/isolinux.bin'
xorriso : NOTE : -return_with SORRY 32 triggered by problem severity FAILURE

This appears to be due to meddling with the config/binary file, and the packages-list/respin.list file. After messing around, this happened a couple more times, but built nonetheless. As long as the line with FAILURE wasn’t present, the ISO build successfully. Or it could’ve been because of faulty isolinux menus. One of those three. Either way, I re-did all three of those, and it’s fixed now.

Size

I got lucky with my partition sizing, however I did want to note that there is at times in excess of 4.5GB, probably somewhere around 5GB. Including the system files itself, the root partition of 7.3GB got over 90% full in the course of the live-build process. Anything less than that (or building two in that partition) would be impossible.

GUI

Compton

tint2

conky

lxappearance && obconf

~/.config/gtk*
~/.config/openbox/rc.xml

Respectively

terminator

Eatmydata

I saw that the tails script used eatmydata to speed up the build process, as well as using time the same way that I did. Now my build command is:

# time eatmydata lb build

It keeps giving me errors with no noticeable speedup, so I’ll have to do some more digging on how to correctly utilize it.

lightdm autologin

I have a feeling that I need some other daemon to keep my session open, but I’m not sure what. I’ll start installing other things and see if that helps.

Actually, this works for autologin, so something I put in is extraneous:

# mkdir test && cd test
# lb config
# cat << EOF >> ./config/package-lists/live.list.chroot
openbox
obsession
obconf
obmenu
openbox-menu
xinit
lightdm
EOF
# sed -i 's/^LB_BOOTAPPEND_LIVE.*/LB_BOOTAPPEND_LIVE="boot=live components hostname=the_worst quiet splash" ./config/binary
# time eatmydata lb build

It’s not:

  • /etc/skel
  • /etc/lightdm
  • the extra programs
  • /usr
  • isolinux/
  • /live

    it was the append to the bootloader

So here was my problem. When LightDM is used as the DM, and the hostname variable is set on the kernel append line, it does not successfully auto login - Ubuntu seems to do this:

Changing username and password will not change login behaviour, because this is done in /usr/share/initramfs-tools/scripts/casper-bottom/15autologin.

Nope…not even that…

Hostnames and underscores

So, per a couple of RFCs, apparently hostnames can’t have underscores in them. Therefore, when I put an underscore in the hostname, lightdm was failing out initially. It would work if I wanted to log in manually, but it wouldn’t catch it for an autologin.

ZDoom

ZDoom doesn’t have a .deb package that I can get. It only gets compiled from source. For this, I am going to do this before I put it together.in the auto/config bash script. Basically it’s just the installation setup that’s on the wiki.

The zdoom executable and zdoom.pk3 archive need to be in the same directory. I’ve put them in /opt/zdoom, and made /usr/local/bin/zdoom be a shell script that calls it in that location.

Archives

Live-build can add additional repos during build time using directives in the config/archives directory. There it is possible to specify a repo line as one might do in sources.list to retrieve a package. However, a separate file is required that includes the key.

By default, the repos aren’t added to the live iso. This is a problem, because I would definitely like to have the ability to update on boot of the system. So:

If you add the same line to config/archives/live.list.binary, the repository will be added to your live system’s /etc/apt/sources.list.d/ directory.

AirVPN client

AirVPN maintains a repo for its client at eddie.website. Specifying that in the airvpn.list.chroot file, I only had to make sure to add their gpg key in airvpn.key.chroot. Naming apparently matters here. Luckily everything was easy to find at airvpn.org/linux/.

Kali Tools

Kali was a little bit harder

init-system-helpers

John the Ripper

For some dumb reason, john was put in /usr/sbin/john. To allow execution as user, I’ll have to add a NOPASSWD condition to /etc/sudoers. Since this would typically be called from the menu, it should be enough to allow execution only. In this setup, a user calling which john would return a command not found. But like was explained above, in this scenario, that’s OK. However, what’s not OK is that john still results in command not found. So I’ll have to put a script in /usr/local/bin that calls /usr/sbin/john for the user as a type of redirection.

/etc/sudoers

Since I’ll be editing the /etc/sudoers file from time to time, yet it needs to be in mode 0440 to be considered valid. For this, I’ll add a line to do so in the config script. I messed this up the first time when I built off a previous build, but found a workaround to the error I came up with. The live-build told me that I couldn’t sudo due to /etc/sudoers being world-writable. I obviously couldn’t chmod it, as the user wasn’t permitted to do so, and sudo wasn’t working! In the end I used pkexec in the same way that I would use sudo, and changed the permissions on the file to get sudo back.

While I’m here, I’ll also add the sudo password to the update and upgrade commands. Now apt-get {update,upgrade} doesn’t have to use a sudo password, so the update script can run in the background if necessary.

Laptop-mode-tools

Laptop-mode-tools is a program that, eh…more here.

Conf

All the conf files are under /etc/laptop-mode.

Service

Tmuxinator menu options

Creating the file is sufficient for tmuxinator’s operation. I used a here document to create the file. Turns out if you put the delimiter (EOF) in quotes, then it won’t do any substitution, but if it is not, it will substitute the variables.

notify-send

The xfce4-notifyd daemon accepts input from the command notify-send, which allows me to send an error as a notification if the setup does not go as planned. I’ll also install a whisperback system later to allow anonymous bug reporting.

Repos

Tails

Tails is very hard to find. The git repos are at immerda.ch, and the apt repos are at boum.org. The signing key is either at their own location or at “your favorite keyserver”. That’s good they’re distributing it, but the rest doesn’t make much sense on the surface. I’ll not try to understand the reasoning, but only hope that there’s good reason for it.

Whisperback

The reason why I wanted Tails is to get whisperback, the anonymous bug reporting software. To do this I’ll have to set up a mail server with some custom configuration in order to only allow limited bug reporting to be sent only to me. That’ll be fun.

BunsenLabs

I added the repos and the repo keys. All the backports and the unstable packages I could find in other repos, or just didn’t need. The only one I added was the custom packages that BL maintains.

Live

So when I was putting together the preferences, I added one for live with a Pin-Priority of 1001. But when I did, it put the live.list.binary file, the one with all the other repolists that was only meant for the binary stage for upgrade purposes, into the chroot. Subsequently, it found duplicate repo setups when it went to install certain files, and failed to build. I only hope that the main repos take precedence for this setup. And it seems like they do, everything built fine with the three repos.

This seems like a bug, and I will try to report it as such.

HTTPS and Apt

Apt requires apt-transport-https to use repositories that redirect to secure repos. In auto/config this is passed with the --debootstrap-options '--include=apt-transport-https,ca-certificates' option to lb config. If this is added after completing the bootstrap stage, you’ll have to flush the cache, or else it will skip the entire phase.

Also, make sure that they keys to the repos are correct, otherwise it will fail horribly, and sometimes after 2+ hours of working, even after the creation of the sqashfs. That was super disappointing when I saw that error message.

Dpkg-name

live-build needs any packages that are put in config/packages.chroot, to be named according to the expected naming scheme. That is <packagename>-<version>.deb. This specification is in every .deb file, and there’s a program called dpkg-name that can rename it if it’s not named correctly. To make the command idempotent, you can pass the -o flag that overwrites the .deb file that is named the same, in the event that you’re re-running the auto/config script.

Skype

Skype has a .deb package that I’m using in RESPIN. I don’t want to keep re-downloading that, so I decided to test for it. Of course, it could be named one of two things: either the name I downloaded it as, or the name dpkg-name corrected it to. To test for this then, I needed to see test for ^skype* in regex terms. However, [[ does not accept multiple arguments to, say, -f. So, expanded, it would look like if [[ -f skypeforlinux-64.deb skype-5.1.0.1-amd64.deb ]], which wouldn’t work. The accepted way to search directories is to nest a test statement inside of a for loop.

for file in ${lb_pwd}/config/packages.chroot; do
    if [[ -f "${file}" ]] && [[ ${file} =~ ^skype* ]]; then
        need_file=false
        break
    fi
done