I recently wrote a program that records all tty activity. That means bash sessions, ssh, raw tty access, screen and tmux sessions, the lot. I used script. The latest version of my software can be found on github.

Note that it’s been tested only with bash so far, and there’s no encryption built in.

To just record all shell commands typed, use the standard eternal history tricks (bash).

One of the more useful things I did was to start logging all my technical problems. Whenever I hit a problem, I write an entry in problem-log.txt. Here’s an example

2022-08-02
Q: Why isn't the printer working? [ SOLVED ]
A: sudo cupsenable HL-2270DW

// This isn't in the problem log, but the issue is that CUPS will silently disable the printer if it thinks there's an issue. This can happen if you pull a USB cord mid-print.

I write the date, the question, and the answer. Later, when I have a tough or annoying problem, I try to grep problem-log.txt. I’ll add a note if I solve a problem using the log, too.

This was an interesting project to look at 5 years later. I didn’t see benefits until 1-2 years later. It does not help me think through a problem. It’s hard to remember to do. But, over time it’s built up and become invaluable to me. I hit a tricky problem, and I can’t immediately find an answer on the web. I find out it’s in problem-log.txt. And, someone’s written it exactly with my hardware (and sometimes even my folder names) correctly in there. Cool!

Here’s another example:

2018-10-21
Q: How do I connect to the small yellow router?

Not every problem gets solved. Oh well.

I want my debian boot to work as follows:

1. If it’s in my house, it can boot without my being there. To make that happen, I’ll put the root disk key on a USB stick, which I keep in the computer.
2. If it’s not in my house, it needs a password to boot. This is the normal boot process.

As in part 1, this guide is debian-specific. To learn more about the Linux boot process, see part 1.

First, we need to prepare the USB stick. Use ‘dmesg’ and/or ‘lsblk’ to make a note of the USB stick’s path (/dev/sdae for me). I chose to write to a filesystem rather than a raw block device.

sudo mkfs.ext4 /dev/sdae # Make a filesystem directly on the device. No partition table.
sudo blkid /dev/sdae # Make a note of the filesystem UUID for later

Next, we’ll generate a key.

sudo mount /dev/sdae /mnt
sudo dd if=/dev/urandom of=/mnt/root-disk.key bs=1000 count=8

Add the key to your root so it can actually decrypt things. You’ll be prompted for your password:

sudo cryptsetup luksAddKey ROOT_DISK_DEVICE /mnt/root-disk.key

Make a script at /usr/local/sbin/unlockusbkey.sh

#!/bin/sh
USB_DEVICE=/dev/disk/by-uuid/a4b190b8-39d0-43cd-b3c9-7f13d807da48 # copy from blkid's output UUID=XXXX

if [ -b $USB_DEVICE ]; then
  # if device exists then output the keyfile from the usb key
  mkdir -p /usb
  mount $USB_DEVICE -t ext4 -o ro /usb
  cat /usb/root-disk.key
  umount /usb
  rmdir /usb
  echo "Loaded decryption key from USB key." >&2
else
  echo "FAILED to get USB key file ..." >&2
  /lib/cryptsetup/askpass "Enter passphrase"
fi

Mark the script as executable, and optionally test it.

chmod +x /usr/local/sbin/unlockusbkey.sh
sudo /usr/local/sbin/unlockusbkey.sh | cmp /mnt/root-disk.key

Edit /etc/crypttab to add the script.

root PARTLABEL=root_cipher none luks,keyscript=/usr/local/sbin/unlockusbkey.sh

Finally, re-generate your initramfs. I recommend either having a live USB or keeping a backup initramfs.

sudo update-initramfs -u

[1] This post is loosely based on a chain of tutorials based on each other, including this
[2] However, those collectively looked both out of date and like they were written without true understanding, and I wanted to clean up the mess. More definitive information was sourced from the actual cryptsetup documentation.

In this article, I migrate an existing debian 10 buster release, from an unencrypted root drive, to an encrypted root. I used a second hard drive because it’s safer–this is NOT an in-place migration guide. We will be encrypting / (root) only, not /boot. My computer uses UEFI. This guide is specific to debian–I happen to know these steps would be different on Arch Linux, for example. They probably work great on a different debian version, and might even work on something debian-based like Ubuntu.

In part 2, I add an optional extra where root decrypts using a special USB stick rather than a keyboard passphrase, for unattended boot.

Apologies if I forget any steps–I wrote this after I did the migration, and not during, so it’s not copy-paste.

Q: Why aren’t we encrypting /boot too?

1. Encrypting /boot doesn’t add much security. Anyone can guess what’s on my /boot–it’s the same as on everyone debian distro. And encrypting /boot doesn’t prevent tampering–someone can easily replace my encrypted partition by an unencrypted one without my noticing. Something like Secure Boot would resist tampering, but still doesn’t require an encrypted /boot.
2. I pull a special trick in part 2. Grub2’s has new built-in encryption support, which is what would allow encrypting /boot. But grub2 can’t handle keyfiles or keyscripts as of writing, which I use.

How boot works

For anyone that doesn’t know, here’s how a typical boot process works:

1. Your computer has built-in firmware, which on my computer meets a standard called UEFI. On older computers this is called BIOS. The firmware is built-in, closed-source, and often specific to your computer. You can replace it with something open-source if you wish.
2. The firmware has some settings for what order to boot hard disks, CD drives, and USB sticks in. The firmware tries each option in turn, failing and using the next if needed.
3. At the beginning of each hard disk is a partition table, a VERY short info section containing information about what partitions are on the disk, and where they are. There are two partition table types: MBR (older) and GPT (newer). UEFI can only read GPT partition tables. The first thing the firmware does for each boot disk is read the partition table, to figure out which partitions are there.
4. For UEFI, the firmware looks for an “EFI” partition on the boot disk–a special partition which contains bootloader executables. EFI always has a FAT filesystem on it. The firmware runs an EFI executable from the partition–which one is configured in the UEFI settings. In my setup there’s only one executable–the grub2 bootloader–so it runs that without special configuration.
5. Grub2 starts. The first thing Grub2 does is… read the partition table(s) again. It finds the /boot partition, which contains grub.cfg, and reads grub.cfg. (There is a file in the efi partition right next to the executable, which tells grub where and how to find /boot/grub.cfg. This second file is confusingly also called grub.cfg, so let’s forget it exists, we don’t care about it).
6. Grub2 invokes the Linux Kernel specified in grub.cfg, with the options specified in grub.cfg, including the an option to use a particular initramfs. Both the Linux kernel and the initramfs are also in /boot.
7. Now the kernel starts, using the initramfs. initramfs is a tiny, compressed, read-only filesystem only used in the bootloading process. The initramfs’s only job is to find the real root filesystem and open it. grub2 is pretty smart/big, which means initramfs may not have anything left to do on your system before you added encryption. If you’re doing decryption, it happens here. This is also how Linux handles weird filesystems (ZFS, btrfs, squashfs), some network filesystems, or hardware the bootloader doesn’t know about. At the end of the process, we now have switched over to the REAL root filesystem.
8. The kernel starts. We are now big boys who can take care of ourselves, and the bootloading process is over. The kernel always runs /bin/init from the filesystem, which on my system is a symlink to systemd. This does all the usual startup stuff (start any SSH server, print a bunch of messages to the console, show a graphical login, etc).

Setting up the encrypted disk

First off, I used TWO hard drives–this is not an in-place migration, and that way nothing is broken if you mess up. One disk was in my root, and stayed there the whole time. The other I connected via USB.

Here’s the output of gdisk -l on my original disk:

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048         1050623   512.0 MiB   EF00  # EFI, mounted at /boot/efi
   2         1050624       354803711   168.7 GiB   8300  # ext4, mounted at /
   3       354803712       488396799   63.7 GiB    8200  # swap

Here will be the final output of gdisk -l on the new disk:

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048          526335   256.0 MiB   EF00  efi # EFI, mounted at /boot/efi
   2         1050624       135268351   64.0 GiB    8200  swap # swap
   3       135268352       937703054   382.6 GiB   8300  root_cipher # ext4-on-LUKS. ext4 mounted at /
   4          526336         1050623   256.0 MiB   8300  boot # ext4, mounted at /boot

1. Stop anything else running. We’re going to do a “live” copy from the running system, so at least stop doing anything else. Also most of the commands in this guide need root (sudo).
2. Format the new disk. I used gdisk and you must select a gpt partition table. Basically I just made everything match the original. The one change I need is to add a /boot partition, so grub2 will be able to do the second stage. I also added partition labels with the c gdisk command to all partitions: boot, root_cipher, efi, and swap. I decided I’d like to be able to migrate to a larger disk later without updating a bunch of GUIDs, and filesystem or partition labels are a good method.

3. Add encryption. I like filesystem-on-LUKS, but most other debian guides use filesystem-in-LVM-on-LUKS. You’ll enter your new disk password twice–once to make an encrypted partition, once to open the partition.

   cryptsetup luksFormat /dev/disk/by-partlabel/root_cipher
   cryptsetup open /dev/disk-by-partlabel/root_cipher root
   

4. Make the filesystems. For my setup:

   mkfs.ext4 /dev/disk/by-partlabel/root
   mkfs.ext4 /dev/disk/by-partlabel/boot  
   mkfs.vfat /dev/disk/by-partlabel/efi
   

5. Mount all the new filesystems at /mnt. Make sure everything (cryptsetup included) uses EXACTLY the same mount paths (ex /dev/disk/by-partlabel/boot instead of /dev/sda1) as your final system will, because debian will examine your mounts to generate boot config files.

   mount /dev/disk/by-partlabel/root /mnt
   mkdir /mnt/boot && mount /dev/disk/by-partlabel/boot /mnt/boot 
   mkdir /mnt/boot/efi && mount /dev/disk/by-partlabel/efi /mnt/boot/efi
   mkdir /mnt/dev && mount --bind /dev /mnt/dev # for chroot
   mkdir /mnt/sys && mount --bind /sys /mnt/sys
   mkdir /mnt/proc && mount --bind /dev /mnt/proc
   

6. Copy everything over. I used rsync -axAX, but you can also use cp -ax. To learn what all these options are, read the man page. Make sure to keep the trailing slashes in the folder paths for rsync.

   rsync -xavHAX / /mnt/ --no-i-r --info=progress2
   rsync -xavHAX /boot/ /mnt/boot/
   rsync -xavHAX /boot/efi/ /mnt/boot/efi/
   

7. Chroot in. You will now be “in” the new system using your existing kernel.
   chroot /mnt

8. Edit /etc/crypttab. Add:
   root PARTLABEL=root_cipher none luks

9. Edit /etc/fstab. Mine looks like this:

   /dev/mapper/root / ext4 errors=remount-ro 0 1
   PARTLABEL=boot /boot ext4 defaults,nofail 0 1
   PARTLABEL=efi /boot/efi vfat umask=0077,nofail
   PARTLABEL=swap none swap sw,nofail 0 0
   tmpfs /tmp tmpfs mode=1777,nosuid,nodev 0 0
   

10. Edit /etc/default/grub. On debian you don’t need to edit GRUB_CMDLINE_LINUX.

    GRUB_DISABLE_LINUX_UUID=true
    GRUB_ENABLE_LINUX_PARTLABEL=true
    

11. Run grub-install. This will install the bootloader to efi. I forget the options to run it with… sorry!

12. Run update-grub (with no options). This will update /boot/grub.cfg so it knows how to find your new drive. You can verify the file by hand if you know how.
13. Run update-initramfs (with no options). This will update the initramfs so it can decrypt your root drive.
14. If there were any warnings or errors printed in the last three steps, something is wrong. Figure out what–it won’t boot otherwise. Especially make sure your /etc/fstab and /etc/crypttab exactly match what you’ve already used to mount filesystems.
15. Exit the chroot. Make sure any changes are synced to disk (you can unmount everything under /mnt in reverse order to make sure if you want)
16. Shut down your computer. Remove your root disk and boot from the new one. It should work now, asking for your password during boot.
17. Once you boot successfully and verify everything mounted, you can remove the nofail from /etc/fstab if you want.
18. (In my case, I also set up the swap partition after successful boot.) Edit: Oh, also don’t use unencrypted swap with encrypted root. That was dumb.

I had previously hand-rolled a status monitor, status.za3k.com, which I am in the process of replacing (new version). I am replacing it with a linux monitoring daemon, mon, which I recommend. It is targeted at working system administrators. ‘mon’ adds many features over my own system, but still has a very bare-bones feeling.

The old service, ‘simple-status‘ worked as follows:

• You visited the URL. Then, the status page would (live) kick of about 30 parallel jobs, to check the status of 30 services
• The list of services is one-per-file in a the services.d directory.
• For each service, it ran a short script, with no command line arguments.
• All output is displayed in a simple html table, with the name of the service, the status (with color coding), and a short output line.
• The script could return with a success (0) or non-success status code. If it returned success, that status line would display in green for success. If it failed, the line would be highlighted red for failure.
• Scripts can be anything, but I wrote several utility functions to be called from scripts. For example, “ping?” checks whether a host is pingable.
• Each script was wrapped in timeout. If the script took too long to run, it would be highlighted yellow.
• The reason all scripts ran every time, is to prevent a failure mode where the information could ever be stale without me noticing.

Mon works as follows

• The list of 30 services is defined in /etc/mon/con.cf.
• For each service, it runs a single-line command (monitor) with arguments. The hostname(s) are added to the command line automatically.
• All output can be displayed in a simple html table, with the name of the service, the status (with color coding), the time of last and next run, and a short output line. Or, I use ‘monshow‘, which is similar but in a text format.
• Monitors can be anything, but several useful ones are provided in /usr/lib/mon/mon.d (on debian). For example the monitor “ping” checks whether a host is pingable.
• The script could return with a success (0) or non-success status code. If it returned success, the status line would display in green for success (on the web interface), or red for failure.
• All scripts run periodically. A script have many states, not just “success” or “failure”. For example “untested” (not yet run) or “dependency failing” (I assume, not yet seen).

As you can see, the two have a very similar approach to the component scripts, which is natural in the Linux world. Here is a comparison.

• ‘simple-status’ does exactly one thing. ‘mon’ has many features, but does the minimum possible to provide each.
• ‘simple-status’ is stateless. ‘mon’ has state.
• ‘simple-status’ runs on demand. ‘mon’ is a daemon which runs monitors periodically.
• Input is different. ‘simple-status’ is one script which takes a timeout. ‘mon’ listens for trap signals and talks to clients who want to know its state.
• both can show an HTML status page that looks about the same, with some CGI parameters accepted.
• ‘mon’ can also show a text status page.
• both run monitors which return success based on status code, and provide extra information as standard output. ‘mon’ scripts are expected to be able to run on a list of hosts, rather than just one.
• ‘mon’ has a config file. ‘simple-status’ has no options.
• ‘simple-status’ is simple (27 lines). ‘mon’ has longer code (4922 lines)
• ‘simple-status’ is written in bash, and does not expose this. ‘mon’ is written in perl, all the monitors are written in perl, and it allows inline perl in the config file
• ‘simple-status’ limits the execution time of monitors. ‘mon’ does not.
• ‘mon’ allows alerting, which call an arbitrary program to deliver the alert (email is common)
• ‘mon’ supports traps, which are active alerts
• ‘mon’ supports watchdog/heartbeat style alerts, where if a trap is not regularly received, it marks a service as failed.
• ‘mon’ supports dependencies
• ‘mon’ allows defining a service for several hosts at once

Overall I think that ‘mon’ is much more complex, but only to add features, and it doesn’t have a lot of features I wouldn’t use. It still is pretty simple with a simple interface. I recommend it as both good, and overall better than my system.

My only complaint is that it’s basically impossible to Google, which is why I’m writing a recommendation for it here.

So you want to know when your monitoring system fails, or your cron jobs don’t run? Add this to your crontab:

MAILTO=me@me.com

Now install a mail-sending agent. I like ‘nullmailer‘, which is much smaller than most mail-sending agents. It can’t receive or forward mail, only send it, which is what I like about it. No chance of a spammer using my server for something nasty.

The way I have it set up, I’ll have a server (avalanche) sending all email from one address (nullmailer@avalanche.za3k.com) to one email (admin@za3k.com), and that’s it. Here’s my setup on debian:

sudo apt-get install nullmailer
echo "admin@za3k.com" | sudo tee /etc/nullmailer/adminaddr # all mail is sent to here, except for certain patterns
echo "nullmailer@`hostname`.za3k.com" | sudo tee /etc/nullmailer/allmailfrom # all mail is sent from here
echo "`hostname`.za3k.com" | sudo tee /etc/nullmailer/defaultdomain # superceded by 'allmailfrom' and not used
echo "`hostname`.za3k.com" | sudo tee /etc/nullmailer/helohost # required to connect to my server. otherwise default to 'me'
echo "smtp.za3k.com smtp --port=587 --starttls" | sudo tee /etc/nullmailer/remotes && sudo chmod 600 /etc/nullmailer/remotes

Now just run echo "Subject: sendmail test" | /usr/lib/sendmail -v admin@za3k.com to test and you’re done!

This expands on my previous post about how to set up an email server.

We’re going to set up a few spam filters in Dovecot under Debian. We’re going to use Sieve, which lets the user set up whichever filters they want. However, we’re going to run a couple pre-baked spam filters regardless of what the user sets up.

1. Install Sieve.

   sudo apt-get install dovecot-sieve dovecot-managesieved
   

2. Add Sieve to Dovecot

   # /etc/dovecot/dovecot.conf
   # Sieve and ManageSieve
   protocols = $protocols sieve
   protocol lmtp {
    mail_plugins = $mail_plugins sieve
   }
   service managesieve-login {
    inet_listener sieve {
    port = 4190
    }
   }
   protocol sieve {
    managesieve_logout_format = bytes ( in=%i : out=%o )
   }
   plugin {
    # Settings for the Sieve and ManageSieve plugin
    sieve = file:~/sieve;active=~/.dovecot.sieve
    sieve_before = /etc/dovecot/sieve.d/
    sieve_dir = ~/sieve # For old version of ManageSieve
    #sieve_extensions = +vnd.dovecot.filter
    #sieve_plugins = sieve_extprograms
   }
   

3. Install and update SpamAssassin, a heuristic perl script for spam filtering.

   sudo apt-get install spamasssassin
   sudo sa-update
   

   # /etc/default/spamassassin
   ENABLED=1
   #CRON=1 # Update automatically
   

   # /etc/spamassassin/local.cf
   report_safe 0 # Don't modify headers
   

   sudo service spamassassin start
   

4. There’s a lot of custom configuration and training you should do to get SpamAssassin to accurately categorize what you consider spam. I’m including a minimal amount here. The following will train SpamAssassin system-wide based on what users sort into spam folders.

   #!/bin/sh
   # /etc/cron.daily/spamassassin-train
   all_folders() {
           find /var/mail/vmail -type d -regextype posix-extended -regex '.*/cur|new$'
   }
   
   all_folders | grep "Spam" | sa-learn --spam -f - >/dev/null 2>/dev/null
   all_folders | grep -v "Spam" | sa-learn --ham -f - >/dev/null 2>/dev/null
   

5. Make Postfix run SpamAssassin as a filter, so that it can add headers as mail comes in.

   # /etc/postfix/master.cf
   smtp inet n - - - - smtpd
    -o content_filter=spamassassin
   # ...
   spamassassin unix - n n - - pipe user=debian-spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f ${sender} ${recipient}
   

   sudo service postfix restart
   

6. Add SpamAssassin to Sieve. Dovecot (via Sieve) will now move messages with spam headers from SpamAssassin to your spam folder. Make sure you have a “Spam” folder and that it’s set to autosubscribe.

   # /etc/dovecot/sieve.d/spam-assassin.sieve
   require ["fileinto"];
   # Move spam to spam folder
   if header :contains "X-Spam-Flag" "YES" {
    fileinto "Spam";
    # Stop here - if there are other rules, ignore them for spam messages
    stop;
   }
   

   cd /etc/dovecot/sieve.d
   sudo sievec spam-assassin.sieve
   

7. Restart Dovecot

   sudo service dovecot restart
   

8. Test spam. The GTUBE is designed to definitely get rejected. Set the content of your email to this:

   XJS*C4JDBQADN1.NSBN3*2IDNEN*GTUBE-STANDARD-ANTI-UBE-TEST-EMAIL*C.34X
   

9. You should also be able to create user-defined filters in Sieve, via the ManageSieve protocol. I tested this using a Sieve thunderbird extension. You’re on your own here.

I’m posting my email setup here. The end result will:

• Use Postfix for SMTP
• Use Dovecot for IMAP and authentication
• Store usernames, email forwards, and passwords in a Postgres SQL database
• Only be accessible over encrypted channels
• Pass all common spam checks
• Support SMTP sending and IMAP email checking. I did not include POP3 because I don’t use it, but it should be easy to add
• NOT add spam filtering or web mail (this article is long enough as it is, maybe in a follow-up)

Note: My set up is pretty standard, except that rDNS for smtp.za3k.com resolves to za3k.com because I only have one IP. You may need to change your hostnames if you’re using mail.example.com or smtp.example.com.

On to the install!

1. Install debian packages

   sudo apt-get install postfix # Postfix \
         dovecot-core dovecot-imapd dovecot-lmtpd # Dovecot \
         postgresql dovecot-pgsql postfix-pgsql # Postgres \
         opendkim opendkim-tools # DKIM
   

2. Set up security. smtp.za3k.com cert is at /etc/certs/zak3.com.pem, the key is at /etc/ssl/private/smtp.za3k.com.key. dhparams for postfix are at /etc/postfix/dhparams.pem. (If you need a certificate and don’t know how to get one, you can read Setting up SSL certificates using StartSSL)

3. Install Postfix

   # /etc/postfix/master.cf
   smtp       inet  n       -       -       -       -       smtpd
   submission inet  n       -       -       -       -       smtpd
     -o syslog_name=postfix/submission
     -o smtpd_tls_security_level=encrypt
     -o smtpd_sasl_auth_enable=yes
     -o smtpd_reject_unlisted_recipient=no
     -o milter_macro_daemon_name=ORIGINATING
   

   # /etc/postfix/main.cf additions
   # TLS parameters
   smtpd_tls_cert_file=/etc/ssl/certs/smtp.za3k.com.pem
   smtpd_tls_key_file=/etc/ssl/private/smtp.za3k.com.key
   smtpd_use_tls=yes
   smtpd_tls_mandatory_protocols=!SSLv2,!SSLv3
   smtp_tls_mandatory_protocols=!SSLv2,!SSLv3
   smtpd_tls_protocols=!SSLv2,!SSLv3
   smtp_tls_protocols=!SSLv2,!SSLv3
   smtpd_tls_exclude_ciphers = aNULL, eNULL, EXPORT, DES, RC4, MD5, PSK, aECDH, EDH-DSS-DES-CBC3-SHA, EDH-RSA-DES-CDC3-SHA, KRB5-DE5, CBC3-SHA
   
   # Relay and recipient settings
   myhostname = za3k.com
   myorigin = /etc/mailname
   mydestination = za3k.com, smtp.za3k.com, localhost.com, localhost
   relayhost =
   mynetworks_style = host
   mailbox_size_limit = 0
   inet_interfaces = all
   smtpd_relay_restrictions = permit_mynetworks,
     permit_sasl_authenticated,
     reject_unauth_destination
   
   alias_maps = hash:/etc/aliases
   local_recipient_maps = $alias_maps
   mailbox_transport = lmtp:unix:private/dovecot-lmtp
   

4. Install Dovecot

   # /etc/dovecot/dovecot.cf
   mail_privileged_group = mail # Local mail
   disable_plaintext_auth = no
   
   protocols = imap
   
   ssl=required
   ssl_cert = </etc/ssl/certs/imap.za3k.com.pem
   ssl_key = </etc/ssl/private/imap.za3k.com.key
   
   # IMAP Folders
   namespace {
    inbox = yes
    mailbox Trash {
    auto = create
    special_use = \Trash
    }
    mailbox Drafts {
    auto = no
    special_use = \Drafts
    }
    mailbox Sent {
    auto = subscribe
    special_use = \Sent
    }
    mailbox Spam {
    auto = subscribe
    special_use = \Junk
    }
   }
   
   # Expunging / deleting mail should FAIL, use the lazy_expunge plugin for this
   namespace {
    prefix = .EXPUNGED/
    hidden = yes
    list = no
    location = maildir:~/expunged
   }
   mail_plugins = $mail_plugins lazy_expunge
   plugin {
    lazy_expunge = .EXPUNGED/
   }
   

   # /etc/postfix/main.cf
   # SASL authentication is done through Dovecot to let users relay mail
   smtpd_sasl_type = dovecot
   smtpd_sasl_path = private/auth
   

5. Set up the database and virtual users. Commands

   # Create the user vmail for storing virtual mail
   # vmail:x:5000:5000::/var/mail/vmail:/usr/bin/nologin
   groupadd -g 5000 vmail
   mkdir /var/mail/vmail
   useradd -M -d /var/mail/vmail --shell=/usr/bin/nologin -u 5000 -g vmail vmail
   chown vmail:vmail /var/mail/vmail
   chmod 700 /var/mail/vmail
   

   psql -U postgres
   ; Set up the users
   CREATE USER 'postfix' PASSWORD 'XXX';
   CREATE USER 'dovecot' PASSWORD 'XXX';
   
   ; Create the database
   CREATE DATABASE email;
   \connect email
   
   ; Set up the schema 
   
   CREATE TABLE aliases (
       alias text NOT NULL,
       email text NOT NULL
   );
   
   CREATE TABLE users (
       username text NOT NULL,
       domain text NOT NULL,
       created timestamp with time zone DEFAULT now(),
       password text NOT NULL
   );
   
   REVOKE ALL ON TABLE aliases FROM PUBLIC;
   GRANT ALL ON TABLE aliases TO postfix;
   GRANT ALL ON TABLE aliases TO dovecot;
   
   REVOKE ALL ON TABLE users FROM PUBLIC;
   GRANT ALL ON TABLE users TO dovecot;
   GRANT ALL ON TABLE users TO postfix;
   

   # /etc/dovecot/dovecot.conf
   # Since we're giving each virtual user their own directory under /var/mail/vmail, just use that directly and not a subdirectory
   mail_location = maildir:~/
   
   # /etc/dovecot/dovecot-sql.conf defines the DB queries used for authorization
   passdb {
     driver = sql
     args = /etc/dovecot/dovecot-sql.conf
   }
   userdb {
     driver = prefetch
   }
   userdb {
     driver = sql
     args = /etc/dovecot/dovecot-sql.conf
   }
   

   # /etc/postfix/main.cf
   pgsql:/etc/postfix/pgsql-virtual-aliases.cf
   local_recipient_maps = pgsql:/etc/postfix/pgsql-virtual-mailbox.cf 
   

   # /etc/postfix/pgsql-virtual-aliases.cf
   # hosts = localhost
   user = postfix
   password = XXXXXX
   dbname = email
   
   query = SELECT email FROM aliases WHERE alias='%s'
   

   # /etc/postfix/pgsql-virtual-mailbox.cf
   # hosts = localhost
   user = postfix
   password = XXXXXX
   dbname = email
   
   query = SELECT concat(username,'@',domain,'/') as email FROM users WHERE username='%s'
   

   # /etc/dovecot/dovecot-sql.conf
   driver = pgsql
   connect = host=localhost dbname=email user=dovecot password=XXXXXX
   default_pass_scheme = SHA512
   password_query = SELECT \
     CONCAT(username,'@',domain) as user, \
     password, \
     'vmail' AS userdb_uid, \
     'vmail' AS userdb_gid, \
     '/var/mail/vmail/%u' as userdb_home \
     FROM users \
     WHERE concat(username,'@',domain) = '%u';
   
   user_query = SELECT username, \
     CONCAT('maildir:/var/mail/vmail/',username,'@',domain) as mail, \
     '/var/mail/vmail/%u' as home, \
     'vmail' as uid, \
     'vmail' as gid \
     FROM users \
     WHERE concat(username,'@',domain) = '%u';
   

6. Set up users. Example user creation:

   # Generate a password
   $ doveadm pw -s sha512 -r 100
   Enter new password: ...
   Retype new password: ...
   {SHA512}.............................................................==
   

   psql -U dovecot -d email
   ; Create a user za3k@za3k.com
   mail=# INSERT INTO users (
       username,
       domain,
       password
   ) VALUES (
       'za3k',
       'za3k.com'
       '{SHA512}.............................................................==',
   );
   

7. Set up aliases/redirects. Example redirect creation:

   psql -U dovecot -d email
   ; Redirect mail from foo@example.com to bar@example.net
   mail=# INSERT INTO users ( email, alias ) VALUES (
       'bar@example.net',
       'foo@example.com'
   );
   

8. Test setup locally by hand. Try using TELNET. Test remote setup using STARTSSL. This is similar to the previous step, but to start the connection use:

   openssl s_client -connect smtp.za3k.com:587 -starttls smtp
   

   Make sure to test email to addresses at your domain or that you’ve set up (final destination), and emails you’re trying to send somewhere else (relay email)

   A small digression: port 25 is used for unencrypted email and support STARTTLS, 587 is used for STARTTLS only, and 465 (obsolete) is used for TLS. My ISP, Comcast, blocks access to port 25 on outgoing traffic.

9. Make sure you’re not running an open relay at http://mxtoolbox.com/diagnostic.aspx

10. Set your DNS so that the MX record points at your new mailserver. You’ll probably want a store and forward backup mail server (mine is provided by my registrar). Email should arrive at your mail server from now on. This is the absolute minimum setup. Everything from here on is to help the community combat spam (and you not to get blacklisted).

11. Set up DKIM (DomainKeys Identified Mail). DKIM signs outgoing mail to show that it’s from your server, which helps you not get flagged as spam.
    None of these files or folders exist to begin with in debian.

    # Add to /etc/opendkim.conf
    KeyTable                /etc/opendkim/KeyTable
    SigningTable            /etc/opendkim/SigningTable
    ExternalIgnoreList      /etc/opendkim/TrustedHosts
    InternalHosts           /etc/opendkim/TrustedHosts
    LogWhy yes
    

    # /etc/opendkim/TrustedHosts
    127.0.0.1
    [::1]
    localhost
    za3k.com
    smtp.za3k.com
    

    mkdir -p /etc/opendkim/keys/za3k.com
    cd /etc/opendkim/keys/za3k.com
    opendkim-genkey -s default -d za3k.com
    chown opendkim:opendkim default.private
    

    # /etc/opendkim/KeyTable
    default._domainkey.za3k.com za3k.com:default:/etc/opendkim/keys/za3k.com/default.private
    

    # /etc/opendkim/SigningTable
    za3k.com default._domainkey.za3k.com
    

    Display the DNS public key to set in a TXT record with:

    # sudo cat /etc/opendkim/keys/za3k.com/default.txt
    default._domainkey      IN      TXT     ( "v=DKIM1; k=rsa; "
              "p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCggdv3OtQMek/fnu+hRrHYZTUcpUFcSGL/+Sbq+GffR98RCgabx/jjPJo3HmqsB8czaXf7yjO2UiSN/a8Ae6/yu23d7hyTPUDacatEM+2Xc4/zG+eAlAMQOLRJeo3z53sNiq0SmJET6R6yH4HCv9VkuS0TQczkvME5hApft+ZedwIDAQAB" )  ; ----- DKIM
    
    # My registrar doesn't support this syntax so it ends up looking like: 
    $ dig txt default._domainkey.za3k.com txt
    default._domainkey.za3k.com. 10800 IN   TXT     "v=DKIM1\; k=rsa\; p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCggdv3OtQMek/fnu+hRrHYZTUcpUFcSGL/+Sbq+GffR98RCgabx/jjPJo3HmqsB8czaXf7yjO2UiSN/a8Ae6/yu23d7hyTPUDacatEM+2Xc4/zG+eAlAMQOLRJeo3z53sNiq0SmJET6R6yH4HCv9VkuS0TQczkvME5hApft+ZedwIDAQAB"
    

    # Uncomment in /etc/default/opendkim
    SOCKET="inet:12345@localhost" # listen on loopback on port 12345
    

    # /etc/postfix/main.cf
    # DKIM
    milter_default_action = accept
    milter_protocol = 6
    smtpd_milters = inet:localhost:12345
    non_smtpd_milters = inet:localhost:12345
    

12. Set up SPF (Sender Policy Framework). SPF explains to other services which IPs can send email on your behalf. You can set up whatever policy you like. A guide to the syntax is at: http://www.openspf.org/SPF_Record_Syntax.  Mine is

    @ 10800 IN TXT "v=spf1 +a:za3k.com +mx:za3k.com ~all"
    

    You should also be verifying this on your end as part of combating spam, but as far as outgoing mail all you need to do is add a TXT record to your DNS record.

13. Set your rDNS (reverse DNS) if it’s not already. This should point at the same hostname reported by Postfix during SMTP. This will be handled by whoever assigns your IP address (in my case, my hosting provider).

14. Test your spam reputability using https://www.mail-tester.com or https://www.port25.com/support/authentication-center/email-verification. You can monitor if you’re on any blacklists at http://mxtoolbox.com/blacklists.aspx.

15. Set up DMARC. DMARC declares your policy around DKIM being mandatory. You can set up whatever policy you like.  Mine is

    _dmarc 10800 IN TXT "v=DMARC1;p=reject;aspf=s;adkim=s;pct=100;rua=mailto:postmaster@za3k.com"
    

My sources writing this:

• https://wiki.archlinux.org/index.php/Postfix and http://www.postfix.org/postconf.5.html on Postfix
• https://wiki.archlinux.org/index.php/Dovecot on Dovecot
• https://help.ubuntu.com/community/PostfixVirtualMailBoxClamSmtpHowto on SASL auth integration between Dovecot and Postfix
• https://www.digitalocean.com/community/tutorials/how-to-set-up-a-postfix-email-server-with-dovecot-dynamic-maildirs-and-lmtp and https://www.digitalocean.com/community/tutorials/how-to-configure-a-mail-server-using-postfix-dovecot-mysql-and-spamassassin on SQL integration in Dovecot
• https://scaron.info/blog/debian-mail-spf-dkim.html for setting up SPF, DKIM, and rDNS

Takeaways

• You can set up store-and-forward mail servers, so if your mail server goes down, you don’t lose all the email for that period. It’s generally a free thing.
• Postfix’s configuration files were badly designed and crufty, so you might pick a different SMTP server.
• Email was REALLY not designed to do authentication, which is why proving you’re not a spammer is so difficult. This would all be trivial with decent crypto baked in (or really, almost any backwards-incompatible change)
• The option to specify a SQL query as a configuration file option is wonderful. Thanks, Dovecot.
• Overall, although it was a lot of work, I do feel like it was worth it to run my own email server.

So have you ever used a web printer and it was great?

…

Yeah, me neither. It’s probably possible on windows, but try to add more than one OS to the network and it’s horrible. And actually printing is a major pain in Linux anyway. Theoretically ‘lp’ and the like have no problem with remote printers, but I wanted something I understood. So today I’m going to post my setup I use instead.

I have a computer physically connected to the printer. Let’s call it ‘printserver’. On that server there is a folder, /printme, which is constantly monitored by inode. Any file added to that directory is printed.

Suppose I downloaded cutecats.pdf and I want to print it. Then I run:

scp cutecats.pdf printserver:/printme

And voila, the cute cats get printed.

Here’s the setup for the server:

1. Get the printer to work. This is the hard step.
2. Make a directory /printme. Add any missing users, add a new group called ‘print’ and add everyone who needs to print to that, etc.

3. Set up /printme to be a tmpfs with the sticky bit set. (So we don’t fill up the hard drive)

   /etc/fstab
   tmpfs           /printme        tmpfs   rw,nodev,nosuid,noexec,uid=nobody,gid=print,mode=1770,size=1G  0       0
   

4. Install incron and add this to the incrontab (of user ‘print’ or ‘sudo’):

   # incrontab -l
   /printme IN_CLOSE_WRITE,IN_MOVED_TO lp $@/$#
   

   Note that this will preserve files after they’re printed, because my server is low-volume enough I don’t need to care.

Most of the following taken from : http://www.msfn.org/board/topic/151992-install-xp-from-usb-without-extra-tools/, just modified to include syslinux support.

Let me know if there are any omissions; it an XP installer bluescreens on boot for me so I can’t actually test.

1. Obtain an XP iso file
2. Format drive with one FAT parition, marked bootable.

3. syslinux -i /dev/sdXX
   

4. $ cp /usr/lib/syslinux/bios/mbr.bin >/dev/sdX
   

5. $ mount /dev/sdXX /mnt
   

6. mkdir /tmp/xp_iso
   mount xp.iso /tmp/xp_iso
   cp -ar /tmp/xp_iso/* /mnt
   umount /tmp/xp_iso
   rmdir xp_iso
   

7. cp /usr/lib/syslinux/bios/{chain.c32,libutil.c32,menu.c32,libcom.c32} /mnt
   

8. cp /mnt/I386/{NTDETECT.COM,SETUPLDR.BIN,TXTSETUP.SIF} /mnt
   

9. Edit /mnt/syslinux.cfg:

   UI menu.c32# Windows XP
   LABEL windows_xp
   MENU LABEL Run Windows ^XP Setup
   COM32 chain.c32
   APPEND fs ntldr=SETUPLDR.BIN
   

10. umount /mnt
    

11. Boot from the USB stick