Sep, Oct, Nov 2014: Vietnam.

A year ago, I left my job at Streak and moved to Vietnam. I felt like I needed change. Vietnam ended up being wonderful; I was really glad I travelled with my friends Richard and Kathy, which ended up making the experience a hundred times better than it would have been otherwise. The basic environment was: everything is cheap, I newly have endless free time, I was automatically prompted by my friends in the evenings and sometimes during the day to go on small novel adventures involving physical activity, and I had little internet access. This is probably my perfect environment, and I was functioning very well (the vietnamese diet also has small, well-balanced meals which might have helped). For some reason, I was also able to intensely single-task. [I’d like to write more about what Vietnam is like, but this article is quite long enough as it is]

While I was in Vietnam, I made a to-do list. The to-do list had all the burning projects I actually wanted to do. I’ve ended up accomplishing most of them, at a rate of one every week or two, and it’s a decent summary of what I’ve been doing since. Two things made the to-do list a success. First, it had BIG tasks. These are projects like my recent “set up an IRC server” or “start a publishing company”. Because of that, I don’t get bogged down in minutae, and the tasks are always motivating. I find I function better when I try to carefully plan around having any logistics. The second reason, which I realized today, is that I was very careful to only include tasks I was planning to do (subtly different than tasks I wanted to do). The list was descriptive, not normative, although it certainly included some things like doing taxes I wasn’t wild about.

Looking at my journal and it really only starts up again in March, so I’m going to organize this post in terms of the to-do list. There are a couple items that don’t fit:

• I started dating my wonderful pet, Lealend, while I was in Vietnam. I went to visit them for a month in Puerto Rico where they live. This is very very important to me (the most important thing that happened in the last year), but I don’t usually write about things that personal on my blog so I’m not doing to say much. I’ve been emotionally maturing a lot by being with them.
• Conventions. I went to DEF CON, which was probably the best single week this year so far. I’m definitely going again next year. I attended a mirix [paper] in the South Bay, which ended up being stressful for transportation-related reasons but really good while I was there. I’m planning on going to Burning Man this year as well.
• I started contracting, that’s how I’ve been alive for a year. I’ve been doing some work for Zinc and Paul Christiano on a workflowy clone, mostly. I work two hours a day average.
• I’ve been developing a minecraft modpack [I’ll write more about this when it’s stable], and recently taken an interest in livestreaming.

Now on to the to-do list.

• Project: Printserver
  Success: Success but obsolete
  Description: I set up a printserver. It’s a little raspberry pi that talks to my printer, because getting printers set up is a pain and I don’t want to do it all the time. It went great, it saved me a ton of hassle to have it automatically print out my daily agenda every morning, and to just be able to transfer documents over with ‘scp’.
  Future plans: Unfortunately, my printer died and we only recently got a new one. I need to set it up with the new server. I could also make printing completely automatic when new files show up with scp (right now it’s manual so I can switch out paper, but my roommates would be happier with scp I think).

• Project: Set up my phone so dropping/losing it isn’t horrible
  Success: Partial success
  Description: I wanted to root and then automatically backup my phone. I did figure out how to do as much backing up as I can, and it is automatic. Unfortunately it turns out most of the filesystem (including SMS) just isn’t available over Media Transfer Protocol which android uses to display files, so I had to special case the things I desperately needed backed up. I’d prefer the state of the world let me back up everything on the phone, but that’s as much work as I’m willing to do.

• Project: Get digital copies of all books I own
  Success: Success
  Description: I got digital copies of all books I own via a combination of pirating, buying copies, and getting the books scanned by a service. I did not get rid of the physical books.

• Project: Switch to private email
  Success: Not done
  Description: I get a little nervous entrusting Google (or any third party) with the ability to read, lose, or add restrictions on what I can do with my email. I want to set up my own email address (za3k@za3k.com) and have it be my main point of contact. My email does work, but I can’t send outgoing email, and I haven’t switched everything over to it for that reason.

• Project: Download ArXiV
  Success: Done
  Description: As an archive nut, I worry that the ArXiV collection, one of the nicer collections of scientific papers I access regularly, might someday go down or get censored. I downloaded a copy and stashed it away somewhere. Unfortunately ArXiV’s licenses they get papers under doesn’t permit redistribution, so I can’t publicly host it. (This was really cool but I had to decide whether I was going to publicly mention, since it’s a legal gray area)
  Future plans: Someone (not me) should host a torrent. Contact me and I can get you a copy.

• Project: Pack and unpack storage bins (trip to vietnam)
  Success: Success
  Description: Okay I know this sounds stupid, but I spent about a month packing up to go to Vietnam, and all my physical stuff has stayed organized ever since. That’s a really big change for me.

• Project: Host an IRC server
  Success: Success

• Project: Make hibernate work on my laptop
  Success: Success
  Description: This involved switching partitioning around since btrfs doesn’t support swap files. If I recall, my setup is now a swap partition and a root btrfs partition, inside LVM, inside LUKS.

• Project: Extract bitcoins
  Success: Success
  Description: Extract bitcoins from all my computers and centralize them in one place

• Project: iPhone
  Success: Success
  Description: Back up all my personal data from my iPhone, clear the contents, and sell it.

• Project: N-grams
  Success: Obsolete
  Description: The Google N-grams dataset from their book scanning project is freely available, but in a terrible format (split across set-size file chunks, but in random rather than sorted order). My plan was to convert the formatting and offer it as a torrent / s3 bucket. Google has corrected the problem in a revised version of the dataset.

• Project: NNTP over tor
  Success: Didn’t do
  Description: I run a private newsserver, and I wanted to let people access the newsserver (and anything else on that physical server) over tor. I decided the newsserver was too dead to bother with, and I didn’t feel enthusiastic about setting up tor, so I dropped the project.
  Future plans: I don’t care about the original project, but if there’s a compelling stimulus, I want to set up tor for my server to learn how and leave flexibility.

• Project: Textmode backup
  Success: Success
  Description: ‘textmode’ is the name of a virtual machine on my OS X machine. The project was to back up contents of the machine once, and then delete the virtual machine

• Project: Post pdfmailer website
  Success: Success
  Description: I wanted people to be able to get a physical copy of a pdf document they had mailed to them. I think this project was an especial success, because I’d been failing at an over-engineered version of this off and on for a year. I decided to have the website email me instead of trying to do everything automatically, and ended up getting the books to be a factor of 10 cheaper or so by going with a publisher with no API.
  Future plans: I’d like to popularize the website more. I think there are also some small technical improvements to be made. I’m not going to automate things unless it starts using up a lot of my time to process requests myself.

• [Censored project involving an arbitrage opportunity I haven’t cornered]

• Project: Back up email
  Success: Success

• Project: Flatten backups
  Success: Good enough
  Description: Oh just go read the XKCD. Now imagine you’ve been archiving computers onto other computers for 15 years, and buy cheap laptops.

• Project: QR codes for ebooks
  Success: Success

• Project: Business cards
  Success: Not done
  Description: Make some personal business cards

• Project: QR Punchcodes
  Success: Didn’t do
  Description: So you know how QR codes can contain any data? That means you could show them to a camera and the camera could run any code. Like, code to wait for another couple of QR codes, or to print out some more QR codes…

• [Censored project involving an arbitrage opportunity I haven’t cornered]

• [Censored project involving a mildly illegal thing]

• Project: Make a desk out of cardboard
  Success: Ongoing
  Description: I want to make a desk out of cardboard, because it sounds fun. I’m proud of doing the design right here. I’ve finished mocking it out of cardstock, and actually noticed a lot of flaws and fixed the design instead of hoping them away. Now I mostly have to get the cardboard and make it, should be fun.

• Project: Make a whiteboard partition
  Success: Success

• Project: Write about paper backups
  Success: Success

• Project: Sort physical scans
  Success: Success
  Description: As part of packing up all my possessions to go to Vietnam, I scanned every physical document I own (and mostly threw them out). Twenty years of stuff is a lot of stuff, but I eventually sorted it all out. I’ve been increasingly finding that a flat folder structure ends up working out best for me in the long term, so that’s what I used.

• Project: Two-location backup
  Success: Not done
  Description: My backup server uses RAID-1, but I’d like to have a second copy on an external hard drive somewhere.  Also, I have a bunch of external hard drives which are currently not backed up anywhere (mostly with stuff like movies) which I’d like to have some kind of redundancy

• Project: Treemap finances
  Success: Not done
  Description: I’ve made my finances public, but my analysis tools aren’t great. I’d like to update some old work I’ve done and add a web interface to see where I spent money during a particular time span, using a treemap display.

• Project: Archive Github (aka download all the code in the world)
  Success: Not done, on hold
  Description: I’m kind of burnt out on archiving tasks lately, so this doesn’t sound fun to me. I decided to work with archive team on this one. It’ll get done if it sounds low-stress and no one else seems to be doing it, but it’s less likely than the older archiving projects, despite being important for the world.

• Project: Encrypt backup
  Success: Not done
  Description: I’d like a way to back up my data to untrusted media, like tarsnap does, especially a way that avoids leaking file metadata (like access times and file lengths). Failing that, I should at least encrypt the drive backups are to so I can turn off that computer if needed.

• Project: Gwernify
  Success: Not done
  Description: Gwern writes about how to protect links against link rot. He does this for all links on his website. I ambitiously plan to automatically save a copy of every site I visit (not just the actual URL I visit ideally, but the whole page).

• irc.za3k.com
• Port 6667

Or via the webchat, which I recommend.

I’ve also recently updated my home page to look much prettier, in imitation of a Computer Craft cheatsheet I’ve been working on.

I consider Dead Tree Publishing to be Good Enough at this point. It’s launched.

I’m going to add support for URLs instead of uploading PDFs, and fix some bugs here for there, but it’s essentially done.

Meanwhile, I’ve already received my first physical book I’m publishing through the site. I got this nice email from Eric Eve:

Dear Zachary,

Thank you for your interest in my work. Yes, do please feel free to offer print copies of the TADS 3 Tour Guide (or any other of my TADS 3 documentation) along the lines you suggest. The only copyright right I’m interested in enforcing is my right to be identified as the author of the work, and I'm sure that's not as issue here.

Best wishes,

Eric

https://www.rpgsolo.com/ has a table for resolving yes/no questions, in turn taken from FU RPG. Roll a die:

• Yes, and…
• Yes…
• Yes, but…
• No, but…
• No…
• No, and…

Their example:

I listen at the door. Do I hear anything? (I determine the odds are 50/50 so I click the “50/50” button in the Get Answer section and I get back the following.)

Yes, and…

(Now it’s up to me to determine what that means. Since it says “and” that means I got some kind of bonus. So I am going to interpret that to mean that from the sounds I am hearing I have received some extra information. So I type or say to myself),

I hear one person in the room. (Now I ask my next question.) Is the door locked?

No, but…

(The answer is no but it’s not a total loss. I interpret what that means then type the following),

The door seems weak enough that I can probably kick it open.

So we’ve gotten a base system for telling stories. We then added the following:

1. Whenever you roll a result, roll TWICE. In one universe, you get one result. In the other universe, you get another result. (With accompanying description). If you have more than three universes lying around, discard down to three. All actions/questions are for a particular universe declared by the players.
2. (addendum) Actually, roll two dice: only split the universe if the second die comes up “1” or “2”. Otherwise, answer the question normally. This speeds things up a bit.

It was pretty fun in practice. I recommend using a text file over paper, since you’re going to do a lot of copy-paste. We had more fun with no GM than with a GM. No firm result yet on sandbox-worldbuilding vs players in scenarios; both seemed all right.

I wanted a partition to divide my room, and I had a whiteboard sitting around. I sawed it into three parts, and connected them with hinges:

I’m a little embarrassed at having done all this, since it was obvious as soon as I started the partition was way too short to work. I figured I’d still get some experience woodworking (this is my first project). Here’s where it went:

and I never saw it again

Ingredients:

• 8-10 oz fresh steak, with salt and pepper
• 2 tsp capers and/or 2 Tbsp diced olives
• 3 tsp brown mustard
• 4 tsp olive oil
• 2 eggs
• 2 Tbsp finely diced, fresh red onions
• Crushed red pepper to taste

1. Remove all fat and tendons from the steak. Season it lightly with salt and pepper, and sear lightly on high heat to make it safe to eat. Slice the meat into very thin (2mm) strips, arrange in two piles. Coat the meat in olive oil. Push a small divot into each pile.
2. Dice olives and onions. Add capers, mustard, and red pepper. Mix together and pour into meat piles equally, or surround the meat with it.
3. Separate whites and yolks (carefully removing all the white since we’re using raw yolks). Pour one egg yolk into each divot.

Read about raw beef and egg safety first to be well informed.

Previous work:

• Optar (OPTical ARchive), a 2-D barcode format (github)
• Paperback

I wanted (for fun) to see if I could get data stored in paper formats. I’d read the previous work, and people put a lot of thought into density, but not a lot of thought into ease of retreival. First off, acid-free paper lasts 500 years or so, which is plenty long enough compared to any environmental stresses (moisture, etc) I expect on any paper I have.

Optar gets a density of 200kB / A4 page. By default, it requires a 600dpi printer, and a 600+dpi scanner. It has 3-of-12 bit redundancy using Golay codes, and spaces out the bits in an okay fashion.

Paperback gets a (theoretical) density of 500kB / A4 page. It needs a 600dpi printer, and a ~900dpi scanner.  It has configurable redundancy using Reed-Solomon codes. It looks completely unusable in practice (alignment issues, aside from being Windows-only).

Okay, so I think these are all stupid, because you need some custom software to decode them, which in any case where you’re decoding data stored on paper you probably don’t have that. I want to use standard barcodes, even if they’re going to be lower density. Let’s look at our options. I’m going to skip linear barcodes (low-density) and color barcodes (printing in color is expensive).  Since we need space between symbols, we want to pick the biggest versions of each code we can. For one, whitespace around codes is going to dominate actual code density for layout efficiency, and larger symbols are usually more dense. For another thing, we want to scan as few symbols as possible if we’re doing them one at a time.

Aztec From 15×15 to 151×151 square pixels. 1914 bytes maximum. Configurable Reed-Solomon error correction.

Density: 11.9 pixels per byte

Data Matrix From 10×10 to 144×144 square pixels. 1555 bytes maximum. Large, non-configurable error correction.

Density: 13.3 pixels per byte

QR Code From 21×21 to 177×177 square pixels. 2,953 bytes maximum. Somewhat configurable Reed-Solomon error correction.

Density: 10.6 pixels per byte

PDF417 17 height by 90-583 width.  1100 bytes maximum. Configurable Reed-Solomon error correction. PDF417 is a stacked linear barcode, and can be scanned by much simpler scanners instead of cameras. It also has built in cross-symbol linking (MacroPDF417), meaning you can scan a sequence of codes before getting output–handy for getting software to automatically link all the codes on a page.

Density: 9.01 pixels per byte

QR codes and PDF417 look like our contenders. PDF417 turns out to not scan well (at all, but especially at large symbol sizes), so despite some nice features let’s pick QR codes. Back when I worked on a digital library I made a component to generate QR codes on the fly, and I know how to scan them on my phone and webcam already from that, so it would be pretty easy to use them.

What density can we get on a sheet of A4 paper (8.25 in × 11.00 in, or 7.75in x 10.50in with half-inch margins)? I trust optar’s estimate (600 dpi = 200 pixels per inch) for printed/scanned pages since they seemed to test things. A max-size QR code is 144×144 pixels, or 0.72 x 0.72 inches at maximum density. We can fit 10 x 14 = 140 QR codes with maximum density on the page, less if we want decent spacing. That’s 140 QR codes x (2,953 bytes per QR code) = 413420 bytes = 413K per page before error correction.

That’s totally comparable to the other approaches above, and you can read the results with off-the-shelf software.  Bam.

Compared with last update,  the Dead Tree Publishing website is looking nicer.

It’s served over HTTPS now (not needed for security, but it puts people at ease and enabled Chrome’s autocomplete) and you can order multiple books at a time.

Other than some more visual improvements, the main thing missing is proper detection of page size — my server doesn’t understand about page margins, so it things books are bigger than they really are.

Last post I discussed the publishing website I’m working on.

Today I added credit card processing and address forms–it’s functionally complete and available at https://publishing.za3k.com

Next up I have to clean the site up, because it looks like this:

I’ll also add HTTPS.

I started work on my publishing website again (Dead Tree Publishing). The idea is to make a really, really convenient way to get a physical copy of a PDF/epub book. Think: “send me a printed copy of this mailing list / tumblr”. Right now things are looking encouraging.

I use a “back end” publisher who does all the actual printing, and the one I was using before charged quite a lot and wasn’t amazingly fast; I just used them because they were the only publisher who was at all up to date. Seriously, order of $100 – $200 for a 100 page book, just absolutely ridiculous levels of expensive. I’m switching over to a new publisher who can offer that same book for something like $7 (maybe $12 in hardback), which is absolutely reasonable, and with similar 2-week turnaround times.

First you upload a PDF:

Then I tell you what your ordering options are (hardcover, softcover, color), and what they cost. I’m also supposed to ask you your address to ship the book, and for you to pay for it,  but those aren’t done yet.

Hopefully in the next day or two I’ll have something up and running so people can order books, and then make it gradually nicer! I’m very excited about this website existing.