A few days ago, I installed the Windows 10 “April update”, and it broke my GRUB installation. What happened?

My primary disk has an MBR partition table. (Apparently, booting from GPT requires using UEFI, which exposes a whole new exciting set of firmware bugs.) GRUB was installed in a small ext2 partition (primary partition #3), while primary partitions #1 and #2 were used by Windows 10. Installing the April update created another primary partition and moved my ext2 partition to slot #4 so grub can no longer find its files.

Rescuing with grub

When this happens, you are greeted with a nice “rescue shell” mode.

grub rescue>

Now what? Unfortunately, contrary to GRUB 1, GRUB 2 cannot do much without loading additional modules. You cannot even chainload Windows from a PBR because the chainloader command needs the chainload module to be loaded.

You can list partitions and directories with the ls command. The set command shows the variables that have been set; there is a variable named prefix that contains the path to the grub files. So after finding out that /boot/grub is now on (hd0,msdos4) instead of (hd0,msdos3) as before, you can re-set the prefix, load the normal module and execute the normal command to do the, well, normal boot:

set prefix=(hd0,msdos4)/boot/grub
insmod normal
normal

Unfortunately, there is no way (that I could find) to make the new prefix permanent, other than running grub-install from a running system. That is, you would be doing this dance on every boot from now on. Also unfortunately,

• the version of grub2 in pkgsrc is kind of old and
• the grub2 package has not built, as far as I can tell, for anyone in the last two years.

As a result, I would need to boot a grml.org rescue system and reinstall grub from there.

Working around

There is an easier way to make GRUB find its files again. We just need to make sure the partition table matches what it expects.

A partition table is like an array of pointers (start and length). So you can delete two partitions and recreate them with exactly the same (start, length) tuples but different partition numbers, and things will just work. So I did exactly that, using fdisk -u ld0 on NetBSD. It helps to do this on X, where you can open two terminal windows side by side: one shows the previous partition table, one the interactive fdisk session, and you can copy values around with the mouse. tmux (in the NetBSD base system) on a text screen would also work.

So in my case, I swapped partitions #3 and #4, and GRUB worked again!

And one day later, Windows decided to apply the May update to the April update -.-

Addendum: What new partition!?

Honestly, I am not sure what the new partition is actually doing. Even though I want a single Windows partition, there are now three of them:

1. an NTFS partition of about 100 MB,
2. my actual C drive,
3. A partition of type 0x27 of about 450 MB. Googling shows that this might be MirOS BSD (which I doubt) or a hidden recovery partition of some sort.

I am not sure what is going on here. I would appreciate hints from readers as to why these partitions are there.

My new year’s resolution for 2018 has been to blog more. So I decided to create an actual blog!

It started with me closing my Amazon AWS account and writing about it. The posting was up as a Gist on Github, and I shared that URL. This does give a useful viewer and the ability to add comments, but it is hardly discoverable. Anyway, the post was somewhat widely circulated (the provocative title certainly helped) and even made it to Reddit.

After deciding to create a real blog, I started looking into how to do that:

Now I finally have something up and running. The address is simple enough, and based on my Twitter handle:

https://bentsukun.ch/

In true Open Source tradition, the source code for the site is visible online at https://github.com/bsiegert/blog.

For now, I am quite happy with this setup.

The setup

• Hugo with the purehugo theme,
• Firebase Hosting,
• domain at HostTech.

The site itself is created with the excellent Hugo, which has the added advantage of being written in Go :) The installation is as easy as

go get github.com/gohugoio/hugo

or by using the www/hugo package in pkgsrc. Hugo ends up creating a fully static web site, so simple static hosting is enough.

Google Cloud Storage supports hosting a static web site – that’s great, right? Well, not quite. There is no support for HTTPS in plain GCS. If you want HTTPS, apparently you need:

1. The GCS bucket that holds the files,
2. A Cloud Load Balancer instance in front
3. Cloud CDN for delivering content from the edge.

Or, you can use Firebase Hosting, also by Google. This is what I am doing. A simple firebase deploy uploads all the static files, and an SSL certificate is included in the offer.

I tried using Google Domains to register the domain but that service is not available in Switzerland, unfortunately. hosttech.ch to the rescue! It took me a while to figure out that (a) a DNS zone for the domain is included and (b) how to add entries to it.

To get Firebase Hosting to serve directly on the domain, it is necessary to add a google-site-verification TXT record into the DNS for it. And that’s it.

Today, I deleted my Amazon AWS account.

I had been on AWS since about 2011. My usage was mainly for two things:

1. Saving large amounts of files (build logs and such) on S3;
2. Running NetBSD VMs on EC2.

EC2 is based on Xen, and NetBSD runs really well in PV (paravirtualized) mode on Xen. However, XSA-240 means that a malicious PV guest may crash (or even otherwise exploit) the hypervisor, with the recommended fix being to not run untrusted PV guests. Over night, Amazon disabled PV, making NetBSD VMs useless.

In general, EC2 has been moving away from Xen. The newer instance types already no longer supported PV; there are two higher-performance paravirtualized modes (PVH and PVHVM) that are preferred these days, and that NetBSD does not support. The newest machine types use a custom hypervisor based on KVM.

The way the PV change was rolled out highlighted another long-standing EC2 problem: instances would continue running until the server they ran on got rebooted, at which point they were migrated to a random machine. If the target machine had PV disabled, the VM simply did not come up again. I have had the same type of issue in the past, where your VM randomly landed on a “good” or a “bad” machine and did not come up on the bad one. There is no way (AFAIK) to constrain to a certain subset of servers, e.g. running a certain hypervisor version.

Also, of course, there was no warning or announcement, just that VMs stopped working all of a sudden. A bunch of people were completely caught by surprise when their service became unavailable. I hope you have monitoring!?

The Alternative

Which brings me to where I did take my workloads: Google Cloud Platform.

(This has nothing to do whatsoever with who my employer is. I pay for my GCP usage with my own money.)

These days, NetBSD (8+) runs great on Google Compute Engine. There is a script (that I created) to stage instances at https://github.com/google/netbsd-gce, though there are no official NetBSD images around. My S3 usage works equally well using Google Cloud Storage. And I have always been a fan of App Engine, particularly because of its great Go support. https://bulktracker.appspot.com/ runs on App Engine.

Conclusion

My general impression is: Features are roughly on par, prices on GCP are a bit cheaper, and the Google Cloud SDK and command-line tools are better. So rather than let old, unusable VM images continue to rot and pay Amazon 2$ a month for that bit of storage, I let go of that AWS account. Bye, Amazon.

So now I am even posting over at TNF on http://blog.NetBSD.org/. Julian Fagir made new NetBSD flyers, and I committed them to the TNF website.

I know that I should write more here but there is not much new on the MirBSD front.

I updated the showcase to NetBSD-6_BETA on the Dom0, and now X refuses to start. Oh well. X does start when using a GENERIC kernel. This is very bad for showcase use, of course :(. pkgsrc is going into freeze very soon, and I did not do a whole lot of MirBSD fixes this time around. This is due to illness, searching for a new job, and working on the Go programming language, which is expected to hit version 1.0 Real Soon Now(TM).

I brushed up my Algorithms and Data Structures a bit by reading the third volume of TAOCP. Fantastic book.