A while ago, I installed Fedora Asahi Remix on my M2 MacBook Air, and I was very positive about it. So positive, in fact, that I ended up making it the default partition in the bootloader. I haven’t used macOS in weeks. But then, a few days ago, something weird happened:

In the middle of some development work, running cvs update on the pkgsrc repository, the screen suddenly filled with a bunch of “read-only file system” errors. It turned out that both / and /home had remounted themselves read-only, without my intervention.

Running journalctl --system showed some warning messages in an alarming orange-red: btrfs, the file system that the system was running from, had found inconsistencies and had chosen to remount itself read-only. Now what?

I tried running btrfsck /dev/nvme0n1p6. Now, by default, btrfsck will only check, not repair. But even the check aborted with an error about “errors found in FS roots”.

From my BSD experience, I thought booting into single-user mode and running fsck with a r/o root file system would do the trick. A quick init S later … it turns out that you can only get a shell in single-user mode if you have a root password set. By default, logins as root are forbidden, even in single-user mode. But good luck setting a password while / is read-only!

After several reboots, I got the system to work r/w for a while, so I could set a root password. So within single-user mode, I ran

# btrfsck --repair --force /dev/nvme0n1p6

Decent idea, but like the check before, the repair option also aborted with an error, saying “parent transid verify failed”. According to some random search results, this means that the file system is likely damaged beyond repair. Or, as @ParadeGrotesque aptly remarked on Mastodon:

It’s dead, Jim.

Don’t blame me, I don’t make the rules.

So in the end, I managed to back up my home directory with Restic (good thing I had restic installed before!), then wiped the entire installation and re-installed. Bummer.

More on btrfs

Why is btrfs the default file system in the first place? The Fedora Asahi Remix installer doesn’t even let you choose the file system type! I wish it did!

The way I understand btrfs is that the developers are aiming for a feature set on par with ZFS, except fully under the GPL and more “Linux-native” – ZFS was originally developed for Solaris, after all, and its licensing situation is at least a bit dodgy.

btrfs has long had a reputation of playing fast and loose with your data. In the early days, I heard several stories of spectacularly losing data. People have assured me that those days are over, and that btrfs is a reliable system these days.

Honestly, this experience has shown me that btrfs, even today, is anything but reliable. The corruption I witnessed is surely not due to broken hardware. The hardware is absolutely fine.

I suspect that the root cause is probably several unclean shutdowns. Asahi Linux does have an unfortunate bug where it will consume too much battery when in standby, so I ran out of battery several times. And, to be fair, finding the corruption and going read-only is sensible. But what I find absolutely infuriating is the utter inability of its fsck to actually repair anything. The fs roots are damaged, so what? There are incorrect transids, so what? Telling me my FS is fucked and I need to recreate it from scratch is not useful!

As it is, what happened with btrfs made me recall the previous low point of Linux file systems: years ago, I ran a file server with ReiserFS in production. It corrupted itself to such a degree that reiserfsck kept segfaulting. We lost so much user data. Good times.

I installed Fedora 39 the other day. (More on that in one of the next posts.) It has a nifty software installer thing named “Discover”. When I typed “Visual Studio Code” into the search box, it dutifully installed VS Code. As a Flatpak.

This was the first time I interacted with Flatpak, and it did not go well.

Aside: Why VS Code

I want to use VS Code for editing Go code, with gopls, since it provides a really good integration. It turns out that a majority of Go developers use VS Code, so the language server integration is well tested and complete. In short, it’s the well-lit path.

Specifically for myself, there is a second reason: At work, we use a bespoke web-based IDE based on VS Code, with all sorts of integrations to make developers more productive. I have gotten very familiar with this setup, so it makes sense for me to write open source code in this way too and benefit from the muscle memory, so to speak.

There is a philosophical argument that you should avoid VS Code because it is controlled by Microsoft, and it gives Microsoft a certain leverage over the open source Go ecosystem. For what it’s worth, the same argument applies to using GitHub: Such a large percentage of open source code is developed on GitHub these days, and Microsoft could “enshittify” it at any moment if they wanted to. But for both of these, I personally think that it would be easily doable to switch away from them to something free – for instance, move over to Neovim’s LSP integration. In the meantime, I remain pragmatic and use what works for me.

Why not Flatpak

Back to Fedora. I installed Go and gopls from pkgsrc, as you do. However, the Go plugin tells me that it cannot find gopls, or any of the toolchain. Why!? Issue golang/vscode-go#263 has a bunch of people rather confused about this failure mode.

Ultimately, it comes down to this: Flatpaks run isolated from the host OS, which is fundamentally incompatible with developing code in an IDE. Despite the allowlist for access to OS paths, there is no way to allow access to /usr, /lib, etc. This is fundamental to the Flatpak security model. The app runs in a container that is cordoned off from the main OS.

But an IDE for developing native code does need host OS access! It needs to run a build tool, toolchain, native tools, etc. Not only does the VS Code Flatpak not allow doing this, it also does not provide an easy way to install a toolchain into the container. Never mind that I do not want a second copy of my tools. An IDE also needs to be able to run a shell in a terminal.

At this point, I am left wondering: who can actually use this Flatpak, if native compilation is all but impossible? Is it made for frontend devs writing only JS, with all the tooling in JS as well?

Fixes

I am also wondering why the Flatpak is the default type of installation for VS Code on Fedora, given that Upstream has a repository of perfectly fine RPM packages. See https://code.visualstudio.com/docs/setup/linux#_rhel-fedora-and-centos-based-distributions.

The other container-like option would be a Snap, if you are into additional packaging systems on your distro. You can install the Snap in “classic” mode, where it has file system access:

sudo snap install --classic code

My choice has been the native package, and now everything is running fine.