I came across an interesting ethical conundrum at work recently. The following is like a Bac de Philo, the famous philosophy essay for the French school degree. You have four hours, discuss.

Team A maintains a proprietary domain-specific language (DSL) that is widely used inside the company. It is used both during builds and at runtime. Its slowness and high memory usage shows up in profiling as a bottleneck.

Team B is probably the biggest user of this DSL. A senior engineer on team B has vibe-coded an alternative interpreter for this DSL, from scratch, within a few days. They have not read the code. Because this interpreter’s design is specific to one task (evaluating this DSL), it is 10x faster than the existing interpreter.

The engineer on team B wants to replace the old interpreter to save a ton of time and compute resources. They have run both interpreters on all the files that they know of and verified that they produce identical output. On the other hand, they don’t want to maintain this interpreter forever – that’s the job of team A. They go to team A and propose that they should take on the new codebase and replace their existing codebase with it.

1. As the tech lead of team A, would you accept the offer? Why or why not?
2. As the tech lead of team B (not the implementer), what would you do? Would you participate in the negotiations? Would you put pressure on team A? Would you escalate?
3. As a senior lead (say Vice President), with both teams reporting to you indirectly, would you intervene? If so, how?
4. If your answer to any of the above hinges on the fact that the code was written by AI: would your answers change if it was not? If the engineer had spent the weekend in a retreat and coded the whole thing by hand? What if it was in Haskell?

It has been a bit over a year since I wrote Some nuanced thoughts on AI. Much of what I said then still stands.

On the other hand, the AI models have continued improving. Even more so, the “harness” running it has gone through multiple iterations. Back then, the main mode of interacting with an LLM during coding was through autocomplete. The LLM would guess what you wanted to write and propose some text. That UI paradigm is gone now.

The major game changer has been making it agentic. Yeah, I can hear you groan, dear reader. But the LLM can now run tools: it can build stuff. It can run tests. It can ingest test failures and use that for improvements. Most importantly, it can search documentation and can run CLI tools, e.g. for your framework.

The current UI paradigms that I see are:

• a sidebar in an IDE, typically based on VS Code. This is what Antigravity does. IMHO, this pattern is on the way out, as you need to make fewer edits to LLM generated code, and you can make them by prompting.
• a conversation view that can occasionally branch out to show some diffs, or a design doc. Choose your medium: TUI in the terminal, web app, or chatbot right in your Slack or similar.

The last one is what I am now working with (at my dayjob).

With the ability to run tools comes a big security problem: you don’t want the LLM to take down production because it has decided that your deployment is fucked. You don’t want it to be able to delete a database, or all those other things your account can do. What does not work is stopping every couple of seconds and asking the user to approve a command. You’ll get tired of it and simply end up allowing everything. Or the LLM goes around your guardrails, e.g. by writing a Python script that does a dangerous thing.

At work, we have solved (solved-ish) the issue by giving everyone’s LLM agent its own user ID, clearly marked as not-human. The harness runs in an ephemeral VM. The user and the VM are more limited in their access. Code produced by the LLM has to be reviewed by two humans, typically the owner of the bot and a second person. This is the modern version of the “four eyes” principle that apparently has something to do with compliance with the Sarbanes-Oxley act.

So how has my job changed? If you had told me at the beginning of this year that I would be writing almost no code by hand at work, I would not have believed you.

Anyway, I now write almost no code by hand. I still do quick edits, config changes and the like by hand – but mostly in vim in the terminal. Do I mind? No. As a Staff Engineer, I have always been reviewing 10x more code than I’d been writing. So I don’t mind reviewing code. Reviewing code is important.

I have been involved with packaging and patching software (in pkgsrc) and with large scale changes at work, so only having superficial knowledge of a code base is also fine for me. It helps that my main language is Go, which (a) LLMs are very good at and (b) usually does exactly what the code says.

A positive change that I do see: “less important” coding tasks actually get done! I see quarterly goals that sat at “we won’t have time to do this” get to completion in time, because someone can churn out that code “in the background”, or on a calm Friday afternoon. I see tech debt actually decreasing because you can let the LLM tackle those tech debt reduction tasks.

I am still pretty uncomfortable with the AI maximalists at work that want to upend the entire development lifecycle: agents review code produced by other agents with no human involvement; agents writing tickets for other agents that autonomously design a feature. YOLOing your deployment because you “don’t have time” to do it properly. That way still lies madness.

Do I use LLMs for coding in my spare time? Mostly no. I can ask Gemini to write me a complex SQL query, and it does a much better job than I do. I tried feeding it some hard-to-debug crash, and it came up with a plausible theory why it happens and a patch. But the patch did not solve the issue, which is a bummer.

Also, somehow, my motivation for computing in general has decreased. Should I spend the time learning Rust properly? Does it matter anymore?

I don’t know if people are still genuinely enthusiastic about tech. It feels like tech is not sparking joy any more.

And don’t get me started about the prices. A Raspberry Pi 5 with 16G of RAM is 260 CHF. 260! You used to be able to buy more computer for that amount of money. The RAM price madness is ostensibly because of AI, but in reality it’s more about the various people involved lining their pockets, at the expense of all of us.

A while ago, I made an off-hand remark that got some followup questions. I said that in AI applications, Model Context Protocol (MCP) is no longer a favored extension model for tools. The question was essentially – Really? What to use instead?

What is the problem with MCP?

MCP is a chatty protocol. Connecting to any MCP endpoint will hand the AI model a full description of all the possible calls, including some explanatory text and parameters.

The problem with this is that it consumes a lot of tokens. An LLM has something akin to a working memory, called the context window. Adding several MCPs, together with the system prompt, might consume most of the context window already. What tends to happen then is that the LLM forgets about earlier stuff, e.g. what’s in the system prompt. Suddenly, it might start talking to the user in a foreign language, for instance.

Are native tools better?

Agentic development frameworks such as Google ADK let you add tools natively by writing code. This lets you get around some of the high token usage issues.

You can also scale up the number of usable tools by making more specialized sub-agents. The main agent can decide which sub-agent should do the task and hand it off. The A2A protocol (agent to agent) enables this. A2A is usually a native feature of these agent frameworks.

Fundamentally, if you want your system to be able to do a lot of different things, writing these specialized tools for every purpose does not scale well. At work, we have found an approach that works suprisingly well for doing “arbitrary” things though.

“Meta” tools to the rescue

You can create an agent that can do a large variety of tasks with a very narrow subset of tools:

1. Command line interfaces for all the integrations that you would like. For example, a CLI to query the bug tracker, one to send emails, etc.
2. Documentation for all the required CLI tools, ideally in Markdown.
3. An agent tool to search the corpus of documentation.
4. An agent tool to run shell commands.

With this combination of things, LLMs are pretty capable of formulating a search term, reading the results and calling the tool. I think this is because shell syntax is part of their training set.

If the LLM has the capability of creating a shell or Python script and running it, it can do actual computation in this way, e.g. for grepping over a result or calculating “2+2” by asking the Python interpreter.

The amazing thing about points 1 and 2 is that CLIs and docs are super useful to humans too! So by making a system logical, easy to understand and well documented, you are making it easier for the LLM and for the human. Win-win.

I have stuggled with writing this for a while. This has the potential to be controversial: Some people have very strong opinions on this, while my own opinion is more … nuanced. –

I live in two different worlds regarding AI.

During the day, I am working in a big tech company that is betting big on AI. AI is everywhere, and (at least some) people are excited about the possibilities.

Outside of work, when I log in on Mastodon, the typical poster there thinks that anyone using AI for anything should be ashamed, and that AI is unequivocally bad.

Going with the times

In Tech, things change all the time. If there is some new development that you don’t like, you might decide to “boycott” it – however, you risk looking very silly down the line when the thing you despise is just normal, while you made disagreeing with it your whole personality. For example:

• In the late 90s, some people were very upset about computers no longer having a floppy drive, or only supporting USB for peripherals, so they swore that they would never buy that new-fangled stuff. Try finding a laptop with a floppy drive today.
• Also in the 90s, you might have decided to ignore that new-fangled “internet” and “WWW” thing. Anyone can edit Wikipedia, right? That clearly makes the printed encyclopedia in the shelf much more useful!! You would have missed a lot of exciting stuff in tech in the following decades.
• In the 2010s, you might have decided that “There is no cloud, only other people’s computers”, but you would have missed out on a number of cool innovations. (That’s another post for the future.)
• On the other hand, in the early 2020s, you might have decided early on that NFTs are stupid, and you would have been right.

So what about AI? I feel the jury is still out, overall. There are aspects of the hype that are clearly stupid. There are also AI-related things that are useful. I feel that dismissing everything containing AI means depriving myself of some useful tools. Not using the best tools is going to mean that I cannot do my best work.

Maybe that’s the FOMO argument. But it is also the “lifelong learning” argument, particularly in a professional context.

Example: Coding assistants

Coding assistants can be a giant productivity boost. At Google and Microsoft, statistics claim that up to half of coding keystrokes are “autocompleted” from an AI assistant. I can believe it! Never mind that you sometimes end up deleting most of what the assistant has written because it has misunderstood what you wanted to do.

Some situations in which a coding assistant is helpful for me:

• Framework boilerplate. For instance, I recently ported a Go command-line tool to Cobra. The assistent could spit out Command structs and Cobra registration functions based on the other code in the file. I would imagine that JS frameworks work really well for this too.
• The boring parts of the code. “Create an argument parser supporting the -f, -d and -h options”. This, too, is kinda boilerplate.
• You can narrate your code by using comments like subheadings. You write something like // Connect to the server. and let the assistant propose a dozen lines.
• You can ask the chatbot for ideas on how to approach something, like “How do I make the JS on this page communicate back and forth with the server?”, let it suggest a websocket and make a skeleton implementation of the two sides.

What these things have in common: I want to understand the code that the assistent generates. And I have the last word and proofread everything. Typically, there are some bugs in what is generated. I don’t like the Vibe Coding approach, where I just let the machine make random edits to make errors go away.

If – and only if – you are the one that stays in control, an AI assistant can be like “a bicycle for the mind”.

Don’t buy the hype

While all of the above is pretty positive, there are many aspects of the AI hype that I really do not like. Consider “slop” – the endless stream of low-quality, uninspiring content created from LLMs, image generators, and more. Grifters seem to really like those.

Then there is the societal argument about the consequences of large-scale deployment of AI. There is clearly an effort by some to replace knowledge workers with unaccountable machines.

You apply for a credit card, the AI declines your application, and there is no way to appeal.

Translators and linguists are laid off because “AI does the job well enough” (cf. Duolingo).

Personally, I find the endless boasting about GenAI in particular just boring. Half the posts on my LinkedIn now are people leaving their jobs (which is expected), while the other half is discussing generative AI. Are there no other topics?? But look at me writing this, I am part of that problem now :/

Conclusion

As said above, I think the jury is still out. It is probable that the AI bubble will burst at some point, with painful consequences for the entire tech industry. It is also probable that GenAI of some sort will be here to stay. So in my opinion, the best option is to find tools that are useful but do not believe all the hype.