Semantic lightness and the rise of LLMs
Semantic lightness and the rise of LLMs
If we look at using LLMs to write code and you have seen your fair share of similar technologies, there is an obvious reason why enterprise development seems suitable: semantic lightness to using architecture-heavy code. This has been parodied in Enterprise FizzBuzz. It wouldn’t be funny if there was no truth to it. Though, from working in enterprise environments I’ve seen the opposite: under-engineering (sloppy and brittle code that has grown out of the simple stage into hard-to-maintain software). The semantic lightness in such codebases then becomes the amount of copy-paste due to the fear of breaking old code, leading to a sprawl of similar but slightly different versions of the same thing (sometimes to implement a new feature, sometimes to implement the same thing but with an added requirement). The lack of structure and organization also leads to accidental duplication.
The difficulty in writing anything intended for consumption (not yourself only) is knowing your audience. Reading space opera, detective stories, romance novels that are formulaic, action-packed and fast-paced can be fun. If you have read too many of those and would like something that deviates from the norms, that challenges your assumptions, that requires you to think about relationships and logic, then that requires significantly more effort. I read one or another depending on the mood.
Enterprise development prizes formulaic and impersonal writing because it promises to erase the need to have a star writer. If you try to slot in resources (people) that don’t need to think but only need to follow the spec, who writes the spec? Who defines the common structures? The overall organization? The burden of thought is shifted to someone else. You split up the burden, but the communication overhead increases. This is why you sometimes see small teams deliver the equivalent of an office of teams’ worth of functionality. Can a small team take over the burden of maintaining something written by an office of software engineers? You might have significant overhead due to the semantic lightness of the code produced by several teams. Every team worked to produce useful software and wrote more and more code. The amount of code can be the impediment.
It has been common to generate trivial code even before LLMs. We have had code generators such as SwaggerGen, JHipster, WYSIWYG, MDA/UML generation, T4 templates, jinja, database schema generators, code generators based on database schemas, etc. It feels odd that people are talking about lovingly hand-crafted code versus machine-generated, when old devs have generated millions of lines of code whenever there was a need. The difference was that you got code that you knew what it did because the tool didn’t do anything overly complicated like using vast neural nets to approximate the same probability distribution as hand-written or pre-LLM generated regular code. There is a difference in opacity of using a tool that uses few rules to generate millions of lines of code. The difference is also that if you don’t know that all of these tools exist, don’t know how to use them, then the “easier” option is to use an LLM. We are reminded of Rich Hickey’s distinction of “Simple versus Easy”. We should also be reminded of the fact that in some cases you could remove the need to generate vast quantities of code by the introduction of better libraries and abstractions.
Having an LLM write an arbitrary number of lines of code is trivial. Having it be decent looking and have tests is usually not that difficult, but requires a bit more effort and knowledge. Different people have different ideas on how that can be done. Matt Pocock gives his idea around using a process with skill files and prompts. Uncle Bob posits that you can get good results by having deterministic tools constrain the models in order to avoid having to waste model “intelligence” (context) on overly long prompts and skills. The inherent issue with LLM-generated code is the need for verification (that it does what you intended it to do). What Uncle Bob and Matt Pocock talk about from different perspectives is that you can get a better (readable) end result than just letting the models go with minimal steering.
What does another version of known software, already open-sourced in multiple variants, bring to the world? Having a slightly different CRM built using millions of lines of Go? The point here is that the software is novel in the sense that it is newly generated, but not novel in the sense that it does something new. The equivalent of cheap pocket books with predictable plotlines? Or is it the equivalent of a badly translated manual for a complicated product where the translator has not seen the product and does not fully understand the target language? Is it a cart made out of cardboard or a regular car?
There is a semantic difficulty of using simplified language to discuss advanced topics. There is an obvious advantage if you can express an equivalent software using a few lines of readable but dense text compared to lots of structure and a few business rules sprinkled into it. I’ve read books on math that have higher semantic density than far longer texts. The authors told me that the economic incentives for them were different from those for American published authors, who were paid by the page (or by the word, I don’t remember). The economic incentives around writing software also shape how software gets written in large and small organizations. If you reward lines of code as a productivity metric, if middle management or billing is rewarded based on headcount, if people are meant to be interchangeable drones, then you get volume. If you have an obscene amount of text that no human wrote, you have baked in a new form of bus factor (you will need new tools to manage that mess). As is the case when you try to optimize for a few things by creating a complicated web of code that no person can fully understand because of missing abstractions, you can shoot yourself in the foot in other ways. Some abstractions are like vehicles: they allow you to carry more and travel further. To choose not to use any transportation service or mechanical vehicle in order to avoid that dependence is seen as silly.
What is the point I’m trying to make? I think we need to take a close look at how we “write” software. If the information density of code compared to a detailed spec is low, then perhaps there is something wrong with the computer languages, frameworks and libraries that we use. Perhaps we will see novel ways of using more semantically dense languages and libraries in order to outthink people who rely on “basic” construction blocks.
Tags
Do you want to send a comment or give me a hint about any issues with a blog post: Open up an issue on GitHub.
Do you want to fix an error or add a comment published on the blog? You can do a fork of this post and do a pull request on github.
Comments