Prompt injection is the new SQL injection only easier
Prompt injection is the new SQL injection only easier
As I try out more and more of LLM assisted coding. The security issues feel like they are taken extremely lightly. The security issue I am referencing right now is that like when you use untrusted input an adversary can take advantage. This is not the only security issue. Sam Altman deflects from the issue by talking about people “Google” for help on building biological weapons using their tools (i.e. that the training corpus contains sufficient information to allow for that). Other people have also expressed that it is a deflection from real issues.
Using LLMs trains you in becoming desensitized towards getting huge chunks of text pushed in your face. Having software that you cannot vet, because the amount of code has exploded, means that potential issues can be hiding in the snow. In the long term we get to a normalization of deviance make fertile ground for attackers. Or expressed differently there is a lethal trifecta for AI agents: private data, untrusted content, and external communication.
Using LLM assisted coding has many challenges. The way people use the tools changes as people discover ways that these tools can be made to work for them.
The interesting parts around security happens in how we use the tools. For instance there is docs around sandboxing for Claude, sandbox mode for Codex and security for GitHub Copilot.
For power users there is less focus on security as seen on for instance Kilo Code and Roo Code.
When I was but a young boy around 10 years of age (ca 1990), we swapped diskettes. I remember having to reformat my computer due to getting viruses. When I was in my 20ies (2001) I stayed at a landlady’s place in Lund, since I went to the university there. She had a computer that was noticeably full of malware. Later on when I moved to a student room, the local network had so many people with infected computers that if you plugged in a Windows computer that had just been installed, it would get infected before getting enough security patches from the net. I’m guessing that we are in a rhyming situation.
Tags
Do you want to send a comment or give me a hint about any issues with a blog post: Open up an issue on GitHub.
Do you want to fix an error or add a comment published on the blog? You can do a fork of this post and do a pull request on github.
Comments