I use LLMs at work, and I can't deny their utility. Writing code and summarizing a bunch of documents is easier than it used to be. I can ask the agent to do something, and it generally gets done. How well it gets done can be a different question, but then that's an issue with asking people to do things too. So far, one of the biggest differences I've noticed is that when a person does something a couple of times, you can generally trust that they're going to get better at it. That's not at all true with the LLMs. In fact, it seems like if you let the agent go on too long with something, the output is likely to get worse (it loses context).

Learning is where I think I have the biggest issue with the LMM agent craze. First, that the agents don't really learn. You can give them random markdown documents that are supposed to tell them what to know and how to act. Whether they follow the directions, especially if you let them run too long, is another question. There are too many public examples of agents "colouring outside the lines" to allow me to believe it's a good idea to just let them go. And even if you catch an error and tell the agent it made a mistake, the agent will apologise and then just do it again later. If I had a colleague like that, I'd hope they'd be fired fairly quickly.

Then there is the issue with people learning. It's quite clear that agents can take care of simple issues faster than many actual people (I'm not sure it's cheaper, but that's a different topic). The work the agent does needs to be reviewed for correctness and quality, but we do that for work from real people too (e.g. code reviews). Doing the simple work, leaning on a knowledgable person to check its work, sounds like the traditional role of a junior employee; and there has been lots of writing about how agents are coming for the entry-level jobs. I'll admit that having entry-level employees use agents to move more quickly sounds like good business sense, but I don't understand where that entry-level person is going to be able to pick up the skills to check the agent's work. AI-boosters say that the models will soon be good enough that we can trust them to almost always get it right. I wonder though, if we've now outsourced all of the work to agents, who will be able to do the work to make better models? And what do we do when the models are wrong, but we haven't learned the skills to figure it our for ourselves?

As a concrete example, I had an issue the other day where I needed to use an API to do something. I'm trying to figure out how to use the agents effectively and efficiently, the problem was clear, and the answer should be well documented somewhere, so I handed it off. The code that came out worked, but not the way I wanted. I tried prompting to find if there was a different way to get what I wanted, and the agent said no. Luckily, the way the agent thought the API worked was stupid. If it had really been designed the way the agent said it was, there would have been a million complaints. So, I decided to just read the docs myself. Lo and behold, there was a well documented parameter that made the API work the way I wanted. The change was all of 30 lines of code, including a new test case. If I thought I could just trust the agent to know, I never would have been able to build the feature. What will happen when our children just trust the agent (I think that ship has sailed, but maybe we'll get lucky)?

I think there is a place for LLMs and agents in our world. They can make us more productive. They can actually support learning and development. What I worry about is how we figure out what their limits are and how we can set guardrails when the machine's understanding of our instructions is dependent on how much we've told it since the instructions were read, and how semantically close some words are to other words. These machines seem magical, and sentient, and powerful - and they are in a way (i.e. Clarke's third law). A cutting torch seems like that too though (cut stuff with fire!?!). I hope we figure out how to use agents safely and responsibly before we cut all of our hands off.