One way of understanding LLM agents is that they provide abstraction and sensible defaults over existing software.
Some inherent problems with agents are:
- that they have basically unlimited side effects,
- they're totally opaque,
- they're not deterministic,
- their runtime performance is poor, and
There are other problems, but they're arguably not inherent:
- ethical concerns (very theoretically, LLMs could plausibly be trained and used ethically)
- cost (theoretically, the financial, social, and environmental cost of running LLMs could become negligible)
- poor interfaces (i.e. unstructured plaintext, but harness design can already involve something more structured).
- their "implementation" is usually untested for the specific task at hand (but evals are possible)
There are mitigations against the inherent problems:
- side effects can be limited with sandboxing
- "Chain of Thought" and prompt engineering provides some very limited transparency on model behaviour, anecdotally
- LLMs are inherently non-deterministic, but the output space can be constrained in a probabilistic sense: particular outcomes can be made more probable e.g. with prompt engineering, lower temperature, model deployment hygiene, etc.
- their runtime performance is improving over time as the technology develops further
These are weak mitigations! We're better to write software to provide the abstractions we are looking for, and to encode our defaults. This can totally avoid the inherent problems, as well as the others. The limiting factors are our ability to conceptualize the abstraction though, and our labour in implementing. Our ability to conceptualize the abstraction is closely related to our ability to understand the problem we are solving.