Thoughts about the future of software after LLMs.
Autonomous reasoning is not a new term. But here, I'm interested in its meaning in the current context of programming and LLMs. And in this context, reasoning can be thought of simply deciding what to do next based on some state and input.
Application development typically consists of multiple reasoning steps, resulting in an application itself. This reasoning is based on common and domain-specific knowledge. As we now see, LLMs have some common knowledge and can also be given a task or domain-specific knowledge via prompts, directly or using retrieval-augmented generation. And with additional fine tuning to produce structured responses, LLMs now have the ability to take over parts of human reasoning. In other words, they can "develop" parts of the program on the fly, given some instructions. That's what OpenAI Function Calling, LangChain Agents, and Transformers Agents basically do.
In contrast, traditional software has very limited reasoning abilities, implemented explicitly as conditional statements. Logic programming, and languages such as Prolog, tried to broaden reasoning abilities of the computers. But automating human reasoning in a formal way didn't quite work out due to brittle knowledge representation, among other things. A high level analogy for facts and predicates in Prolog could be LMM weights and architecture.
The concept of intelligent systems isn't new either. Expert systems are probably the most relevant in this context. Having been based on logic programming, they inherit the same problems - brittle knowledge representation and limited reasoning.
The question is whether LLMs can pull it off. I believe that once there is a large enough set of available functions, or tools, that an LLM can reliably reason about and "call", using agents instead of developing traditional applications may actually become feasible for certain workflows.
By agent I refer to autonomous reasoning loops that don't need additional coding.
So theoretically, instead of developing a script, an application, or a service to perform a certain task repeatedly given some input, you will just pass a full/relevant context and instructions to the agent and it will perform the necessary actions.
Obviously, such agents cannot replace whole variety of possible applications, but certain types of applications may become redundant.
At Graphsignal we believe that autonomous reasoning will be an essential part of the software stack. But the path to production-grade AI applications and agents lies through continuous visibility, evaluation, and analytics. We've building the most comprehensive observability for new new AI software stack. Check it out and give it a try.