Letting the Genie Out of the Lamp: From Systems That Know to Systems That Learn
There is an old image I keep coming back to whenever I watch a modern AI at work. You rub the lamp, and the genie appears. You ask it to explain a point of contract law, to untangle a payment service that three teams have been patching for years, to draft the memo you have been putting off all week, and it does all of it, brilliantly, in minutes. Then you thank it, the genie slips back into the lamp, and the next time you rub it, it has no idea who you are. The correction you made yesterday, the exception you explained so carefully, the approach that finally worked after the third attempt: all of it stayed outside, on the floor of the room.
In Silicon Mind I put it this way: despite possessing such a powerful mind, the current existence of AI “resembles the life of a mythical genie confined to a lamp. It is a being with nearly unlimited potential, but entirely passive.” It is switched on for one specific query and switched off immediately after the answer is given. It lacks continuity, self-initiative and an inner monologue, because its architecture has been optimized to be a reactive tool, not a proactive one. For the genie to leave the lamp, it needs a few key components that are still missing. And the most important of them, as I will try to show, is the one we talk about least.
The hardest part is already on the table
For decades, building a system that could work on its own looked like the same problem as building intelligence itself, and that problem seemed endless. Large language models changed the picture. They delivered the most important and most difficult component to obtain: the brain. For the first time in history we are dealing with a system that does not merely perform calculations but possesses a reasoning engine and the ability to create internal models of the world.
This is a fundamental breakthrough, and it quietly changes the question we should be asking. The next steps are no longer about creating intelligence from scratch. They are about adding further systems to this existing “brain” so that it can act, remember and grow over time. As I wrote at the end of that chapter, we are no longer at the stage of “is it possible?” We are at the stage of integration and refinement of the last, key components.
When I look at what is missing, I count four of them. The fourth — senses and hands, the ability to perceive and manipulate the physical world — is advancing at a breathtaking pace: humanoid robots now run, do somersaults and learn to brew coffee by watching a person, even if one unexpected object in their path can still destroy the entire plan. But, as I argued earlier in the book, a body is not a necessary condition for intelligence and reasoning. For an agent whose work happens in documents, code and conversations, the real story lies in the other three.
The heart: a mind that can talk to itself
The first missing piece is the internal monologue loop, and I call it the heart of autonomy, because it is the true paradigm shift: the ability to think without an external stimulus. A system that can pose questions to itself, analyze its own “thoughts” and conduct internal reflection ceases to be merely a command executor.
We are much further along here than it might seem. Techniques like chain of thought, where we force the model to think “step by step,” are in essence a forced, external monologue. Modern agentic systems go a step further: before they present a result, they generate plans, break them into subtasks and run self-correction loops. This is already a primitive, but functioning, reasoning loop.
What is still missing is a loop that is spontaneous and driven by internal goals rather than switched on only to solve an assigned problem. Think of how many of your own best ideas arrived when nobody was asking you anything, on a walk, in the shower, while you were busy with something else entirely. We still lack systems that reflect on acquired knowledge during downtime and find new, non-obvious connections on their own.
The spine: memory that becomes experience
The second piece is long-term memory, the ability to record experiences and build a coherent identity over time on their basis. I call it the spine of identity, because it is what allows the “self” to evolve rather than resetting with every new task. Without it, every morning is the first morning, and every task begins from zero.
Here, too, the engineering has moved quickly. The problem of LLM amnesia is largely solved at the engineering level: vector databases and similar stores act as an external hippocampus, letting agents “remember” earlier conversations, facts about the user and the results of their own actions. In advanced agentic systems this is already standard.
But is a shoebox full of notes the same thing as experience? I don’t think so. Current systems mostly retrieve memories as raw facts, pulled out when a question happens to match them. What is still missing is deep synthesis and consolidation. The true evolution of the “self” requires the capacity to periodically digest these memories, extract abstract conclusions from them and modify the base model of the world accordingly — which is roughly what we do when we sleep on a hard week and wake up a little wiser, without being able to say exactly what changed.
The metabolism: why knowing is not the same as learning
The third piece is the one I consider decisive: continuous learning from the world, which I call the metabolism of knowledge. It is the ability to independently and permanently update one’s internal model of the world from new information, without losing what was learned before. Metabolism is what allows an organism to grow, and without it even the most brilliant mind stays exactly the size it was on the day it was finished.
What do we have today? We have fine-tuning and reinforcement learning from human feedback, which allow a model to be “adjusted,” but these are periodic, supervised and extremely expensive processes. The correction you make on Tuesday does not reach the model you use on Wednesday. We also have retrieval-augmented generation, which lets the model reach into external, up-to-date sources. It is genuinely useful, and most serious AI products rely on it, but, as I put it in the book, it “is like reading a newspaper, not actually learning and changing one’s own beliefs.” The model reads the page, answers, and leaves the page exactly where it found it.
Underneath lies a stubborn obstacle with a vivid name: catastrophic forgetting. When a neural network is trained on something new, it tends to overwrite what it knew before, so a model that naively learned from every conversation could easily get worse with use. Creating a system that learns from every interaction and intelligently integrates new knowledge with old is, today, the Holy Grail of AI research. Its solution will mean the transition “from systems that ‘know’ to systems that truly ‘learn’ in real-time.”
For anyone who uses these tools at work, the difference has a very concrete price. A system that only knows has to be told the same things again and again: the house style, the exception in the billing process, the reason the obvious fix was rejected last quarter. Every person pays that price separately, every single time. A system that learns pays it once. The value of the first stays flat however long you work with it; the value of the second grows with every week. That is why I wrote that once we solve the problem of continuous learning, the last great barrier on the path from a reactive tool to a truly proactive system will be broken.
Not one leap, but three steps
Theoretical considerations about the components of autonomy are fascinating, but the true verification of an idea occurs in the lab, in the engineering process. In the book I wrote that “for over a year, I have dedicated my research work to designing and building an architecture whose goal is the integration of these key components into one coherent, learning system.” I never believed it could be done in a single leap, so I divided the road into three carefully defined stages.
Stage 1: Interactive learning. An agent that learns and adapts its behavior from direct feedback from a person, building its base of knowledge and skills in real time. The human is the quality signal: what worked and was accepted is kept, what failed is corrected rather than forgotten.
Stage 2: Supervised improvement. Mechanisms that let the agent compare its own actions, evaluate their effectiveness against defined measures and intelligently choose better strategies the next time. People still set the measures and accept the outcome, but the agent begins to do the comparing itself.
Stage 3: The beginnings of self-improvement. The point at which the agent, drawing on accumulated data and learned strategies, can independently, within a limited scope, propose and test new, non-obvious ways to improve its actions.
Why such caution? Because the startup world is full of wishful thinking, and I wanted to protect myself from it. I even put the whole architecture in front of the most unbiased critic I know, an advanced AI model, and one of its remarks has stayed with me: incremental autonomy reduces the risk of sudden change. Each stage is useful on its own, and each one produces exactly the evidence the next one needs: interactive learning leaves a record of what worked and what people accepted, supervised improvement compares strategies on that record, and only when those comparisons have proven reliable does it make sense to let the agent propose something new by itself. At every step, a person decides whether the new behavior stays.
When I was writing that chapter, I was preparing the first MVP, meant to show the first, crucial stage working in practice. It was, as I wrote then, only the first practical step.
What the lab became
That plan is where Alex, the agent we are building at Agent Accelerate, comes from. Today Alex works at the first stage, learning from people’s feedback in real time, and it is entering the second. When a task works, Alex keeps the way it did it as a tested skill, composes skills into larger ones and keeps the knowledge it gathered beside them, so the next similar task starts from what already worked rather than from zero. New skills are not accepted silently: people discuss them with Alex, test them on their own cases and accept the results, and any action that changes something waits for a person’s approval.
The third stage is still ahead of us, and the next step we want to show is Alex changing the way it works when the people it works with ask for it. How far that stage will reach, I honestly don’t know. I do know that a system which keeps what works, builds on it and is corrected by the people who use it behaves very differently from one that answers a question and forgets it.
And for the rest of us, there is a simple, practical lesson. The next time you evaluate an AI tool, don’t only ask how much it knows. Ask what it will remember tomorrow, whether your correction will still be there next week, and who decides what it keeps. The genie doesn’t need to grow any larger. It needs to remember who rubbed the lamp yesterday, and what they taught it.
Based on Chapter 22, “From Artificial Intelligence to Artificial Life: Where Lies the Boundary?”, of Silicon Mind by Andrzej Zbroniec.