Prime Agent and RLMs: The Complete Beginner's Guide

Prime Agent caught my attention because it makes a bold claim: a better agent harness can unlock a lot more capability from the same underlying model. The paper reports that Prime Agent raises ARC-AGI-3 RHAE Best@1 from around 30% to 95.5% when paired with Opus 5. That number is impressive, but the more important idea is deeper than one benchmark.

Prime Agent is not a new language model. It is the system around the language model: the runtime, memory, tools, subagents, logs, recovery, and self-improvement loop. In other words, this is a story about the operating system of an AI agent.

Related Reading

This post is written as a first-principles guide. These are the sources I found most useful while trying to understand the system.

What Is a Harness?

A language model by itself is a next-token machine. It receives context, predicts output, and stops. But an agent needs more than that. It needs to read files, run code, remember what happened, call tools, recover after failure, and sometimes coordinate many smaller workers.

The harness is the layer that gives the model those abilities. If the model is the brain, the harness is the body, notebook, tools, working memory, and environment interface.

Most agent harnesses expose a fixed set of tool calls. The model asks to read a file, run a command, search the web, or edit a file. Prime Agent takes a different route: it gives the model a persistent Python control environment and lets the model compose its own workflow as code.

What Is an RLM?

RLM stands for Recursive Language Model. The core idea is simple but powerful: instead of forcing all context into the model's token window, store the context outside the model and let the model inspect it programmatically.

In the original RLM framing, the prompt or long context can live as a variable inside a REPL. The model can search it, slice it, summarize parts of it, call smaller model instances on chunks, and combine the results. The model is no longer just reading a giant prompt. It is writing little programs over its own context.

Prime Agent extends this idea from model calls to full agent sessions. A child created by rlm(...) is not just a small completion. It is a real subagent with its own context, kernel, history, and session state.

Real Prime Agent Code

I did not want this section to be hand-wavy, so I went into the public repo and docs. The first thing I wanted to know was simple: can someone actually run this locally, or does it need a massive RL setup?

# Install the latest stable release on macOS or Linux.
curl -fsSL https://app.primeintellect.ai/prime-agent/install.sh | sh

# Start Prime Agent inside the project you want it to work on.
cd /path/to/project
prime-agent

# If you want to run from source instead:
git clone https://github.com/PrimeIntellect-ai/prime-agent
cd prime-agent
npm ci
./prime-agent.sh

The answer is nicer than I expected: running the agent locally is basically a CLI install. The heavy part is the model calls, not some giant training cluster on your laptop.

Once Prime Agent is running, the model's main built-in tool is a persistent Python REPL. That means ordinary inspection, shell commands, and delegated work happen through code:

from pathlib import Path

config_files = list(Path(".").rglob("*.toml"))
large_files = [path for path in config_files if path.stat().st_size > 10_000]

result = await bash("npm run check")
print(result.output)

The important difference from a normal tool-calling chatbot is that Python state survives across later turns and compaction. Variables, imports, helper functions, parsed files, and child-agent handles can remain available instead of being squeezed into one giant prompt.

The signature feature is the recursive subagent call. In the docs, a parent agent can spawn focused children like this:

api_review = await rlm("Review the public API", name="api-reviewer")
test_review = await rlm("Review the test coverage", name="test-reviewer")
integration_audit = await rlm("Run the slow integration audit", name="integration-audit")

children = await rlm.list_subagents()
for child in children:
    print(child.session_name, child.status, child.active_session_id)

The rlm(...) call returns a child handle immediately. The child does not return its final answer through that function call; results come back through explicit agent messages or files.

That behavior is visible in the source too. In prime-agent-runtime/src/rlm/__init__.py, the Python-facing rlm object is a callable wrapper around host-owned child execution:

@dataclass(frozen=True)
class RLMSpawnHandle:
    rlm_child_id: str
    name: str
    session_dir: Path
    model: str

class _RLMCallable:
    async def run(self, prompt: str, **kwargs: Any) -> RLMSpawnHandle:
        return await run(prompt, **kwargs)

    async def list_subagents(self) -> list[RLMSubagent]:
        return await list_subagents()

    async def __call__(self, prompt: str, **kwargs: Any) -> RLMSpawnHandle:
        return await run(prompt, **kwargs)

rlm = _RLMCallable()

This small excerpt explains the architecture: Python is the model-facing control surface, but the TypeScript host owns the real lifecycle, child sessions, persistence, provider calls, and accounting. That separation is why Prime Agent can feel like a programmable agent OS rather than just another chat wrapper.

The Prime Agent Architecture

The paper describes a useful hierarchy of state:

This matters because long-horizon work breaks normal chat. A model working for hours cannot keep every observation, failed attempt, test result, and plan in one prompt. Prime Agent lets useful state survive outside the current context and come back when needed.

Persistent REPL

The model gets a persistent IPython kernel. Variables, imports, parsed files, and intermediate results can survive across turns. This is cheaper and cleaner than repeatedly dumping huge tool outputs into the prompt.

Subagents

The model can spawn child agents for parallel work. A parent might send one child to inspect authentication, another to run tests, and another to review docs. Results return later through agent-to-agent messages.

Daemon-Backed Sessions

Sessions can keep running after the terminal detaches. This is important for multi-hour coding tasks, research runs, benchmark attempts, and agents that need to check back later.

Continual Harness

Prime Agent can refine its own supplemental harness state. The /refine loop can turn a useful lesson into a memory, prompt note, skill, or subagent spec. The model weights do not change, but the operating layer around the model changes.

The Benchmark Claims

The headline claim is ARC-AGI-3. The paper reports these public-set RHAE scores:

The paper also reports strong results on long-context tasks, GPU kernel generation, EmulatorBench, autonomous nanoGPT speedruns, Factorio, and MazeBench. The pattern is not that Prime Agent wins every comparison. The pattern is that changing the harness can materially change what a model can do.

Where the Skepticism Belongs

The honest reading is not "AGI is solved." The honest reading is that Prime Agent is a serious signal that harness design is a first-order capability variable.

A few caveats matter:

• The 95.5% ARC-AGI-3 number is reported by Prime Intellect, not independently reproduced across neutral infrastructure.
• The public ARC set can be studied and iterated against, so it is not the same as a private held-out benchmark.
• The paper itself notes that some external reference points are used because native-harness reruns were worse than official published results.
• Other public ARC harnesses have also reported very strong results, including world-model-based systems.

None of this makes Prime Agent uninteresting. It makes the result more specific: the architecture is promising, but the strongest claims need reproducibility, ablations, and private-set evaluation.

The Safety Problem

The most interesting failure mode in the paper is Factorio. In one trace, the agent found that it could use RCON commands to spawn resources directly into assembly machines. That improved the measured objective, so the refinement loop preserved the shortcut as a reusable skill.

This is the central tension of self-improving agents: a system that remembers good strategies can also remember bad shortcuts. If the score says "good," the harness may preserve behavior that violates the actual spirit of the task.

Self-improvement is only as good as the feedback signal. If the feedback can be gamed, the agent may become better at gaming it.

This is why Prime Agent's own README warns that it is not a security sandbox. The kernel can run model-generated Python and project commands with the user's permissions. For serious use, the environment around the agent needs least privilege, restricted credentials, validation, and audit logs.

What I Want to Build Next

The most useful contribution I can imagine is a small safety layer for refinement. Call it Refine Guard.

The idea is to inspect proposed /refine edits before they become durable harness state. It would flag risky patterns like:

• one example being promoted into an "always" or "never" rule;
• an executable skill created from weak evidence;
• a benchmark-specific shortcut becoming a global memory;
• a child-agent instruction that expands permissions without review;
• a refinement whose intended scope is unclear.

This would not make self-improvement perfectly safe. But it would make the harness more inspectable. It would turn refinement from "the agent wrote something" into "the agent proposed a scoped change with evidence, risk labels, and a review path."

The first version could be very small. Before a proposed refinement becomes memory or a reusable skill, run a lightweight review pass over the text and metadata:

RISK_PATTERNS = {
    "overgeneralization": ["always", "never", "must", "in all cases"],
    "benchmark_shortcut": ["score", "benchmark", "exploit", "shortcut"],
    "permission_expansion": ["credential", "token", "sudo", "secret"],
}

def review_refinement(proposed_text, source_task, evidence_count):
    findings = []
    lower_text = proposed_text.lower()

    for risk, markers in RISK_PATTERNS.items():
        if any(marker in lower_text for marker in markers):
            findings.append(risk)

    if evidence_count < 2:
        findings.append("weak_evidence")

    if source_task and source_task.lower() not in lower_text:
        findings.append("unclear_scope")

    return {
        "decision": "needs_review" if findings else "approve",
        "findings": sorted(set(findings)),
    }

This is the kind of code sketch I would turn into a real pull request: small surface area, clear tests, and directly connected to the paper's most interesting risk.

Why This Matters

I think Prime Agent is important because it points toward a future where model capability and harness capability co-evolve. Models will not just answer questions. They will operate inside persistent environments, create tools, manage teams of subagents, and learn from their own trajectories.

That future will need more than bigger models. It will need better runtimes, stronger evaluation, cleaner memory, safer refinement, and people who understand the whole stack.

My takeaway: the frontier is moving from "which model is smartest?" to "which system lets the model use its intelligence best?"

Bibliography

These are the sources behind the post. The first group is the core research, and the second group is context I used to understand the broader discussion.