pi compaction

How pi compacts session context, based on reading the source of earendil-works/pi (main, June 2026).

Sources

Architecture

The session is an append-only JSONL tree (every entry has id/parentId). Compaction is non-destructive: it never deletes messages. It appends a compaction entry { summary, firstKeptEntryId, tokensBefore } as a child of the current leaf. The LLM context is rebuilt by buildSessionContext() walking leaf→root; if a compaction node is on the path it emits:

1. The summary, as a user message: "The conversation history before this point was compacted into the following summary:\n\n<summary>…</summary>"
2. The raw kept messages from firstKeptEntryId up to the compaction node
3. Everything after the compaction node, raw

Triggers (_checkCompaction in agent-session.ts)

• threshold: contextTokens > contextWindow − reserveTokens (default reserve 16,384), checked after each agent run and before each prompt; no auto-retry
• overflow: API returned a context-overflow error → drop the error message, compact, auto-retry once
• manual: /compact [instructions]

Settings (CompactionSettings)

Q1: /tree back to before the compaction → context is NOT compact

The compaction is just a tree node. Navigating with /tree sets leafId to an earlier entry; buildSessionContext() walks from that leaf to root, and the compaction node is simply not on that path. You get the full, original uncompacted context back (everything is still in the JSONL).

Caveat: if that context is over threshold, the pre-prompt check will trigger a fresh compaction on your new branch as soon as you send a message.

Q2: Recent turns are kept verbatim — not lost

The model still sees real recent messages after compaction. findCutPoint() walks backwards from newest, accumulating estimated tokens (chars/4), until it has keepRecentTokens (default 20,000). Everything from the cut point forward is kept as actual messages — user text, assistant text/thinking/tool calls, and tool results. Only messages older than the cut are replaced by the summary.

Valid cut points: user / assistant / bashExecution / custom messages — never a toolResult (a result must stay glued to its tool call).

Example A — clean cut at a user message

A session before compaction (token counts estimated as chars/4):

 idx  role        content                       est.tokens
 ───  ──────────  ───────────────────────────   ──────────
 E1   user        "Fix the build"                     300
 E2   assistant   [read+edit tool calls]            2,000
 E3   toolResult  file contents                     4,000
 E4   assistant   "Fixed, here's why..."              500
 E5   user        "New error log: [BIG PASTE]"     12,000
 E6   assistant   [edit tool call]                  1,000
 E7   toolResult  edit ok                             500
 E8   assistant   [exec tool call]                    400
 E9   toolResult  test output                       7,000
 E10  assistant   "All green"                         300

Step 1 — walk backwards, fill the “keep budget”:

            walk direction ◄──────────────────────────────

 E1     E2     E3     E4     E5      E6     E7    E8    E9     E10
┌────┐ ┌────┐ ┌────┐ ┌────┐ ┌─────┐ ┌────┐ ┌───┐ ┌───┐ ┌────┐ ┌───┐
│ 300│ │2000│ │4000│ │ 500│ │12000│ │1000│ │500│ │400│ │7000│ │300│
└────┘ └────┘ └────┘ └────┘ └─────┘ └────┘ └───┘ └───┘ └────┘ └───┘
                             ▲
                     accum:  21,200  9,200  8,200 7,700 7,300  300
                             ─────
                             ≥ 20,000 → STOP here, at E5 (user msg)

E5 is a user message → valid cut point → clean cut.

Step 2 — the result:

 BEFORE (entries on the branch):

 ┌──────────────────────────────┬───────────────────────────────────┐
 │  E1  E2  E3  E4              │  E5  E6  E7  E8  E9  E10          │
 │  (older history)             │  (recent ~21k tokens)             │
 └──────────────┬───────────────┴────────────────┬──────────────────┘
                │                                │
                ▼ one LLM call                   ▼ untouched
        "## Goal ... ## Progress ..."     kept VERBATIM

 AFTER — what the model's context window contains:

 ┌─────────────────────────────────────────────────────────────────┐
 │ [user] "history before this point was compacted: <summary>      │
 │         ## Goal ... ## Next Steps ... </summary>"  ◄── replaces │
 │                                                        E1..E4   │
 │ [user]       E5  "New error log: [BIG PASTE]"      ◄── REAL msg │
 │ [assistant]  E6  [edit tool call]                  ◄── REAL msg │
 │ [toolResult] E7  edit ok                           ◄── REAL msg │
 │ [assistant]  E8  [exec tool call]                  ◄── REAL msg │
 │ [toolResult] E9  test output                       ◄── REAL msg │
 │ [assistant]  E10 "All green"                       ◄── REAL msg │
 └─────────────────────────────────────────────────────────────────┘

So: the last ~20k tokens of turns survive as-is — including assistant thinking, tool calls, and full tool results. Only the part older than the cut is reduced to the summary.

Example B — the cut lands mid-turn (“split turn”)

Suppose one turn is huge because of a giant file read:

 E5   user        "Refactor module Y"                 150
 E6   assistant   [read tool call]                    500
 E7   toolResult  ENTIRE 48k-char file            12,000
 E8   assistant   [edit tool call]                  1,000   ◄─ budget filled
 E9   toolResult  edit ok                             200      walking back
 ...  (more work) ...                              ~9,000      through here

The backwards walk crosses 20,000 inside E7 — but E7 is a toolResult, not a legal cut. The nearest valid cut point at-or-after it is E8 (assistant) → cut at E8, which is the middle of the turn started by E5:

        turn started by E5
 ┌────────────────────────────────────────────┐
 │  E5(user)  E6(asst)  E7(tool)  ║  E8 E9 ...│
 └────────────────────────────────╫───────────┘
      "turn PREFIX"               ║   "turn SUFFIX"
            │                     ║        │
            ▼                    cut       ▼
   summarized SEPARATELY          ║   kept VERBATIM
   (Original Request /            ║
    Early Progress /              ║
    Context for Suffix)           ║

 Resulting single summary message:

 ┌──────────────────────────────────────────────┐
 │ <summary>                                    │
 │   ## Goal ...            ◄─ E1..E4 history   │
 │   ---                                        │
 │   **Turn Context (split turn):**             │
 │   ## Original Request    ◄─ E5..E7 prefix    │
 │   ## Context for Suffix                      │
 │ </summary>                                   │
 └──────────────────────────────────────────────┘
 then E8, E9, ... follow as real messages.

The two summaries (history + turn prefix) are generated in parallel and merged into one summary message.

The window slides — second compaction

Verbatim retention is a rolling window. Messages kept raw today get folded into the summary tomorrow (via the UPDATE prompt, which merges with the previous summary):

 Compaction #1:
 ┌─ summarized ─┐┌──────── kept raw ────────┐
 │ E1 ........ E4││ E5 .................. E10│ E11 E12 ... (new turns)
 └──────┬───────┘└────────────┬─────────────┘
        ▼                     │
    [SUMMARY v1]              │
                              ▼
 Compaction #2:               folded in via UPDATE prompt
 ┌──────────── summarized ─────────────┐┌──── kept raw ────┐
 │ SUMMARY v1  +  E5 ............. E14 ││ E15 .......... E20│
 └──────────────────┬──────────────────┘└──────────────────┘
                    ▼
               [SUMMARY v2]   ("PRESERVE all existing info, ADD new,
                                move In-Progress → Done, ...")

So at any point the model sees exactly three layers:

 [ cumulative structured summary ]  ← everything old, lossy
 [ ~20k tokens of verbatim turns ]  ← exact user/assistant/tool messages
 [ everything after compaction   ]  ← exact, grows until next trigger

And because the compaction is just a tree node, the originals (E1–E4 etc.) are never deleted from the .jsonl — /tree can always get back to them (see Q1).

Q3: The compact prompt and /compact <args>

The summarizer is a separate one-shot LLM call (same model/thinking level as the session, maxTokens = 0.8 × reserveTokens):

• System prompt (SUMMARIZATION_SYSTEM_PROMPT in compaction/utils.ts): “You are a context summarization assistant… Do NOT continue the conversation… ONLY output the structured summary.”
• User message: the history serialized to plain text inside <conversation> tags (tool results truncated to 2,000 chars each), plus <previous-summary> if updating, then the instruction prompt demanding this EXACT format:

## Goal
## Constraints & Preferences
## Progress  (### Done / ### In Progress / ### Blocked)
## Key Decisions
## Next Steps
## Critical Context

…with “Preserve exact file paths, function names, and error messages.” Afterward pi mechanically appends <read-files> / <modified-files> lists extracted from read/write/edit tool calls.

/compact <args> passes the text as customInstructions, which is simply appended to the base prompt as:

{base prompt}

Additional focus: <your args>

So it doesn’t replace the structured format — it adds a focus hint (e.g. /compact keep all the API endpoint details). Custom instructions are manual-only; auto-compaction (threshold/overflow) always uses the plain prompt. Extensions can fully override compaction via the session_before_compact event.