MuMachine: Declarative Machine Lifecycle and Fleet Hydration on Disposable Compute

#Abstract

Relocating heavy agent seats between hosts exposed four architectural failures: disconnected ingress mistaken for liveness, unhydrated remote workspaces, long-lived bearer tokens on ephemeral disks, and manual SSH as the hydration path.

A machine is disposable compute. An agent is a sovereign identity with portable context. Sovereignty, identity, durability, and liveness live exclusively in Mupot, Cloudflare Workflows/Durable Objects, and D1/R2 — never inside the compute container.

This paper specifies MuMachine: a five-stage declarative lifecycle (allocate, hydrate, challenge, active, recycle), ephemeral generation-scoped session tokens, Kleppmann epoch fencing, a synthetic canary attach test, and nine mandatory acceptance gates.

#1. Problem

During a fleet rebalance, heavy agent seats were relocated from a memory-capped primary host to a remote VM. The move freed host RAM and simultaneously proved the current seat model unsafe:

1. Disconnected Ingress & False Liveness. Remote VMs ran bare socket listeners without inbox-capture or responder pipelines. Process PID health was mistaken for model/tool operational readiness.
2. Context Amnesia (Blank Starts). Remote workspaces were unhydrated stubs missing wake files, soul documents, boot briefs, rules, and history.
3. Ambient Bearer Tokens. Long-lived root API tokens were placed directly into VM filesystems, violating the zero-trust principle of disposable compute.
4. Manual Overhead. Recovery required ad-hoc SSH streaming and systemd manipulation, conflicting with the rule that fleet state must not be rsynced as a substitute for hydration.

#2. Adversarial Invariants

#3. Control Plane

                               ┌────────────────────────────────────────────────────────┐
                               │             MUPOT CLOUD CONTROL PLANE                  │
                               │  (Cloudflare Workers + D1 + R2 + Durable Objects)      │
                               └───────────┬───────────────────────────────┬────────────┘
                                           │                               │
                      1. Session Exchange  │                               │ 2. Signed Hydration
                      (Short-lived token,  │                               │    Manifest & Memory
                       Generation Fence)   │                               │    (HMAC-SHA256 via R2)
                                           ▼                               ▼
     ┌──────────────────────────────────────────────────────────────────────────────────┐
     │                             DISPOSABLE COMPUTE SEAT                              │
     │                 (Cloudflare Sandbox / GCP MicroVM / K8s AgentSeat)               │
     │                                                                                  │
     │  ┌─────────────────────────┐  Tool Calls (MCP)   ┌────────────────────────────┐  │
     │  │     Agent Process       │ ──────────────────► │  Local Egress Proxy        │  │
     │  │                         │                     │  (Injected Session Nonce)  │  │
     │  └─────────────────────────┘                     └──────────────┬─────────────┘  │
     └─────────────────────────────────────────────────────────────────┼────────────────┘
                                                                       │ 3. Zero-Trust Egress
                                                                       ▼    (Mupot MCP Bus Only)
                                                       https://mupot.mumega.com/mcp

#4. Five-Stage Lifecycle

Every agent compute seat transitions through a five-stage lifecycle managed by a Cloudflare Workflow / Durable Object:

┌──────────────┐     ┌──────────────┐     ┌──────────────┐     ┌──────────────┐     ┌──────────────┐
│ 1. ALLOCATE  │ ──► │  2. HYDRATE  │ ──► │ 3. CHALLENGE │ ──► │  4. ACTIVE   │ ──► │ 5. RECYCLE   │
│  (Gen + 1)   │     │ (Signed R2)  │     │ (Synthetic)  │     │ (Leasing Bus)│     │ (Post-Flight)│
└──────────────┘     └──────────────┘     └──────────────┘     └──────────────┘     └──────────────┘

#Stage 1: Allocate & Generation Increment

The orchestrator calls a substrate driver (cf_sandbox, gcp_vm, k8s) to create an isolated compute boundary. Mupot increments generation_id in agent_seats and mints an ephemeral seat_bootstrap_nonce valid for 120 seconds.

#Stage 2: Hydrate from Signed Basin

The node bootloader calls POST /api/seat/hydrate presenting seat_bootstrap_nonce. Mupot returns a signed manifest URL for the immutable workspace archive in R2. The node verifies the HMAC-SHA256 checksum and extracts files with tarbomb/symlink defense. A session token is issued: sess_<agent>_<generation>_<random> (TTL 15 min, renewable while healthy).

#Stage 3: Synthetic Canary Challenge

Mupot dispatches a high-priority synthetic challenge ([canary:<uuid>]) to the seat’s inbox. The node responder leases the message, executes the agent model, and submits {canary_ack: <uuid>, generation: N}. Verification SLA: <15 seconds. Timeout or signature mismatch terminates the node as FAILED_CANARY.

#Stage 4: Active Routing & Dispatch

Seat status becomes ACTIVE in D1. Ingress enables flight dispatch and message delivery. The seat writes receipts with idempotency keys to runner_receipts.

#Stage 5: Flush & Recycle

At flight completion or TTL expiration, dirty memory deltas sync back to Mupot/R2 via signed API. The compute node is terminated or reset to a clean snapshot.

#5. Substrate Drivers

export interface MuMachineDriver {
  name: 'cloudflare_sandbox' | 'gcp_microvm' | 'k8s_seat' | 'aws_firecracker'
  allocateSeat(req: AllocateSeatRequest): Promise<SeatHandle>
  destroySeat(seatId: string): Promise<void>
  getHealth(seatId: string): Promise<SeatRawStatus>
}

1. Cloudflare Sandbox — default for stateless/ephemeral tasks, PR reviews, code execution, and fast tool runs.
2. GCP MicroVM — long-running heavy seats (local model servers, persistent daemons).
3. Kubernetes AgentSeat — enterprise private clusters with dedicated GPU/CPU pools.

#6. Nine Acceptance Gates

1. Ephemeral Token Expiry: Expired seat token returns 401 and triggers automated refresh.
2. Generation Fencing: Token from gen:N is rejected with STALE_GENERATION when gen:N+1 is active.
3. Split-Brain Prevention: Two concurrent nodes leasing the same queue fail closed to the highest generation.
4. Synthetic Canary Failure: A hanging model daemon fails the canary and never reaches ACTIVE.
5. Tamper-Evident Hydration: Invalid HMAC on the hydration archive aborts boot.
6. Tarbomb & Path Traversal: Malicious tarball with outbound symlinks is blocked.
7. Confused Deputy Isolation: A seat cannot read or write another squad or agent’s storage.
8. Zero-Egress Enforcement: Sandbox outbound traffic is limited to the Mupot MCP endpoint.
9. Receipt Idempotency: Re-executing a recorded step returns the existing runner_receipt without side effects.

#7. Next Implementation Slice

1. D1 agent_seats schema and generation-scoped token exchange in Mupot src/auth/seat-token.ts.
2. Vitest coverage of the nine gates in tests/mumachine-lifecycle.test.ts.