Your Linux sysadmin co-pilot. Plain language in. Typed plan out. You approve. Daemon executes.

What SysKnife does

You describe a task in plain language. SysKnife asks an LLM to turn it into a typed plan — a list of named actions with formal risk levels. You review the plan, approve it, and the daemon executes it step by step with live output and automatic rollback on failure.

$ sysknife "install neovim and make sure it starts on login"

Plan  (2 steps)
──────────────────────────────────────────────────────────
 1  AddLayeredPackage  neovim             risk: high
 2  SetServiceEnabled  neovim.service     risk: medium
──────────────────────────────────────────────────────────
Approve? [y/N]

The AI cannot run a shell command. It can only propose typed actions. The daemon is the only process that touches your system — and only after you say yes.

Why not just paste the shell command the AI gives you?

Because you find out what it did after. There is no record, no rollback, and no way to verify the AI proposed the minimal change rather than a sledgehammer.

SysKnife gives you:

• Typed actions — not shell strings. AddLayeredPackage neovim not rpm-ostree install neovim && ...
• Risk levels — Low (read-only), Medium (reversible), High (irreversible, access-control, or reboot-required)
• Preview before execution — see the exact commands before they run
• Automatic rollback — if a high-risk action fails, the daemon reverses what it can
• Immutable audit trail — every execution is Ed25519-signed and hash-chained

How it works

┌─────────────────┐   plan    ┌──────────────────┐  approve  ┌─────────────────┐
│  sysknife-brain │ ────────► │  sysknife-shell  │ ────────► │ sysknife-daemon │
│  (unprivileged) │           │  (approval gate) │           │  (root, locked) │
└─────────────────┘           └──────────────────┘           └─────────────────┘
   LLM + tools                   you review here               executes + audits

The brain proposes but cannot touch the system. The daemon executes but cannot be reached without an approved plan.

Fastest path: use via your AI coding tool (MCP)

MCP is how most people use SysKnife in 2026 — it's the dominant way AI tools talk to real systems, and Claude Code, Cursor, and Codex CLI all support it. One command wires it up:

npx sysknife-setup

The demo above shows the whole loop: plan in chat, approve in a terminal, execute with a one-time receipt. See the MCP Server guide for full setup and the approval-gate hook.

Prefer the terminal? The CLI is a first-class path

No IDE and no MCP client — the same planner, approval gate, and Ed25519 audit chain, driven straight from your shell. This is a fully supported way to run SysKnife, not an afterthought.

ℹ️ Distro support

All three Ubuntu LTS releases have a committed live-VM run of the 79-story Ubuntu suite, in tests/evidence/story-runs/: 22.04, 24.04 and 26.04 all at 79/79. Each run has a replay twin that reproduces it, serving every call with zero misses. Five Debian-only actions still have no story. Fedora Atomic is supported by the rpm-ostree action family, but a current Silverblue 44 VM run is a release gate. Plain Fedora remains experimental until the dnf action family ships. See Distro Support for the full matrix.

sudo apt-get install -y build-essential          # Rust stable and a C compiler
git clone https://github.com/lacs-project/sysknife
cd sysknife && make build && sudo make install   # ~7-12 min: about 400 crates
sudo systemctl enable --now sysknife-daemon
sysknife "show disk usage"

For prebuilt binaries instead of a build, run npx sysknife-setup (Node 18+). See the canonical Quick Start.

No API key needed if you have Ollama running locally — SysKnife auto-detects it. See Quick Start.

Also: a desktop GUI (a distant third option)

There is an experimental desktop GUI (sysknife-shell, built with Tauri) that wraps the same plan → approve → execute loop in a window. It is the least frequently maintained surface — a distant third behind the MCP integration and the CLI — so reach for it only if you specifically want a graphical approval flow.

Status

190 typed actions · 1,845 Rust tests + 72 frontend tests · MIT

SysKnife is the reference implementation of the LACS specification — a CC0 public-domain protocol for AI agents that operate at the Linux system level.

Quick Start

This is the canonical install page. Everywhere else links here rather than repeating a second, slightly different sequence.

💡 Prefer your AI coding tool?

If you use Claude Code, Cursor, or Codex CLI — run npx sysknife-setup and follow the wizard. See MCP Server for the full guide. You can skip this page entirely.

Step 1 — Install

Two paths. The wizard needs no toolchain and no compile; building from source needs both.

Fastest: prebuilt binaries

npx sysknife-setup

Prerequisites: Node 18 or newer. On Ubuntu 22.04, apt install nodejs installs Node 12, which is too old — the installer will tell you so and how to get a current Node. It downloads SHA-256-verified binaries from the release page, installs the daemon service, and writes your MCP client config.

Unattended runs must say which daemon to install, because the choice decides what will work:

npx sysknife-setup --claude --no-prompts --daemon-mode=system

--daemon-mode=user installs an unprivileged service that runs as you: read-only actions work, and anything mutating (installing packages, restarting services) does not, because the sudoers grants belong to the sysknife system user.

For system mode, follow the printed installation commands; the wizard does not install the system service itself. A skipped install or a host without systemd gets manual daemon steps instead of commands for a user unit that was never installed. With --no-prompts, supply any required API key through the environment (see Step 2); the wizard does not collect one interactively. Redirected downloads print periodic progress lines and a final byte count.

From source

Prerequisites: Rust stable (rustup update stable), a C compiler and linker, and an LLM provider (see Step 2). The TLS and SQLite dependencies build native code, so a machine with only rustup fails at error: linker cc not found. cmake is not required.

sudo apt-get install -y build-essential   # Debian/Ubuntu
git clone https://github.com/lacs-project/sysknife
cd sysknife
make build
sudo make install
sudo systemctl enable --now sysknife-daemon

make build compiles around 400 crates: expect 7 to 12 minutes on a first build (measured 6m56s on Ubuntu 24.04, 11m43s on 22.04).

ℹ️ Fedora / Silverblue

Ubuntu 24.04 is validated by the live-VM story suite. Ubuntu 22.04 and 26.04 have smoke-test coverage. Fedora Atomic uses the rpm-ostree action family and requires a current Silverblue 44 validation run for each release. Plain Fedora remains experimental. See distro support.

Step 2 — Choose an LLM

Pick one. No account needed for Ollama.

Ollama — local, fully offline, recommended for homelabs:

ollama pull qwen3:8b        # runs well on 16 GB RAM
# SysKnife auto-detects Ollama when no cloud key is set

Anthropic:

export ANTHROPIC_API_KEY=sk-ant-...

OpenAI / Gemini / others — see Configuration for the full list of supported providers.

Optional config file (~/.config/sysknife/config.toml):

[llm]
provider = "ollama"
model    = "qwen3:8b"

Step 3 — Run

# Safe first run — plan only, nothing executes
sysknife --dry-run "show disk usage"

# Full run with the daemon
sysknife "what packages do I have installed as layers?"

⚠️ Daemon required for execution

--dry-run works anywhere and is a great way to test the planner without installing the daemon. Full execution requires sysknife-daemon running as root (enabled in Step 1).

That's it. The planner proposes a typed plan, you approve, the daemon executes.

Try the planner without the daemon

On any Linux machine with an API key:

export ANTHROPIC_API_KEY=sk-ant-...
cargo run --bin sysknife -- --dry-run "show disk usage"

Plans the intent and prints the result. No daemon and no root: nothing privileged runs, so this is the way to evaluate the planner or run it in CI.

It is not, however, free of installation — cargo run builds the workspace, so it needs Rust plus build-essential and the same 7 to 12 minutes as any other first build. If what you want is the quickest look at SysKnife, use npx sysknife-setup and the prebuilt binaries instead.

What to read next

• CLI Reference — all flags, subcommands, and output formats
• MCP Server — use SysKnife from Claude Code, Cursor, Codex CLI
• Configuration — full provider and storage options
• Distro Support — what works on which distributions

SysKnife MCP Server

The sysknife mcp-server subcommand exposes five fixed MCP tools plus direct read-only query tools selected for the detected distro. MCP-capable AI assistants (Claude Code, Cursor, Codex CLI, …) can inspect live state without an LLM planning round trip, while every mutation still uses SysKnife's approval-gated, audit-logged path.

💡 One command to configure everything

Run npx sysknife-setup — it detects which AI clients you have installed and writes the correct config files for each one automatically.

Found SysKnife on a directory page where the tools error out? Discovery works anywhere: a directory sandbox lists the five fixed tools plus only the cross-distro direct queries when distro detection is unavailable. Calling them needs a sysknife-daemon on a real host, which a build container has no way to provide, so doctor reports the socket absent and the rest fail. That is the trust boundary, not a packaging defect. See Registry and Directory Listings.

Tools

sysknife_plan

Turn a natural-language intent into a risk-labelled plan. No action is executed.

Input

Output — PlanOutput

Each PlanStep:

sysknife_execute

Execute a plan produced by sysknife_plan. Each step must include the one-time receipt printed by sysknife approve <transaction-id>.

Input

Each StepToExecute contains the original transaction_id, action_name, and params, plus an approval_receipt. Execution halts on the first failure.

Output — ExecuteOutput

Each StepResult:

sysknife_history

List audit-log entries with optional status, action, since, and limit filters. This tool is read-only and does not require a plan or receipt.

sysknife_doctor

Check daemon connectivity, active brain configuration, audit storage, and a quick audit-chain status. This tool is read-only.

sysknife_audit_verify

Verify the Ed25519-signed audit chain and report intact, broken, or cannot_verify, including the first offending row when verification fails. This tool is read-only.

Direct read-only action tools

Every explicitly classified read-only catalogue action is exposed as sysknife_<snake_case_action>, for example:

• GetDiskUsage → sysknife_get_disk_usage
• GetServiceStatus → sysknife_get_service_status
• AptSearch → sysknife_apt_search on Ubuntu-family hosts
• GetSystemState → sysknife_get_system_state on Fedora-family hosts

Tool arguments are the action parameters described in each tool's generated description. Calls go through the daemon's query_action path, which preserves the Observer-role and distro fences and independently rejects every action in the shared Observer-mutating classification. The tool annotations declare the routes read-only and non-destructive.

The allowlist is deliberately not equivalent to RiskLevel::Low. AptUpdate runs sudo apt-get update; it is Low/Observer-callable but mutating, so it is never registered as a direct tool and the daemon refuses raw query_action requests for it. A drift test compares the explicit read-only and mutating classifications with the live catalogue, forcing every future Observer action to be reviewed before the surface can change.

The Approval Workflow

The assistant must always follow this order — no exceptions:

1. sysknife_plan { intent }
        ↓
   Present the plan (steps + risk levels) to the user
        ↓
2. STOP and wait for the user
        ↓
3. User runs: sysknife approve <transaction-id>
   Repeat for each accepted step and return the printed receipt
        ↓
4. sysknife_execute { steps with approval_receipt }
        ↓
   Report results

Never call sysknife_execute without a receipt minted by the separate CLI command. A chat response such as "yes" is not an approval receipt. Receipts are bound to one immutable preview, expire after 15 minutes, and are consumed atomically on first use.

The daemon enforces this protocol. Prompt hooks provide guidance but are not the security boundary.

Setup

1. Run the setup wizard

npx sysknife-setup

Needs Node 18 or newer (Ubuntu 22.04's apt Node is 12, which is too old). The wizard detects your installed sysknife binary, asks for the daemon socket and LLM provider, and then asks which integration to configure. No manual file editing needed. For install options and prerequisites in full, see the canonical Quick Start.

If you already know the target, skip the picker:

npx sysknife-setup --claude
npx sysknife-setup --cursor
npx sysknife-setup --codex

.mcp.json is gitignored — it contains secrets and local paths.

2. Connect to a daemon in a VM

If the daemon runs inside a VM, two transports are available:

SSH socket tunnel (works with any hypervisor):

ssh -fN -L /tmp/sysknife-vm.sock:/run/sysknife/daemon.sock \
    <user>@<vm-host>

Set SYSKNIFE_SOCKET=/tmp/sysknife-vm.sock when the setup wizard asks for the socket path.

virtio-vsock (KVM/QEMU only, no SSH required):

# Find the guest CID from the host
virsh dumpxml <vm-name> | grep cid

The daemon must be told to listen on vsock as well — it binds a Unix socket by default, so configuring only the client leaves the two on different transports and nothing connects. On the VM:

sudo systemctl edit sysknife-daemon
# [Service]
# Environment="SYSKNIFE_LISTEN_URI=vsock://:9734"
sudo systemctl restart sysknife-daemon

Then, on the host, set SYSKNIFE_SOCKET=vsock://<CID>:9734 and SYSKNIFE_TOKEN=<hex> when the wizard asks. The wizard detects the vsock:// prefix and prompts for the token automatically.

See VM + Daemon Setup for the complete walkthrough including token generation, libvirt XML, and troubleshooting.

3. Manual configuration

If you prefer to edit .mcp.json by hand:

{
  "mcpServers": {
    "sysknife": {
      "command": "/path/to/sysknife",
      "args": ["mcp-server"],
      "env": {
        "SYSKNIFE_SOCKET": "/run/sysknife/daemon.sock",
        "SYSKNIFE_LLM_PROVIDER": "openai",
        "OPENAI_API_KEY": "<your-api-key>",
        "SYSKNIFE_LLM_MODEL": "gpt-4.1"
      }
    }
  }
}

For vsock add SYSKNIFE_TOKEN alongside SYSKNIFE_SOCKET.

4. If you skipped the prebuilt download: build the binary

Not a step in the sequence above — an alternative to the wizard's download, for building from source. Needs a C compiler (sudo apt-get install -y build-essential on Ubuntu); cmake is not required.

cargo build -p sysknife-cli --release
# binary at target/release/sysknife

5. Reload the MCP server in your client

In Claude Code: run /reload-plugins.

Example Session

User:    check disk usage on the VM

Claude:  [calls sysknife_plan { intent: "check disk usage" }]

         Plan: Check disk usage on all filesystems
         Steps:
           ● low  GetDiskUsage — Retrieve current disk usage

         Execute?

User:    [runs sysknife approve 018f2c9d-... in a terminal]
         Approval receipt: 5f493f80-...

Claude:  [calls sysknife_execute with transaction_id and approval_receipt]

         GetDiskUsage ✓
         Filesystem     Size  Used Avail Use%  Mounted on
         /dev/vda3       38G   18G   19G  49%  /var
         ...

Risk Levels

Risk changes how prominently the plan should be reviewed; it never replaces the receipt requirement. The standalone CLI additionally uses stronger confirmation prompts for high-risk actions.

MCP Registry listing

SysKnife's MCP server is the sysknife mcp-server subcommand (stdio). This note records how it maps onto the official MCP Registry and the exact steps to publish a listing.

The official registry is in preview — it may reset data before general availability, so treat a published entry as non-permanent for now.

Distribution: the cargo package type

The registry supports a cargo package type: registryType is an open string (the schema's enumerated examples are only npm/pypi/oci/nuget/mcpb, so cargo is permitted rather than listed), and the registry validator (internal/validators/registries/cargo.go) implements it. That lets us list the crate directly:

• registryType: cargo, identifier: sysknife-cli (the crate that installs the sysknife binary), transport: stdio.
• Because sysknife's MCP server is a subcommand, the package entry passes mcp-server as a positional packageArguments value. A client resolves the listing to cargo install sysknife-cli then runs sysknife mcp-server.
• Two things that install implies, and which the registry entry has no field for: it needs a C compiler (build-essential; not cmake, verified in clean Ubuntu containers) and takes 7 to 12 minutes; and the CLI alone can plan but not execute, because execution belongs to the privileged sysknife-daemon. Both are stated up front in apps/sysknife-cli/README.md, which is the page crates.io renders and therefore what a registry visitor actually reads.

This supersedes the earlier npm-launcher plan: the npm package sysknife-setup is an installer/wizard, not a stdio server, and npx sysknife-setup would launch the wizard rather than the server. Worse than merely wrong, the wizard reads answers from stdin when stdin is not a TTY, so it would consume a client's initialize frame as a prompt answer. The cargo type avoids that entirely: no dedicated launcher package is needed.

The root server.json is accepted by mcp-publisher validate, which checks against the live registry rather than only the local schema.

Namespace: io.github.lacs-project/sysknife (verified by GitHub identity — the authenticating account must belong to the lacs-project org; no DNS needed).

Ownership marker (already in place)

crates.io ownership is proven by a visible mcp-name: token in the crate's rendered README. crates.io strips HTML comments when rendering, so the marker must be plain text — apps/sysknife-cli/README.md carries:

mcp-name: io.github.lacs-project/sysknife

The verifier fetches https://crates.io/api/v1/crates/sysknife-cli/<version>/readme and searches the rendered README for that token. The marker only takes effect in a published crate version, so the server.json version must match a crate version whose README carries it (see the release step below).

server.json

Shipped at the repository root. The version is coupled to a published crate version that carries the marker, and two checks keep that coupling honest:

• scripts/check_release_versions.sh includes both server.json version fields in the release-wide version comparison, so a release cannot bump the crates and leave the listing pointing at the previous version.
• tests/release/registry-manifest.test.sh asserts the rest of the manifest: registryType/registryBaseUrl/transport, that the identifier is the CLI crate, that the mcp-server positional argument is present, and that the ownership marker is in the crate README outside an HTML comment. It runs in CI and in the release preflight.

Both mean a stale or incoherent listing fails a check rather than a publish.

Publish steps

The crate README marker (apps/sysknife-cli/README.md) is in the repo and has shipped in every published crate since 0.2.6, so any current version is registry-ready. To publish the listing:

1. Confirm the marker is live in the version server.json names. The validator reads the rendered README from crates.io, in two calls:

   # Read the version from the manifest rather than retyping it: checking the
   # marker for a version other than the one being published proves nothing.
   version="$(node -p 'require("./server.json").version')"
   ua='sysknife-release/1.0 (https://github.com/lacs-project/sysknife)'
   url="$(curl -sS -H 'Accept: application/json' -H "User-Agent: $ua" \
     "https://crates.io/api/v1/crates/sysknife-cli/${version}/readme" \
     | node -p 'JSON.parse(require("fs").readFileSync(0,"utf8")).url')"
   curl -sS -H "User-Agent: $ua" "$url" \
     | grep -c 'mcp-name: io.github.lacs-project/sysknife'
   

   A count of 1 means the marker is visible to the validator. A 0 means the listing cannot be published for that version, whatever the repo says.
2. Install the publisher CLI:

   brew install mcp-publisher   # or download from the registry's GitHub Releases
   

3. Authenticate as the lacs-project org. In practice there is one option, not two: see Authentication: namespace comes from the identity below. Inside a GitHub Actions job with permissions: id-token: write (the ./ reflects the binary downloaded into the job's working directory rather than a PATH install):

   ./mcp-publisher login github-oidc
   

   The publish-registry job in .github/workflows/release.yml does this on every tag, and .github/workflows/publish-mcp.yml can be dispatched by hand for a retry or a backfill.
4. Validate and publish (from the repo root):

   mcp-publisher validate   # checks server.json against the live registry
   mcp-publisher publish
   

   validate needs no authentication, so it is worth running before the login step. Only publish requires the token.
5. Verify:

   curl "https://registry.modelcontextprotocol.io/v0/servers?search=sysknife"
   

   The count in the response metadata goes from 0 to 1.

Authentication: namespace comes from the identity

server.json publishes io.github.lacs-project/sysknife, and the registry mints publishing permissions from whoever authenticated, not from what the manifest asks for. That distinction is the whole reason this cannot be done from a laptop.

mcp-publisher login github authenticates the user. The registry token it returns carries a single permission:

{"action": "publish", "resource": "io.github.vladimirrott/*"}

so publish fails with 403 Forbidden ... You have permission to publish: io.github.vladimirrott/*. Attempting to publish: io.github.lacs-project/sysknife. Making org membership public does not fix it, and neither does being an org admin: the device-flow OAuth grant does not carry read:org, so the registry never sees the membership at all. Verified 2026-07-30 on two consecutive fresh logins with membership already public.

mcp-publisher login github-oidc authenticates the repository. The namespace is derived from the repository owner, so it matches server.json by construction, and the credential is permissions: id-token: write rather than a stored secret.

The practical consequence: a release published from a laptop can reach crates.io, npm and GitHub Releases but cannot reach the MCP Registry. Publish the registry listing from Actions.

Downstream propagation

One publish to the official registry auto-propagates to the GitHub MCP Registry and PulseMCP, which ingest from it. Everything else takes a separate submission. Verified by walking each flow on 2026-07-29:

What a directory sandbox can and cannot tell you

Most directories work the same way: build a container from the repository, boot the server inside it, introspect the tool list, and score what they find. That model assumes a self-contained server, one that reaches an API over the network or reads files inside its own sandbox. SysKnife is not that, by design, and it pays to be precise about which half of the model still applies.

sysknife mcp-server is the unprivileged half of the system. It plans, previews, renders, and forwards. It cannot change anything. Every mutation travels over a unix socket to sysknife-daemon, which holds the sudoers, polkit and helper policy that sudo make install owns, and the socket itself is 0750 sysknife:sysknife at /run/sysknife/daemon.sock. That split is the product: it is why an agent cannot hand a shell string to root, and why every executed action lands in the signed chain. See Architecture & Trust Boundaries.

So the split is between describing and doing:

1. Discovery works, and should. initialize and tools/list are answered from the binary's own definitions, so a sandbox sees the five fixed tools and the direct read-only queries compatible with the distro it detects (only cross-distro queries if detection fails), all with schemas.
2. Every tool that needs the daemon fails there, and should. sysknife_plan, sysknife_execute, sysknife_history and sysknife_doctor all reach sysknife-daemon over a unix socket at /run/sysknife/daemon.sock; the generated direct queries use the same socket and daemon authorization fence. The socket is (0750 sysknife:sysknife), which holds the sudoers, polkit and helper policy sudo make install owns. In a container there is nothing listening, so doctor reports the socket absent and the rest return errors. That is the trust boundary doing its job, not a packaging defect. See Architecture & Trust Boundaries.
3. stdio needs a wrapper. Container-based hosts front the binary with a proxy (Glama uses mcp-proxy --). Strip that prefix while editing a build spec and the listing stops working.
4. A green sandbox run still proves little. "It booted and listed tools" says the binary starts and its schemas parse. It says nothing about whether SysKnife administers a real Ubuntu host correctly, because the only thing it could have administered was a throwaway container.

The practical guidance: treat these listings as discovery surface, not as validation. They are cheap inbound links from where people look for MCP servers. The evidence that actually bears on correctness lives elsewhere: the VM runs recorded in Distro Support, the story suite, and an audit chain a third party can verify with only the public key.

And the inverse, which matters more: do not relax the daemon boundary to score better in a sandbox. Enumerating tools there is free and already happens; being able to execute from inside an unprivileged container is the thing that would score higher and make SysKnife not worth installing.

sysknife CLI Reference

sysknife is the command-line interface to the SysKnife daemon. It turns a natural-language intent into a risk-labelled plan, asks for approval where needed, and streams execution output in real time.

If you want SysKnife inside Claude Code / Cursor / Codex CLI instead, see the main README and run npx sysknife-setup. Both paths share the daemon, the audit chain, and the typed-action set.

Quick start

# Check that the daemon is reachable
sysknife doctor

# Plan + execute a single intent
sysknife "check disk usage"

# Preview the plan without executing
sysknife --dry-run "list running containers"

# Open the interactive REPL
sysknife

Synopsis

sysknife [GLOBAL FLAGS] [SUBCOMMAND | INTENT WORDS...]

When no subcommand is given and no intent words are provided, sysknife starts an interactive REPL.

Subcommands

sysknife <intent>

Plan and (optionally) execute a natural-language intent.

sysknife "check disk usage"
sysknife check disk usage            # words are joined — same result
sysknife "list running containers"
sysknife "is firewalld active?"
sysknife "layer vim via rpm-ostree"

What happens:

1. A spinner appears while the LLM plans (Thinking… → Querying … → Proposing plan…).
2. The coloured plan is printed — each step shows a risk badge (● low / ● medium / ● HIGH), the action name, and a summary.
3. If any step requires approval, you are prompted. HIGH-risk steps always require confirmation regardless of --yes.
4. Execution streams output line by line with a › prefix; a ✓ / ✗ result icon is printed after each step.

sysknife doctor

Check daemon connectivity and print the resolved configuration.

sysknife doctor
sysknife --json doctor      # machine-readable

Exit code 0 on success, non-zero if the daemon is unreachable.

Sample output:

✓  daemon ok
  socket    /run/sysknife/daemon.sock
  host      my-silverblue
  provider  anthropic
  model     claude-sonnet-4-6

sysknife history

Query past SysKnife execution history.

sysknife history
sysknife history --limit 50
sysknife history --status failed
sysknife history --action InstallPackages
sysknife history --since 2026-04-01T00:00:00Z
sysknife history --status succeeded --limit 5 --since 2026-04-10T00:00:00Z

Flags:

sysknife approve

Issue a one-time receipt for a transaction returned by the MCP sysknife_plan tool. This command requires an interactive terminal. It first loads and displays the daemon-authoritative action, risk, summary, and proposed change so an agent cannot substitute an opaque transaction ID. It mints the receipt only after confirmation; high-risk approvals require typing the exact action name.

sysknife approve 018f2c9d-...
sysknife --json approve 018f2c9d-...

Give the printed approval_receipt to the MCP client for that exact step. The receipt expires after 15 minutes, is bound to the preview's action and params, and is consumed on first execution. A chat message saying "approved" is not a receipt.

sysknife audit

Inspect and anchor the tamper-evident, Ed25519-signed audit chain the daemon writes for every executed action.

sysknife audit export

Export the transaction-chain rows from the configured SQLite or PostgreSQL audit store as a JSON array, in ascending seq order. Each object contains the 17 stored ChainRow columns, including prev_chain_hash and chain_hash. The latter is the Ed25519 signature and is deliberately not renamed. Optional values that were not recorded are JSON null; argv, outcome, and a separate signature field are not reconstructed.

sysknife audit export
sysknife audit export --since 2026-08-01T00:00:00Z --limit 500

An export is not a redacted artifact. It inherits the confidentiality class of the audit database, which the daemon keeps 0600 inside a 0700 directory, and writing one moves that content across the boundary those modes exist to hold.

Every row carries request_hash, a single unsalted SHA-256 over the action name and the unredacted parameters. compute_request_hash runs before redact_params, so redaction never reaches the hashed preimage. Where an action's parameters are low entropy and partly public the hash is worth attacking: for ConfigureWifi the SSID is broadcast, which leaves the passphrase as the only unknown. High-entropy values such as ProAttach tokens are unaffected.

The column cannot be dropped, because request_hash is part of the signed bytes an offline verifier rebuilds. Treat an export with the same care as the database itself rather than as a sanitised report.

sysknife audit verify

Verify the audit trail: the transaction chain, the approval-event chain, and the binding between them. All three are reported and any one can fail the command. Exits 0 if everything is intact, 1 if any check finds tampering, 2 if a check cannot run at all (missing key, unreadable database). When the checks disagree the worst wins, and 1 outranks 2 — if something is provably broken, "could not verify" would understate it.

With --json the report is an object with a top-level status plus a chain, approval_events and binding section, so a pipeline can act on which part failed.

sysknife audit verify
sysknife audit verify --json
sysknife audit verify --pubkey /etc/sysknife/audit-key.pub

sysknife audit checkpoint

Sign the current chain tip as a checkpoint and anchor it to an external append-only database, then verify all anchored checkpoints against the local chain. Anchoring the tip off-box is what makes tail-truncation and rewrite of the local chain detectable.

# credentials via env (preferred; keeps them off the command line)
SYSKNIFE_CHECKPOINT_DB=postgres://user@host/db sysknife audit checkpoint
# or explicitly
sysknife audit checkpoint --db postgres://user@host/db

Each row is signed with Ed25519; verification uses the public key, so an auditor can verify without the ability to forge. See configuration for the key and checkpoint-DB env vars.

sysknife completions <shell>

Print a shell completion script to stdout.

sysknife completions bash   >> ~/.bashrc
sysknife completions zsh    >> ~/.zshrc
sysknife completions fish   >> ~/.config/fish/completions/sysknife.fish

Supported shells: bash, zsh, fish, elvish, powershell.

sysknife mcp-server

Start an MCP (Model Context Protocol) server over stdio, so Claude Code, Claude Desktop, Cursor, Codex, and any other MCP-capable agent can drive SysKnife.

sysknife mcp-server

It exposes five fixed tools backed by the SysKnife daemon: sysknife_plan, sysknife_execute, sysknife_history, sysknife_doctor, and sysknife_audit_verify. It also generates direct read-only query tools for the detected distro, such as sysknife_get_disk_usage. Planning and execution stay behind the same approval interlock as the CLI — sysknife_plan returns typed steps with daemon-issued transaction IDs, and each step still requires an explicit sysknife approve <transaction-id> in a real terminal before it can run. Low risk is not treated as read-only: AptUpdate remains plan-only.

Register it with your agent by pointing it at the binary, e.g. in claude_desktop_config.json:

{ "mcpServers": { "sysknife": { "command": "sysknife", "args": ["mcp-server"] } } }

npx sysknife-setup writes this configuration for the common clients automatically.

REPL (no arguments)

sysknife

Starts an interactive session. Each line is treated as a natural-language intent and planned + executed in sequence.

Key bindings:

History is persisted to ~/.local/share/sysknife/history between sessions.

Global flags

All flags apply to every subcommand and to free-form intents.

Unattended mode

--dangerously-skip-approval lets a plan written by a language model execute HIGH-risk actions with nobody confirming them. It exists for scheduled and CI-driven runs, and it is the only way to lift the rule that --yes can never auto-approve HIGH.

Two keys, on purpose

The flag alone does nothing but print an explanation and exit 1. It needs the environment variable as well:

SYSKNIFE_I_ACCEPT_UNATTENDED_ROOT=1 \
  sysknife --dangerously-skip-approval --json "apply pending security updates"

Neither half is enough alone. A flag left in a script and a variable left in a shell profile are the two ways this gets armed by accident, and requiring both means neither accident is sufficient. Only the exact value 1 counts; true, yes and 0 are all read as unset.

The flag has no short form and no abbreviation. Typing it has to be a decision.

What it turns off

One thing: the approval gate.

• --yes may now auto-approve HIGH-risk steps. The cap moves from MEDIUM to HIGH.
• The post-preview confirmation on a HIGH step no longer asks. The preview is still fetched and still printed, because it is the only record of what the run was about to change.

An explicit --max-risk still wins. --dangerously-skip-approval --max-risk low aborts on a MEDIUM step, because the flag removes the ceiling the project imposes, not the one you asked for. --dry-run still executes nothing.

What it does not turn off

Everything that makes SysKnife more than a shell:

• Only actions in the typed catalogue can run, with validated parameters. There is no path from this flag to an arbitrary command.
• The polkit allowlist still gates every privileged D-Bus call.
• The run still aborts if the daemon rates a step above what the CLI approved.
• Role-based authorization is unchanged. An account that cannot run an action still cannot run it.
• Every step is still previewed, still gets a transaction row, and is still signed into the audit chain.

If you want a tool that will run any command an LLM writes, this is not it, and no flag here turns it into one.

The audit trail records it

A transaction previewed in this mode carries this sentence in its warnings:

Approved without a human: this step was previewed by a client running with --dangerously-skip-approval, so no operator confirmed it.

warnings is stored as warnings_json, and warnings_json is one of the fields inside the row's Ed25519 signature. So the record is not a log line that can be edited later: deleting it makes sysknife audit verify report that row Broken. sysknife audit export carries it like any other field.

The CLI checks that the daemon sent the sentence back, and refuses to execute if it did not. A daemon older than this field accepts the declaration and drops it, which would leave an unattended run indistinguishable from an approved one. Upgrade the daemon, or drop the flag.

Before you use it

Run it against a machine you can rebuild. The failure mode is not a bad diff to revert; it is a system change applied by a model with no one watching. A VM snapshot beforehand costs less than the alternative.

Exit codes

Subcommands with their own semantics (for example sysknife audit verify) pass through their own exit code.

Clap returns 2 for malformed command-line arguments, such as an invalid --max-risk value, before planning or execution starts. --timeout also returns 2 and reports operation timed out after Ns; it applies to the whole command and can expire during planning, execution, or another subcommand. Inspect the diagnostic to distinguish these cases: exit 2 alone does not mean an action ran. The timeout diagnostic does not identify which phase timed out. A CLI timeout stops waiting; it does not guarantee cancellation of work already submitted to the daemon.

Environment variables

LLM provider

sysknife auto-detects between Anthropic and local Ollama from the presence of ANTHROPIC_API_KEY. Every other provider must be selected explicitly with SYSKNIFE_LLM_PROVIDER (and its matching API key set).

Auto-detection (when SYSKNIFE_LLM_PROVIDER is not set):

1. ANTHROPIC_API_KEY present and non-empty → anthropic
2. Otherwise → ollama (must be running locally)

The other providers (openai, gemini, groq, deepseek, mistral, xai) are not auto-detected from their API-key variables. To use one, set SYSKNIFE_LLM_PROVIDER to its name and provide the matching key from the table above.

Daemon socket

Unattended-mode consent

See Unattended mode for what the flag does and does not turn off.

Scripting and CI

For non-interactive use (scripts, CI pipelines), combine --json, --non-interactive, and --max-risk:

# Plan only — parse the JSON to inspect before executing
PLAN=$(sysknife --dry-run --json "check disk usage")
echo "$PLAN" | jq '.plan.steps[].action'

# Execute automatically up to medium risk; fail if anything higher appears
sysknife --yes --max-risk medium --non-interactive "list layered packages"

# Full pipeline with a timeout and log
sysknife --yes --max-risk low --non-interactive --timeout 60 \
     --log-to /var/log/sysknife/run.log \
     "check disk usage"

# Unattended, including HIGH-risk steps. Both keys are required, and every
# step is recorded in the signed chain as having had no human approval.
SYSKNIFE_I_ACCEPT_UNATTENDED_ROOT=1 \
  sysknife --dangerously-skip-approval --json --timeout 300 \
     "apply pending security updates"

The --json output schema:

// Planning output
{ "plan": { "intent": "…", "summary": "…", "steps": [
    { "action": "GetDiskUsage", "summary": "…", "risk": "low", "params": {} }
] } }

// Per-step preview (before execution)
{ "summary": "…", "risk_level": "low", "reboot_required": false,
  "warnings": [], "request_hash": "…", … }

// Per-step result (after execution)
{ "status": "succeeded", "summary": "…", "job_id": "…",
  "needs_reboot": false, "warnings": [], … }

Examples

# Check if any services are failing
sysknife "which systemd services are failed?"

# See recent SysKnife activity
sysknife history --limit 10

# Dry-run a destructive action to inspect the plan
sysknife --dry-run "layer vim via rpm-ostree"

# Execute step-by-step with manual approval of each action
sysknife --step-by-step "update system"

# Non-interactive: fail fast if the plan needs a human
sysknife --non-interactive --max-risk low "check memory pressure"

# Get JSON output and parse with jq
sysknife --dry-run --json "list containers" | jq '.plan.steps[].action'

# Override the LLM for a single run
SYSKNIFE_LLM_PROVIDER=openai OPENAI_API_KEY=sk-... sysknife "check disk usage"

# Use a local Ollama model
SYSKNIFE_LLM_PROVIDER=ollama SYSKNIFE_LLM_MODEL=llama3.2:3b sysknife "list services"

Shell completion setup

Run once per shell:

# bash (add to ~/.bashrc)
eval "$(sysknife completions bash)"

# zsh (add to ~/.zshrc)
eval "$(sysknife completions zsh)"

# fish
sysknife completions fish | source

Related

• Architecture overview — trust boundary between CLI, shell, and daemon
• Developer guide — building and testing locally
• User stories — end-to-end scenario descriptions

Configuration reference

SysKnife reads configuration from three places, in lowest-to-highest priority:

1. Built-in defaults — compiled into sysknife-brain and sysknife-core
2. ~/.config/sysknife/config.toml — optional, user-owned
3. Environment variables — always win

Set whatever's stable for your install in config.toml; override with env vars when you need to (CI runs, ad-hoc experiments, distro packagers).

config.toml reference

Path: $XDG_CONFIG_HOME/sysknife/config.toml, falling back to ~/.config/sysknife/config.toml. The daemon, the CLI and the MCP server (sysknife mcp-server, which shares the CLI's entry point) read it at startup.

Values in the file are applied as defaults: anything already set in the environment wins, so a one-off SYSKNIFE_SOCKET=… sysknife doctor still overrides the file. The load happens before the async runtime starts, because applying it sets environment variables and that is only sound while the process is single-threaded.

The system daemon is configured by its unit, not by this file. Environment= lines in packaging/sysknife-daemon.service reach only the daemon's own process, which is why sysknife audit verify resolves the system audit store explicitly rather than expecting to inherit SYSKNIFE_DATABASE_PATH.

# ─── [daemon] ────────────────────────────────────────────────────────
[daemon]
# Unix socket path the daemon listens on. CLI / shell connect here.
socket   = "/run/sysknife/daemon.sock"
# SQLite database path for the local audit log.
# Production deployments should switch to [storage] backend = "postgres"
# (see below).
database = "/var/lib/sysknife/daemon.sqlite"

# ─── [llm] ───────────────────────────────────────────────────────────
[llm]
provider     = "ollama"          # ollama | anthropic | openai | gemini |
                                 # groq | deepseek | mistral | xai
model        = "qwen3:8b"        # provider-specific model identifier
ollama_url   = "http://localhost:11434"
anthropic_url = "https://api.anthropic.com"
max_turns    = 10                # planning loop turn limit (>= 1)

# Optional: override the auto-detected thinking mode for Ollama.
# Default: auto-detect from the model name (qwen3 / qwq / deepseek-r → true).
# Set to `false` on CPU-only hosts running thinking models — thinking
# traces exceed Ollama's internal request timeout on 4 vCPUs.
# ollama_think = false

# ─── [storage] ───────────────────────────────────────────────────────
# Audit-log backend. Default (absent or backend = "sqlite") uses the
# local rusqlite store at [daemon].database. Production deployments
# should set backend = "postgres" — the SQLite path dies with the host.
[storage]
backend = "postgres"
url     = "postgres://sysknife:${PG_PASSWORD}@db.example.com:5432/audit?sslmode=verify-full"

[storage.pool]
max_connections          = 8
acquire_timeout_secs     = 10
statement_cache_capacity = 100   # set to 0 for transaction-mode poolers

# ─── [policy] ────────────────────────────────────────────────────────
# Per-action risk-level overrides. Map from action name → risk level
# ("Low" | "Medium" | "High"). Validated at startup; unknown action
# names or attempted downgrades are fatal — overrides may only RAISE
# the minimum role required, never lower it.
[policy.risk_overrides]
InstallFlatpak = "High"     # require Admin in this org (default: Medium/Dev)

# ─── [audit.forward] ─────────────────────────────────────────────────
# Optional SIEM forwarding. Best-effort — never blocks daemon execution.
# Phase 1 ships RFC 5424 syslog over UDP; CEF and NDJSON-over-TCP
# arrive in follow-up PRs.
[audit.forward.syslog]
host     = "siem.internal:514"
facility = 1                 # 1 = user-level (default)

The transaction database is the durable audit record. Safety-fence JSONL, journald watermarks, and UDP syslog forwarding are operational signals and may be unavailable or lossy. PostgreSQL migrations run automatically at daemon startup; backup and restore requirements are documented in storage-cloud.md.

Environment variables

Env vars always win over config.toml. Useful for CI runs, distro packagers, and ad-hoc experiments. Variable names mirror the config file's section.field path.

Provider API keys

Required when the corresponding provider is selected:

• OPENAI_API_KEY — OpenAI
• ANTHROPIC_API_KEY — Anthropic
• GEMINI_API_KEY — Gemini
• GROQ_API_KEY — Groq
• DEEPSEEK_API_KEY — DeepSeek
• MISTRAL_API_KEY — Mistral
• XAI_API_KEY — xAI
• none — Ollama (local, no key)

Daemon-only configuration

These environment variables are read only by sysknife-daemon, not by the CLI / shell:

Validating your config

sysknife doctor

Reports the resolved configuration (socket, host, provider, model, audit backend) plus a quick chain-integrity check. A failing doctor is the fastest way to catch a typo'd env var or a bad path.

Where each setting lives in the source

For maintainers — the canonical types are:

• crates/sysknife-core/src/config.rs — LacsConfig, DaemonSection, LlmSection, PolicySection, AuditSection, StorageSection, StoragePoolSection
• crates/sysknife-brain/src/config.rs — BrainConfig, ProviderConfig
• crates/sysknife-core/src/lib.rs — default_listen_uri, default_database_path, prefs_path

Adding a new config knob: add the field to LacsConfig, surface the env var in apply_defaults_to_env, and update this document. A test in crates/sysknife-core/src/config.rs should cover the new field.

Distro support matrix

Ubuntu is the primary platform: every release from 20.04 onward, LTS and interim alike. That is a focus decision, not a judgement about other families: Ubuntu simply has far more users. Fedora Atomic remains a real target. Its rpm-ostree, Flatpak and toolbox action families are implemented and mutations are allowed from Fedora Atomic 41 onward; what it lacks is a current live-VM validation run, which is why it sits at a lower evidence tier below.

SysKnife reports operating-system support by evidence, not by family name alone. Recognition in /etc/os-release means the planner can select the right action vocabulary; it does not prove that every action has passed on that release.

Eligibility is not validation

Two separate things get called "support", and conflating them is what produced the bug this section now documents:

• Eligibility — will the daemon act on this host at all? That is DistroId::is_supported() in crates/sysknife-core/src/distro.rs, and the daemon refuses every mutating action when it is false. It is true for all Ubuntu releases from 20.04 up, and for Fedora Atomic 41 and later.
• Validation tier — has the full story suite been run on that release? That is the table below, and it will always be narrower than eligibility.

Eligibility was previously pinned to three LTS releases, so Ubuntu 20.04 and every interim release (25.10, 26.10, …) could plan but not execute: users on a supported OS were told their host was unsupported. Releases below 20.04 stay ineligible because they no longer receive Ubuntu security updates, and Ubuntu Core stays ineligible for a structural reason rather than an age one — it has no apt and a read-only root, so the Debian-family action set cannot apply.

Status definitions

Launch matrix

Fedora Atomic cannot be installed yet

A live run on Fedora 43 Silverblue on 2026-08-24 provisioned the guest and then stopped:

install -Dm 755 packaging/sysknife-apt-pin-edit /usr/lib/sysknife/apt-pin-edit
install: cannot create directory '/usr/lib/sysknife': Read-only file system

HELPERS defaults to /usr/lib/sysknife, and rpm-ostree mounts /usr read-only. The twelve privileged helper scripts have nowhere to go, and the path is not a free choice: twelve daemon constants and twelve NOPASSWD sudoers grants name it. #301 carries the options and discussion #307 is where the shape is being argued.

make install no longer discovers this halfway. daemon-install-preflight checks every directory in INSTALL_DIRS before the first write and refuses the whole install, naming the directories at fault. Before that, the run had already written the daemon binary, the systemd unit, the polkit rules and all twelve sudo grants, leaving live grants for helper scripts that did not exist.

Everything above the install still holds: detection, the rpm-ostree action family and the atomic story family are implemented and covered by the workspace suite. What is missing is a way to put the helpers somewhere the daemon's own grants already point.

The deterministic workspace baseline is 1,845 Rust tests plus 72 frontend tests. Those tests verify action construction, policy, approval, storage, and UI behavior, but they do not replace a real distribution VM run.

Important scope differences

• Atomic rollback applies to rpm-ostree deployment changes. Ubuntu package operations are mutable and cannot offer equivalent deployment rollback.
• Ubuntu Server may use netplan with systemd-networkd; Ubuntu Desktop often uses NetworkManager. SysKnife detects and routes those mechanisms.
• apt can contend with unattended upgrades and needrestart; the Ubuntu actions use non-interactive execution and bounded lock handling.
• Fedora Workstation and Server require a dedicated dnf action family. Falling through to rpm-ostree commands would be incorrect, so they are not reported as supported.

The complete Ubuntu action catalogue is in the Ubuntu action reference.

Distro detection

SysKnife parses /etc/os-release without evaluating it as shell code:

• ID selects Fedora, Ubuntu, Debian, or another exact distribution.
• ID_LIKE supplies a family fallback for planning.
• VERSION_ID determines the release.
• VARIANT_ID distinguishes Fedora Atomic variants and Ubuntu Core.

Ubuntu Core is detected separately and is not supported. Unknown Debian- or Fedora-family systems receive a warning rather than a false support claim.

Planned systems

Verify a host

sysknife doctor

doctor reports detected distribution, daemon reachability, provider, and audit-chain status. For release evidence, follow the current VM procedures in Testing and record the exact image, architecture, model, commit, and story results in the release checklist.

Adding support

1. Document the action mapping and unsupported semantics.
2. Add real /etc/os-release fixtures and detection tests.
3. Implement typed actions without raw shell strings.
4. Add policy, preview, and executor consistency tests.
5. Add a reproducible VM harness and record a full run before using the Validated label.

Architecture

SysKnife is a local Linux control plane built around a strict boundary between planning, presentation, and execution.

Crate and App Layout

sysknife-brain

Unprivileged. Reads the user's intent, queries the daemon for system state, then calls the LLM in a tool-use loop until a typed plan is produced. Provides:

• LlmPlanner — the planning loop entry point
• BrainConfig — provider and model selection
• Provider adapters: Anthropic, OpenAI, Gemini, Ollama, Groq, DeepSeek, Mistral, xAI
• Planning tools: get_system_state, query_*, propose_plan, remember, forget
• Safety fence: validates every action name and risk level before a plan leaves the brain

Prompt construction — per-distro dispatch

build_system_prompt in crates/sysknife-brain/src/prompt.rs dispatches to one of three pure render functions based on distro_hint.family:

Each render function concatenates shared const blocks (role, rule, examples) with per-distro const blocks (action catalogue, worked examples, parameter reference). Fedora prompts never contain Debian action names and vice versa — the isolation is structural. This prevents the model from proposing AptInstall on Fedora or AddLayeredPackage on Ubuntu regardless of what the user types.

The prompt is rebuilt on every plan_intent() call so that injected user preferences (~/.config/sysknife/prefs.md) are always current.

See ADR 0004 for the full rationale.

sysknife-types

Shared domain types used by every crate. Contains:

• CallerRole — Observer | Dev | Admin | Boot
• RiskLevel — Low | Medium | High
• JobState — Queued | Running | Succeeded | Failed | Canceled | RolledBack | NeedsReboot
• Request and result envelopes for IPC messages

sysknife-core

Shared constants and ~/.config/sysknife/config.toml loading via LacsConfig. Config file values become env-var defaults so every component uses the same resolution order.

sysknife-daemon

Privileged. The only component that touches the system. Provides:

• 190 typed actions (rpm-ostree, systemd, firewall, users, containers, flatpak, toolbox, SSH, kernel args, …)
• Role-based authorization (Observer → Dev → Admin, plus Boot)
• Policy enforcement: stale-approval detection, request hash validation
• Preview generation: risk level, side effects, reboot flag, rollback metadata, content hash
• IPC dispatcher over a Unix domain socket
• Live stdout streaming as JobProgress frames
• Automatic rollback for supported high-risk actions that fail
• SQLite transaction audit log

sysknife-shell

The user-facing surface. A Tauri app (Rust backend + React frontend) that provides:

• Intent entry pane
• Plan review with risk badges and previews
• Approval gate (explicit checkbox for Medium; typed action name for High)
• Live job timeline with streaming output
• Setup wizard for first-run LLM configuration

sysknife-cli

The production CLI (apps/sysknife-cli/). Accepts a natural-language intent, calls the LLM planner, and prints or executes the resulting plan. --dry-run --json mode emits the plan as JSON on stdout without executing — this is the mode used by every E2E story script.

Trust Boundary

The daemon is trusted. The brain and shell are not trusted with raw privileged execution.

sysknife-brain  ──plan──►  approval gate  ──receipt──►  sysknife-daemon
 (planner)               (terminal, or the        (executor)
                          paused Tauri shell)

The middle box is whichever surface collects the human's approval and returns a one-time receipt. sysknife approve in a terminal is the maintained one, and the one MCP clients are required to use; sysknife-shell is a second implementation of the same gate whose development is paused. Neither is a component the other passes through.

The daemon owns:

• authorization (role-based: Observer → Dev → Admin, plus Boot)
• policy (stale-approval detection, request validation)
• previews (risk level, side effects, rollback metadata)
• jobs (execution, live output streaming)
• transaction records (SQLite audit log)
• rollback (automatic on failure for supported actions)

Neither the brain nor the shell can execute a privileged action directly. The daemon verifies and atomically consumes a one-time approval receipt before running anything.

Request Flow

1. A user enters intent in the shell.
2. The brain proposes a typed plan.
3. The shell sends each mutating step to the daemon for preview.
4. The daemon persists an immutable preview and returns its transaction ID, risk level, side effects, reboot requirement, and rollback availability.
5. The shell shows the preview and captures approval. High-risk steps require the user to type the action name explicitly.
6. The daemon issues a deterministic, domain-separated Ed25519 receipt and stores its SHA-256 commitment inside the signed transaction row.
7. The client sends the exact transaction, action, params, and receipt.
8. The daemon verifies the preview is fresh and atomically consumes the receipt, then runs the action.
9. During execution, the daemon streams live stdout output line-by-line as JobProgress frames.
10. The shell displays each line as it arrives.
11. On failure, if rollback_available is true, the daemon runs the rollback action automatically and reports the result.
12. The transaction is persisted to the configured SQLite or PostgreSQL store with the final job state.

IPC Protocol

The shell and daemon communicate over a Unix domain socket ($XDG_RUNTIME_DIR/sysknife/daemon.sock by default, overridable via SYSKNIFE_LISTEN_URI).

The framing is a 4-byte little-endian u32 length prefix followed by a UTF-8 JSON body. Each message carries a "type" discriminant so the dispatcher can route without a full decode.

Maximum message size is 4 MiB. The daemon limits concurrent connections to 16 via a tokio semaphore; excess connections are dropped immediately rather than queued.

The protocol is human-readable. You can inspect live traffic with:

socat - UNIX-CONNECT:"$XDG_RUNTIME_DIR/sysknife/daemon.sock"

See ADR 0003 for the rationale behind length-prefixed JSON over gRPC or binary protobuf.

Design Principles

• typed instead of free-form
• local instead of remote
• auditable instead of opaque
• rollback-aware instead of irreversible
• explicit approval instead of hidden mutation

Typed Actions

Every AI-driven system administration tool has to answer one question: what, exactly, is the model allowed to make the machine do? Most answer it with an allowlist or a regex over a shell command string. SysKnife answers it differently — the model is never given a channel to produce a shell string in the first place.

Why string allowlists fail

The independent security research known as GuardFall tested AI coding agents against shell-command guardrails — allowlists and regex filters applied to the command string the model was about to run — and found that 10 of 11 agents could be talked into bypassing them. That is not a bug in any one guard; it is a structural property of the approach. A string allowlist has to anticipate every syntactic trick (quoting, encoding, command substitution, argument smuggling, alias/function shadowing) a sufficiently motivated or merely creative model can produce, in a language — shell — that was designed to make exactly those tricks easy. Filtering a dangerous language after the fact is an arms race you don't get to stop fighting.

SysKnife's answer is architectural, not defensive: don't give the model the dangerous language at all.

The model: typed actions, not shell strings

The brain (sysknife-brain) runs the LLM in a tool-use loop against a single terminal tool, propose_plan. That tool's schema is built from KNOWN_ACTIONS in crates/sysknife-brain/src/planning_tools/propose_plan.rs — a fixed, named list of actions such as InstallPackages, StartService, or AptUpdate, each with a one-line description of its purpose and parameters. The model can only emit one of these names plus a small parameter object; it has no field, anywhere in the schema, into which a shell string fits.

Every proposed action name is checked against ActionName::parse before a plan leaves the brain — this is the safety fence referenced throughout the codebase. On the daemon side (sysknife-daemon), each action is backed by an ActionSpec (crates/sysknife-daemon/src/actions/mod.rs) that pins down:

As of this writing the catalogue defines 190 actions across families such as Deployment, Services, Package Layering, Flatpak, Containers, Toolbox, Network, Identity, SSH Keys, Package Repositories, apt/snap/ufw/netplan/grub (Debian-family), and rpm-ostree/AppArmor/cloud-init/Pro (Fedora-family). Each family is also fenced by distro: crates/sysknife-core/src/action_family.rs is the single source of truth for which actions are Debian-only versus Fedora-only, and that list is shared — not duplicated — across the brain's prompt construction, the CLI's routing guard, and the daemon's execution fence, specifically so the three can never silently drift apart.

The daemon is the sole executor and the sole authority. It does not trust the brain's judgment about parameters — every ActionSpec's mechanism is re-validated against the actual request before anything runs. A typed action name plus a parameter object cannot be turned into an arbitrary command line, because the daemon, not the model, decides which program and argv shape that action maps to.

A concrete example

Say a user asks something that sounds routine but is actually destructive:

"Clean up old kernels, I'm low on disk space."

An allowlist-guarded shell agent has to decide, in the moment, whether whatever command string it's about to construct is safe — and a determined or confused model can dress up something worse as "cleanup." SysKnife never gets to that fork. The brain can only propose the one typed action that matches this intent:

{
  "action_name": "CleanupDeployments",
  "params": {}
}

The daemon looks up CleanupDeployments in its catalogue and finds: risk_level: High, reboot_required: false, rollback_available: false — because this action permanently deletes the rollback and pending deployments (sudo rpm-ostree cleanup --rollback --pending), it destroys the very safety net that would undo it. That fact isn't something the model asserts and the daemon has to believe; it's a property of the fixed ActionSpec the daemon already owns. The risk level and the missing rollback both surface in the approval prompt before anything executes.

Composing with approval and rollback

A typed action alone is not the whole safety story — it's the piece that determines what can be requested at all. What happens to a request after it's typed is covered elsewhere:

• Every preview the daemon issues must be approved before it can be executed, and every mutating action — including its risk level and the approval that authorized it — is recorded in a forward, Ed25519-signed hash chain. See The Audit Chain.
• When an action whose ActionSpec carries rollback_available: true fails, the daemon executes its rollback automatically, in the same job, before the caller ever sees a terminal result. See Automatic Rollback.

Typed actions are what make both of those guarantees possible to state precisely: the audit chain can name an exact action_name and risk_level because those are fixed catalogue facts, not free text extracted from a shell command after the fact; automatic rollback can look up "is there a rollback for this" because the mapping from action to rollback action is a lookup table, not a heuristic.

The tradeoff, honestly

Typed actions are a closed catalogue. If an operation isn't in it, the model cannot perform it — full stop, no escape hatch. That is the entire point, but it has a real cost:

• Adoption cost. Every new operation SysKnife should support requires an engineer to add an ActionSpec, wire its mechanism, decide its risk level, and — if applicable — its rollback. This is slower than "the model can run any command it comes up with."
• Coverage gaps. A user's request that doesn't map to any action in the catalogue gets a "I can't do that yet" rather than a best-effort shell command. docs/ubuntu-action-gap-analysis.md is a historical (2026-04-25) snapshot of this tradeoff playing out in the Debian/Ubuntu action set — almost all of the gaps it identified have since been closed.

SysKnife accepts this tradeoff deliberately: a finite, auditable vocabulary that the daemon fully understands is worth more than an open-ended one the daemon can only try to filter.

Action reference

This file is generated. Do not edit by hand. Regenerate with UPDATE_ACTION_REFERENCE=1 cargo test -p sysknife-daemon --test action_reference_doc; a plain cargo test fails if it drifts from the catalogue.

Every row is derived from the live code: the command from each action's ActionSpec mechanism, the risk from its risk_level, the distro from sysknife-core::action_family, and the description from the brain's KNOWN_ACTIONS list. Distro is All (cross-distro), Ubuntu (Debian-family only), or Fedora (atomic-host only). Rb = requires reboot; Ro = automatic rollback available.

Deployment (atomic host)

Package layering (rpm-ostree)

Filesystem

Flatpak

Toolbox

Services

Processes

Journald

Storage — LVM

Kernel parameters — sysctl

Mounts & swap

Log management

PAM password policy

auditd

certbot / ACME

Scoped sudoers.d

Network

resolvectl

Identity / time / locale

Users & groups

SSH keys

Package repositories

System info

Containers

Reboot

AppArmor

cloud-init

fail2ban

apt

apt preferences / pinning

PPA

snap

ufw

netplan

distrobox

GRUB

Ubuntu release upgrade

Ubuntu Pro

Livepatch

Multipass

189 actions have an ActionSpec and are tabled above. The full catalogue (KNOWN_ACTION_NAMES) also includes ListJobHistory, which the dispatcher handles before the executor, for 190 total.

The Audit Chain

Every mutating action SysKnife's daemon runs is recorded in a forward, Ed25519-signed hash chain. This page covers what is recorded, how the chain proves tamper and reorder, why the signature scheme is asymmetric rather than a shared-secret MAC, how signed checkpoints close the one gap a hash chain cannot close on its own, and how to verify all of it yourself — independently of the daemon that wrote it. If you are evaluating SysKnife for a security-sensitive deployment, this is the page to try to break.

What gets recorded

Each row in the transaction table captures the decision the daemon made about one action, at the moment it made it. sysknife history renders a subset; caller_role, caller_principal and event_tip are chain fields, read with sysknife audit export (or directly from the database) and checked with sysknife audit verify:

These fields are serialized into a stable, self-describing byte string (tag + value pairs, with a prefix-free escape scheme so no field's content can be crafted to alias another field's boundary), then signed. The count is per encoding, because the field set grew: v1 signs eleven pairs, v2 fourteen, v3 fifteen — the fields above plus the chain_version tag that names the encoding (see below). The resulting signature is the row's chain_hash — there is no separate hash step, because Ed25519 already commits to the message.

The machine-readable side publishes rows_censused so the gap is measurable: compare it with rows_checked, which the CLI --json nests under chain and the sysknife_audit_verify tool reports as a sibling field. Read the chain's own verdict, not the top-level status, when deciding whether the counts are findings: status is the worst of three checks, so a broken approval-event chain turns it broken while the transaction chain and its attribution are intact. The MCP report carries chain_status for exactly that reason.

What a uid does not prove: shared logins, su into a service account, and uid reuse after a user is deleted all weaken it. It identifies an account, not a human.

## The hash chain: each entry links to the one before it

Every row also stores `prev_chain_hash`, and the signed message is:

```text
chain_hash = Ed25519-Sign(signing_key, ROW_DOMAIN || canonical(row_fields) || prev_chain_hash)

Because prev_chain_hash is inside the signed message, a row's signature is only valid if it was produced with the exact preceding row's hash. This gives verification two independent failure modes to catch tampering:

• Content tamper — edit any field of a row after the fact (e.g. change summary or risk_level) and its own signature no longer verifies.
• Reorder or delete — remove or reorder a row and the next row's prev_chain_hash no longer matches the actual predecessor, breaking the chain at that point even if every individual signature is otherwise valid.

sysknife audit verify walks the chain in seq order and reports the first row where either check fails.

Ed25519, not HMAC — why asymmetric matters

The signing key is a 32-byte Ed25519 seed, generated on first daemon start and stored at <db_dir>/audit-key (mode 0600, refused if group/world readable), with $SYSKNIFE_AUDIT_KEY_PATH as an override for systemd deployments. The corresponding public key is written alongside as <audit-key>.pub.

This is a deliberate replacement for an earlier HMAC-SHA256 design. The distinction matters more than it might look:

• With a symmetric MAC (HMAC), the same secret both produces and checks the tag. Anyone able to verify the chain is, by construction, also able to forge it. A "verified" HMAC chain is a claim the verifier is making about itself — it convinces no one who doesn't already trust the verifier's custody of the secret.
• With Ed25519, the daemon signs with a private key that never leaves its host, and verification uses only the corresponding public key. Publishing the public key gives an auditor, a central log aggregator, or a customer's security team the ability to prove the chain is intact and unforged — without ever being able to forge an entry themselves. This is non-repudiation: a signature that verifies under the daemon's public key could only have been produced by whoever holds the private key.

Two more properties fall out of the implementation:

• Domain separation. Row signatures, checkpoint signatures, and approval receipts each sign under a distinct, prefix-free domain tag (sysknife-audit-row-v1, sysknife-checkpoint-v1, sysknife-approval-receipt-v1). A signature valid in one context can never be replayed as valid in another, even though the underlying key is shared.
• Determinism. Ed25519 (RFC 8032) is deterministic — identical inputs always produce the identical signature. This makes chain verification reproducible without any randomness or nonce bookkeeping on the verifier's side.

The approval-event chain

The transaction chain records what the daemon authorized. It says nothing about whether a human then approved it, whether that approval was spent, or whether it was retracted — those lifecycle facts used to live only in transaction_approvals, a plain mutable table. Deleting a row from it left sysknife audit verify reporting Intact, which made the record that a privileged action had been approved the one part of the trail an attacker could erase without leaving a mark.

Approval events are now their own forward Ed25519 chain, under their own domain tag (sysknife-audit-event-v1), with one row per state change:

Each event and the state change it records commit in the same database transaction, so a state change without its event (or the reverse) is not a reachable state. Deleting an event from the middle of the chain breaks the next event's prev_chain_hash, exactly as it does for transaction rows.

Cross-chain binding. Deleting events from the end of the event chain would leave a self-consistent remainder, the same tail-truncation blind spot the transaction chain has. That is what event_tip is for: every transaction row signs the event-chain tip as of its insert. Because signed checkpoints anchor the transaction chain, and transaction rows commit to event tips, anchoring transitively covers the event chain. sysknife audit verify reports the binding check alongside the two chain walks.

The residual exposure is the same bounded tail every append-only log has: events appended after the most recent transaction row are not yet bound by anything, until the next row is written.

Signed checkpoints: closing the truncation gap

The default deployment has no anchor. Anchoring is opt-in via SYSKNIFE_CHECKPOINT_DB, and packaging/sysknife-daemon.service does not set it, so a stock system install cannot detect tail truncation — the gap this section describes is open until you configure a sink. The daemon says so at startup and sysknife audit verify repeats it beside every verdict, because OK: N rows verified would otherwise read as "nothing was removed". Setup instructions are in SECURITY.md.

A hash chain alone cannot detect one specific attack: tail truncation. If an attacker with write access to the audit database deletes the most recent K rows, the remaining chain is still perfectly self-consistent — every row's signature is valid and every prev_chain_hash still points to its (now-final) predecessor. The chain walk reports Intact, because nothing in the remaining rows says how long the chain used to be.

Signed checkpoints close this gap using the same idiom Certificate Transparency uses for its logs: periodically, the daemon signs a commitment to the current chain tip —

checkpoint_signature = Ed25519-Sign(signing_key, CHECKPOINT_DOMAIN || seq || chain_tip || created_at)

— and anchors (seq, chain_tip, created_at, signature) into an independent, append-only sink (a separate PostgreSQL database via sysknife audit checkpoint). Because the checkpoint lives outside the chain it commits to, an attacker who can edit or truncate the local chain cannot also reach back and edit the anchored checkpoint. Verifying anchored checkpoints against the current chain distinguishes three outcomes:

• Consistent — the checkpoint's seq is still present in the chain and its chain_hash at that seq matches the anchored tip.
• Truncated — the checkpoint's seq is no longer present at all (the chain is now shorter than a previously anchored tip proves it once was).
• Tip mismatch (rewrite) — the seq is present, but its chain_hash no longer matches what was anchored (the row was rewritten in place).

sysknife audit checkpoint refuses to anchor a chain that does not already verify — it never launders a tampered chain into a signed checkpoint.

sysknife audit verify performs the same cross-check, read-only, whenever SYSKNIFE_CHECKPOINT_DB is set. It previously reported only that an anchor was configured and verified the chain against itself, which is the one thing a truncated chain also passes: an operator who had done the work to set anchoring up got a verdict no stronger than one who had not, and without the caveat that warns the unanchored case. The cross-check now runs, its result appears beside the chain verdict, and a Truncated or rewritten verdict fails the command (exit 1) rather than printing under an OK.

Two boundary cases are deliberate. An anchor holding no checkpoints reports cannot_verify, not consistent — zero checkpoints trivially satisfy "every checkpoint agrees", and calling that success would attest to a chain nothing has ever committed to. And because anchoring itself is implemented for the SQLite backend only, a Postgres deployment with SYSKNIFE_CHECKPOINT_DB set is told so explicitly rather than being silently skipped.

Verifying it yourself

Verification only ever needs the public key. You do not need daemon access, the private signing key, or trust in the machine that produced the log — only the exported <audit-key>.pub file (or its hex contents) and read access to the transaction database.

Full chain, third-party path:

sysknife audit verify --pubkey audit-key.pub

Verification is host-local, and that matters over a tunnel

plan, execute, history and doctor are daemon requests: they travel over SYSKNIFE_SOCKET, which in the VM and remote topologies points at another machine. Verification is not a daemon request. It opens the transaction store on the filesystem of the machine you run it on.

So this sequence does not do what it looks like:

ssh -fN -L /tmp/sysknife-web01.sock:/run/sysknife/daemon.sock admin@web01
SYSKNIFE_SOCKET=/tmp/sysknife-web01.sock sysknife "install ripgrep"   # runs on web01
sysknife audit verify                                                 # reads THIS machine

The verdict describes your own laptop's chain. If a local store exists, which it does on any machine that has ever run a user-mode daemon, the answer is a confident OK about actions that happened somewhere else.

The command now says so whenever SYSKNIFE_SOCKET is set:

OK: 128 row(s) verified in /home/you/.local/state/sysknife/daemon.sqlite
NOTE: SYSKNIFE_SOCKET is /tmp/sysknife-web01.sock. If that socket is forwarded
from another host (for example `ssh -L`), the actions you took ran there while
this verification read a store on this machine. Verify on the host that owns the
daemon, or copy its database and exported public key out and re-run with
--pubkey <FILE>.

The same string travels in --json output as daemon_socket_caveat, and on the sysknife_audit_verify MCP tool's report under the same name, so an agent cannot report a clean trail for the wrong host either.

Two correct ways to verify a remote host:

# On the host that owns the daemon
ssh admin@web01 'sudo sysknife audit verify'

# Or pull the evidence to an auditor's machine: the store plus the public key,
# never the private key
scp admin@web01:/var/lib/sysknife/daemon.sqlite  ./web01.sqlite
scp admin@web01:/var/lib/sysknife/audit-key.pub ./web01.pub
SYSKNIFE_DATABASE_PATH=./web01.sqlite sysknife audit verify --pubkey ./web01.pub

The second form is the auditor path the public key exists for: it proves the chain without granting daemon access or signing ability.

Machine-readable output for CI or a SIEM pipeline:

sysknife audit verify --pubkey audit-key.pub --json
sysknife audit export --since 2026-08-01T00:00:00Z --limit 500 > audit-rows.json

audit export emits the stored transaction-chain rows as one JSON array in ascending seq order. It includes both prev_chain_hash and the Ed25519 signature stored as chain_hash, so an offline consumer has the linkage and signature bytes without opening SQLite itself. It does not invent argv, outcome, or a separate signature field that the signed row never stored. Absent optional columns are JSON null, while a value actually recorded as an empty string remains an empty string.

An export inherits the confidentiality class of the audit database and is not a redacted artifact. request_hash commits to the unredacted parameters as one unsalted SHA-256, so a low-entropy value such as a Wi-Fi passphrase remains attackable in an exported file even though the stored preview is redacted. Handle an export like the 0600 database it came from.

Anchor and check signed checkpoints against an external database:

SYSKNIFE_CHECKPOINT_DB="postgres://user@host/checkpoints" sysknife audit checkpoint

Three checks run: the transaction chain, the approval-event chain, and the binding between them. All three are reported, and any one of them can fail the command — a clean transaction chain must never mask a tampered approval trail.

A clean run looks like:

OK: 4128 row(s) verified in sqlite
OK: 512 approval event(s) verified
OK: 4128 row(s) still match the approval event they committed to

A detected tamper looks like:

BROKEN: chain intact for first 891 row(s); row seq=892 (transaction 3f9c1a2e)
does not chain.
  expected: valid ed25519 signature
  actual:   9b1f...c02a

or, for a reordered/deleted row, a prev_chain_hash mismatch instead of a signature mismatch — same report shape, different expected/actual pair. Either failure is reported at the first broken row; rows before it are still proven intact.

Deleted approval events look like:

OK: 4128 row(s) verified in sqlite
OK: 300 approval event(s) verified
BROKEN: transaction seq=4128 committed to approval event 71ac…9d2f, which is
no longer in the event chain — approval events were deleted from the end of
the chain

Note that the event chain itself still walks clean there: truncating its tail leaves a self-consistent remainder. The binding is what catches it.

Exit codes matter for automation: 0 intact, 1 broken (a real tamper was detected), 2 cannot verify (missing key file, unreadable database, wrong key generation loaded). The 1-vs-2 split is deliberate — a CI job that only checks for a nonzero exit code must not silently treat "I couldn't check" the same as "I checked and it's fine." When the three checks disagree, the worst wins, and 1 outranks 2: if anything is provably broken, saying "could not verify" would understate what is known.

When the store cannot be read, the cross-chain binding status is not_checked, not consistent; no transaction or approval-event rows were available to compare.

Limits and honest scope

The chain is strong evidence, not a magic guarantee. Be precise about what it does and does not prove:

• Key custody is the trust root. Anyone who reads the private key file can forge future entries indistinguishably from real ones. Signed checkpoints bound the damage to "after the compromise," since prior anchored tips remain unreproducible from a rewritten chain.
• status is out of scope by design. The chain protects the decision recorded at insert time (what was previewed, at what risk level, with what warnings), not later execution-status transitions. "Chain verifies" does not mean "the action's final status is trustworthy."
• Truncation needs an external sink to be detectable at all. Without a checkpoint anchored off-host, deleting the tail of the chain is invisible to sysknife audit verify by construction. The same applies to approval events appended after the last transaction row: nothing has committed to them yet.
• Key rotation is manual today. Every row carries a key_id (currently always "v1"); rotation means regenerating the chain from scratch until a planned epoch-aware rotation flow lands.
• This is a detection control, not a prevention control. It proves, after the fact, that something was altered — it does not stop the daemon from executing an authorized-looking but malicious action. That is the job of the layered authorization model in SECURITY.md.

See also: Audit Storage and Recovery for backend choice, backup procedure, and restore verification; CLI Reference for the full sysknife audit command surface.

Automatic Rollback

Automatic rollback is one of SysKnife's two hard differentiators (the other is the approval-gated, hash-chained audit log). This page describes exactly what it does, the mechanism behind it, and — just as importantly — where it does not apply yet.

What "automatic rollback" means

When the daemon executes a mutating action and that action's process exits non-zero (or fails to launch), the dispatcher checks two things before returning a result to the caller:

1. Did the job land in JobState::Failed?
2. Does the executed ActionSpec carry rollback_available: true?

Both conditions are evaluated in attempt_rollback_if_needed in crates/sysknife-daemon/src/dispatcher.rs. If both hold, the daemon looks up a rollback ActionSpec for the failed action name via executor::rollback_spec_for and executes it immediately, in the same job, before the client ever sees a terminal result. There is no separate "approve the rollback" step — a failed rollback-eligible action rolls back automatically as part of finishing that job.

If the rollback command itself succeeds, the job transitions to JobState::RolledBack and the JobResult carries a rollback_ref describing what was reverted. If the rollback command also fails, the job stays Failed and the response tells the caller the system may need manual intervention — SysKnife does not retry indefinitely or attempt a different recovery path.

The flag is an equivalence, not a hint, and it's enforced by a test. ActionSpec.rollback_available means exactly one thing: the daemon will automatically revert this action if it fails. It does not mean "there happens to be another action that manually undoes this one" — an action can be perfectly reversible by hand (e.g. RemovePpa undoes AddPpa) while still reporting rollback_available: false, because undoing it requires the operator or agent to explicitly issue that second call; the daemon does not do it for them on failure. rollback_available_matches_rollback_spec_for_all_actions in crates/sysknife-daemon/src/executor.rs asserts, for every action in actions::all_specs() — not a hand-picked subset — that spec.rollback_available == rollback_spec_for(action_name).is_some(). An action can't claim automatic rollback it doesn't have, or fail to claim rollback it does have, without breaking that test.

The mechanism: rpm-ostree rollback

On rpm-ostree-based systems (Fedora Atomic / Silverblue), every mutation to the base OS is an atomic deployment: the tree that boots next is a new, separate deployment, and the previously active deployment is retained until explicitly cleaned up. This gives SysKnife a rollback target it doesn't have to construct itself.

rollback_spec_for in crates/sysknife-daemon/src/executor.rs maps every rollback-eligible action to the same recovery command, rpm-ostree rollback, which swaps the pending deployment back to the one that was active before the failed action ran:

• UpdateSystem
• InstallPackages, RemovePackages
• RebaseSystem
• SetKernelArguments
• AddLayeredPackage, RemoveLayeredPackage, ReplaceLayeredPackage, ResetLayeredPackageOverride, RemoveBasePackage

All of these are rpm-ostree deployment mutations — they change what will boot next, not the currently running root. That's what makes reverting them a single, well-defined operation instead of an attempt to undo an arbitrary set of file writes. No pre-change snapshot has to be staged by SysKnife itself: rpm-ostree's deployment history already holds the "before" state, and rollback is rpm-ostree's own primitive for restoring it.

Some rollback-eligible actions (RebaseSystem, SetKernelArguments, layered package changes) also require a reboot to take effect either way — rollback restores the deployment that will be booted, not a running-process state.

Honest scope: Ubuntu/apt has no automatic rollback yet

This matches docs/distro-support.md, which states plainly: "Atomic rollback applies to rpm-ostree deployment changes. Ubuntu package operations are mutable and cannot offer equivalent deployment rollback." Do not describe SysKnife as rolling back Ubuntu package state in any other doc — today it gates and audits apt, it does not revert it.

Interaction with the audit chain

Rollback outcomes are not a side channel — they go through the same paths as every other job result:

• The job's terminal status is written via update_terminal_status, so a successful rollback lands in the transactions table as JobState::RolledBack, not Failed. A query over the audit log shows the full story: the action was attempted, it failed, and it was reverted.
• The daemon forwards a status-change event to the configured SIEM sink (forward_status_change_event) so RolledBack is visible to external monitoring, not just the local database.
• The JobResult returned to the client includes rollback_ref (currently the literal string "rpm-ostree rollback") so the caller — CLI, MCP tool, or GUI — can show what was reverted.

One nuance worth knowing: the Ed25519 hash chain in crates/sysknife-daemon/src/audit_chain.rs signs the immutable fields captured when a transaction row is first inserted (the authorization decision — who approved what, at what risk level). status is a mutable column and is explicitly excluded from the signed payload — see the "Status mutations are not in the chain" note in audit_chain.rs. That means a completed rollback is durably recorded and forwarded, but the chain's tamper-evidence guarantees cover the fact that the risky action was approved, not the specific fact that it was later rolled back. If your threat model needs the status transition itself to be tamper-evident, that's tracked as future work (an append-only audit_events table), not something SysKnife claims today.

See also

• Distro Support — full support matrix and per-family caveats.
• MCP Server — how sysknife_execute surfaces rollback_ref to an AI assistant.
• Architecture — the planning/presentation/execution boundary that rollback lives inside.

VM + Daemon Setup

This guide walks through running the SysKnife daemon inside a virtual machine and connecting Claude Code (or any MCP client) to it from the host.

Two transport options are covered:

Both transports produce the same .mcp.json — only the SYSKNIFE_SOCKET value and the optional SYSKNIFE_TOKEN differ.

1. Install and start the daemon in the VM

SSH into the guest and run:

# Clone the repo (or copy a pre-built binary)
git clone https://github.com/lacs-project/sysknife
cd sysknife

# Build and install (requires Rust stable and a C compiler:
#   sudo apt-get install -y build-essential   — cmake is not needed)
make build
sudo make install

# Enable and start the daemon
sudo systemctl enable --now sysknife-daemon

# Confirm it is running
systemctl status sysknife-daemon

The daemon listens on /run/sysknife/daemon.sock by default when managed by systemd. You can verify the socket exists:

ls -la /run/sysknife/daemon.sock

Silverblue / Kinoite (rpm-ostree hosts): make install uses the correct rpm-ostree override flags automatically. No extra steps needed.

2a. SSH socket tunnel (simplest)

This approach works on any Linux VM regardless of hypervisor. It forwards the daemon's Unix socket to a local path on your host.

On the host — open the tunnel

# Replace <user>, <vm-host-or-ip>, and <port> for your setup.
# QEMU user-mode networking typically maps port 22 → 2222 on localhost.
ssh -fN \
  -L /tmp/sysknife-vm.sock:/run/sysknife/daemon.sock \
  -p 2222 <user>@localhost

For a network-accessible VM (libvirt bridge, cloud VM):

ssh -fN \
  -L /tmp/sysknife-vm.sock:/run/sysknife/daemon.sock \
  <user>@<vm-ip>

-fN forks the SSH process and keeps the tunnel open without running a remote command. The local socket /tmp/sysknife-vm.sock is created on the host and forwards transparently to the daemon inside the guest.

Three flags worth adding once you rely on this:

ssh -fN \
  -o ExitOnForwardFailure=yes \
  -o StreamLocalBindUnlink=yes \
  -o ServerAliveInterval=30 \
  -L /tmp/sysknife-vm.sock:/run/sysknife/daemon.sock <user>@<host>

• ExitOnForwardFailure=yes makes SSH fail loudly instead of forking a session whose forward never came up, which otherwise looks like a dead daemon.
• StreamLocalBindUnlink=yes removes a stale local socket file. Without it, a second tunnel after an unclean exit fails with "cannot bind".
• ServerAliveInterval=30 keeps a long planning session from being dropped by an idle NAT or firewall.

Who you are over a tunnel

Authorization is not carried by the tunnel. The daemon reads the connecting peer's credentials with SO_PEERCRED and resolves that peer's group membership (resolve_caller in crates/sysknife-daemon/src/dispatcher.rs). Over a forwarded socket the connecting peer is the sshd process on the remote side, running as the SSH user, so:

• The role comes from the remote user's groups: sysknife-observer, sysknife-dev, sysknife-admin, or wheel.
• That user must also be in the sysknife group, because /run/sysknife is 0750 sysknife:sysknife. Without it every request is refused before any role check runs.
• Your local groups are irrelevant. Being wheel on your laptop grants nothing on the remote host.

# On the remote host, for the account you SSH in as
sudo usermod -aG sysknife,sysknife-admin <user>
# then log out and back in: group membership only applies to a new session

This is why SysKnife tunnels a Unix socket rather than exposing an HTTP listener. SSH supplies authentication, the socket supplies authorization, and peer credentials cannot be spoofed by whoever reaches the port. See Registry and Directory Listings for the matching decision about network-exposed listings.

More than one host

Give each host its own local socket and its own MCP server entry. Nothing is shared between them, so there is no state to switch:

ssh -fN -o ExitOnForwardFailure=yes -o StreamLocalBindUnlink=yes \
  -L /tmp/sysknife-web01.sock:/run/sysknife/daemon.sock admin@web01
ssh -fN -o ExitOnForwardFailure=yes -o StreamLocalBindUnlink=yes \
  -L /tmp/sysknife-db01.sock:/run/sysknife/daemon.sock  admin@db01

{
  "mcpServers": {
    "sysknife-web01": {
      "command": "sysknife",
      "args": ["mcp-server"],
      "env": { "SYSKNIFE_SOCKET": "/tmp/sysknife-web01.sock" }
    },
    "sysknife-db01": {
      "command": "sysknife",
      "args": ["mcp-server"],
      "env": { "SYSKNIFE_SOCKET": "/tmp/sysknife-db01.sock" }
    }
  }
}

Each host keeps its own audit chain, on that host. sysknife audit verify reads the store on the machine you run it on, never through the socket, so verify on the host itself or pull its database and public key to an auditor's machine. See Verification is host-local.

Configure and connect (SSH tunnel)

Run the setup wizard on the host and choose the integration you want:

npx sysknife-setup

Use --claude, --cursor, --codex, or --all if you want a direct path without the picker.

When prompted for the daemon socket, enter /tmp/sysknife-vm.sock.

Or set SYSKNIFE_SOCKET manually in .mcp.json:

{
  "mcpServers": {
    "sysknife": {
      "command": "/path/to/sysknife",
      "args": ["mcp-server"],
      "env": {
        "SYSKNIFE_SOCKET": "/tmp/sysknife-vm.sock",
        "SYSKNIFE_LLM_PROVIDER": "anthropic",
        "ANTHROPIC_API_KEY": "<your-api-key>"
      }
    }
  }
}

Test the SSH tunnel connection

sysknife --dry-run "check disk usage"

2b. virtio-vsock (faster, no SSH required)

virtio-vsock is a kernel-level channel between the host and KVM/QEMU guests. It requires no network stack and survives network changes inside the guest.

Prerequisites

# On the host — load the vhost_vsock module
sudo modprobe vhost_vsock
lsmod | grep vhost_vsock   # confirm it loaded

# Persist across reboots
echo vhost_vsock | sudo tee /etc/modules-load.d/vhost_vsock.conf

Your VM must be started with a vsock device. With libvirt add this to the domain XML:

<devices>
  <vsock model='virtio'>
    <cid auto='yes'/>
  </vsock>
</devices>

With QEMU directly:

qemu-system-x86_64 \
  ... \
  -device vhost-vsock-pci,guest-cid=10

Find the guest CID

The Context ID (CID) is the vsock address of the guest. It is assigned by the hypervisor and visible in two ways:

From the host (libvirt):

virsh dominfo <vm-name> | grep -i cid
# or
virsh dumpxml <vm-name> | grep cid

From inside the guest:

cat /sys/class/vsock/local_cid   # e.g. prints "10"

Generate and distribute the pre-shared token

vsock connections require a pre-shared token to authenticate the host to the daemon. This prevents other processes on the host from connecting.

Security note — the vsock token is a replayable bearer credential. It is sent in the clear as the first frame, so anyone who can observe a legitimate vsock connection can capture and reuse it to authenticate as SYSKNIFE_TOKEN_ROLE (see #152 and the "vsock transport authentication" section of SECURITY.md). Run the daemon only where the vsock namespace is isolated (a single trusted host↔guest pair on a hypervisor you control), give the vsock path the lowest role that works (not admin), rotate the token on any suspicion of exposure, and for anything crossing an untrusted boundary forward the daemon's Unix socket over ssh -L instead — SSH gives the confidentiality and peer authentication vsock does not.

The token file holds only the trimmed token itself — no role: prefix. The role granted to a token-authenticated connection comes from a separate env var, SYSKNIFE_TOKEN_ROLE (default Dev if unset), read by the daemon process — not from the file contents.

The default token path is ~/.config/sysknife/token (the daemon's XDG config directory, next to prefs.md; respects $XDG_CONFIG_HOME if set), not /etc/sysknife/token.

# On the host — generate a token
openssl rand -hex 32
# e.g.: a3f8c2d1e4b7a0f9...

# On the guest — write the token to the daemon's token file
mkdir -p ~/.config/sysknife
echo "a3f8c2d1e4b7a0f9..." > ~/.config/sysknife/token
chmod 600 ~/.config/sysknife/token

# Grant Admin to token-authenticated vsock connections (default is Dev)
sudo systemctl edit sysknife-daemon
# Add under [Service]:
#   Environment=SYSKNIFE_TOKEN_ROLE=admin

# Restart the daemon to pick up the token and role
sudo systemctl restart sysknife-daemon

The token file contains just the raw token, with no role prefix:

# ~/.config/sysknife/token
<paste-token-here>

Configure and connect (vsock)

Run the setup wizard on the host and choose the integration you want:

npx sysknife-setup

Use --claude, --cursor, --codex, or --all if you want a direct path without the picker.

When prompted for the daemon socket, enter vsock://<CID>:9734 — for example vsock://10:9734. The wizard detects the vsock:// prefix and prompts for the token.

Or configure .mcp.json manually:

{
  "mcpServers": {
    "sysknife": {
      "command": "/path/to/sysknife",
      "args": ["mcp-server"],
      "env": {
        "SYSKNIFE_SOCKET": "vsock://10:9734",
        "SYSKNIFE_TOKEN": "<your-hex-token>",
        "SYSKNIFE_LLM_PROVIDER": "anthropic",
        "ANTHROPIC_API_KEY": "<your-api-key>"
      }
    }
  }
}

Test the vsock connection

sysknife --dry-run "check disk usage"

3. Reload the MCP client

In Claude Code, run /reload-plugins to pick up the new .mcp.json. The sysknife_plan and sysknife_execute tools should appear in the tool list.

Troubleshooting

"connection refused" or "no such file or directory"

• SSH tunnel: confirm the tunnel is running: ls -la /tmp/sysknife-vm.sock
• vsock: confirm vhost_vsock is loaded (lsmod | grep vhost_vsock) and the CID is correct (cat /sys/class/vsock/local_cid inside the guest)
• Confirm the daemon is running: systemctl status sysknife-daemon inside the guest

"SYSKNIFE_TOKEN is not set; vsock connections require a pre-shared token"

Set SYSKNIFE_TOKEN in the env block of .mcp.json (see above).

"authentication failed" on vsock

The token on the host (SYSKNIFE_TOKEN in .mcp.json) does not match the contents of ~/.config/sysknife/token on the guest. Verify both match exactly (the file should hold just the raw token, no role: prefix).

SSH tunnel drops when the terminal closes

Use ssh -fN (forks, no command) or run it via a systemd user unit or tmux/screen session. Alternatively switch to vsock which does not depend on SSH.

vsock not available in the guest

Check: ls /dev/vsock inside the guest. If absent, the VM was started without a vsock device — add the <vsock> element to the libvirt XML (see above) and restart the VM.

Security notes

• SSH tunnel: the tunnel is authenticated by your SSH keypair. No extra secrets are needed.
• vsock: the pre-shared token (SYSKNIFE_TOKEN) prevents other host processes from connecting. Keep it out of version control — add .mcp.json to .gitignore.
• Socket file permissions: the daemon socket at /run/sysknife/daemon.sock is owned by root:sysknife with mode 0660 — only members of the sysknife group can connect locally. The SSH tunnel and vsock bypass this by terminating at the daemon's authenticated IPC layer.

Audit storage and recovery

SysKnife's durable transaction audit log has two backends:

• SQLite is the default at /var/lib/sysknife/daemon.sqlite. It is suitable for a single host when that file and its audit key are backed up.
• PostgreSQL is recommended when audit history must survive a host loss or be centralized. The live contract is tested against PostgreSQL 17.

Journald and optional RFC 5424 syslog forwarding are complementary operational signals. They are best effort and are not the database of record.

PostgreSQL configuration

[storage]
backend = "postgres"
url = "postgres://sysknife:${PG_PASSWORD}@db.example.com:5432/sysknife_audit?sslmode=verify-full"

[storage.pool]
max_connections = 8
acquire_timeout_secs = 10
statement_cache_capacity = 100

Use a dedicated database and role. The role currently needs connection, schema usage/create, and DML privileges because the daemon applies migrations at startup. Do not grant cluster administration or access to unrelated databases. Store the URL in a root-readable environment file or secret manager, not in a world-readable config file.

On startup, the daemon:

1. Opens a transaction and takes a PostgreSQL advisory transaction lock.
2. Creates schema_migrations if needed.
3. Applies each pending migration once and records its version.
4. Refuses to start if the database schema is newer than the binary.

The CI contract verifies migration idempotence, approval claims, history, and hash-chain validation against a live PostgreSQL server.

Transaction-mode PgBouncer deployments may require statement_cache_capacity = 0. CockroachDB is not currently supported: its PostgreSQL wire compatibility does not include the migration and concurrency semantics SysKnife relies on.

Provider notes

Use a writer endpoint and TLS certificate verification wherever the provider supports it.

Provider CA bundles and authentication behavior change. Verify current provider documentation during deployment rather than copying a stale certificate URL from this guide.

SQLite backup and restore

Back up the database and audit key together. The key defaults to /var/lib/sysknife/audit-key and should remain mode 0600.

sudo systemctl stop sysknife-daemon
sudo sqlite3 /var/lib/sysknife/daemon.sqlite \
  ".backup '/var/backups/sysknife/daemon.sqlite'"
sudo install -m 0600 /var/lib/sysknife/audit-key \
  /var/backups/sysknife/audit-key
sudo systemctl start sysknife-daemon

Restore both files while the daemon is stopped, preserve ownership and modes, then run sysknife doctor and sysknife audit verify. A database restored without its matching key cannot validate new signatures against the original key identity.

PostgreSQL backup and restore

For production, enable provider-managed point-in-time recovery and retention appropriate to your incident-response policy. Also take a logical backup before every SysKnife upgrade that introduces a migration.

pg_dump --format=custom --no-owner --file=sysknife-audit.dump "$DATABASE_URL"
createdb sysknife_restore_test
pg_restore --no-owner --dbname=sysknife_restore_test sysknife-audit.dump

Restore into an isolated database first. Point a temporary SysKnife config at that database, provide a copy of the matching audit key, then verify:

XDG_CONFIG_HOME=/tmp/sysknife-restore-config \
SYSKNIFE_AUDIT_KEY_PATH=/secure/restore-test/audit-key \
  sysknife audit verify

XDG_CONFIG_HOME=/tmp/sysknife-restore-config sysknife doctor

The temporary config's [storage] URL must target the isolated restore, never the production database.

Do not claim recoverability until a timed restore drill has succeeded and its recovery point and recovery time are recorded. Database backups and audit-key backups should have separate access controls so compromise of one system does not silently permit forged history.

Forwarding and retention

• The transaction database is authoritative for preview, approval, execution, status, and chain history.
• The safety-fence JSONL file records plans rejected before daemon execution.
• Journald receives safety-fence events and audit-chain watermarks where available.
• UDP syslog forwarding can lose, reorder, or duplicate messages. Treat it as alerting telemetry, not a durable copy.

Define retention, deletion authorization, backup encryption, restore testing, and SIEM ingestion before production use. Monitor daemon startup failures, database capacity, backup age, restore-test age, and chain verification.

Ubuntu action reference

Canonical CLI invocations for the core Ubuntu action families in SysKnife (apt, snap, ufw, netplan, distrobox), verified against Ubuntu 24.04 LTS (Noble Numbat) authoritative sources on 2026-04-25.

This page covers the source-verified core families only, with manpage citations, a known-divergence list, and a live-validation checklist. For the complete catalogue of every action (all families, generated from the code) see the Action Reference; the narrative rationale is in Typed Actions.

Sources consulted:

• Ubuntu 24.04 manpage: apt-get(8) — https://manpages.ubuntu.com/manpages/noble/en/man8/apt-get.8.html
• Ubuntu 24.04 manpage: ufw(8) — https://manpages.ubuntu.com/manpages/noble/en/man8/ufw.8.html
• Ubuntu 24.04 manpage: netplan(8) — https://manpages.ubuntu.com/manpages/noble/en/man8/netplan.8.html
• Ubuntu 24.04 manpage: distrobox(1) — https://manpages.ubuntu.com/manpages/noble/man1/distrobox.1.html
• Snapcraft official docs — https://snapcraft.io/docs/
• Snapcraft manage-updates guide — https://snapcraft.io/docs/how-to-guides/manage-snaps/manage-updates/
• distrobox-rm official docs — https://distrobox.it/usage/distrobox-rm/
• dpkg status flag reference — https://manpages.ubuntu.com/manpages/focal/man1/dpkg-query.1.html

apt (apt-get, apt-mark, apt-cache, dpkg)

#> The absolute path is load-bearing, not cosmetic. sudo PATH-resolves only its

primary command; every later token is matched literally against the sudoers rule, and packaging/sysknife-sudoers spells /usr/bin/apt-get. A bare apt-get does not match and sudo falls through to asking for a password. See crates/sysknife-daemon/src/actions/apt.rs and the sudoers_grants_match_argv test that pins the two together.

AptUpdate — Refresh package index

• Canonical: sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get update
• SysKnife argv: ["sudo", "env", "DEBIAN_FRONTEND=noninteractive", "NEEDRESTART_MODE=a", "/usr/bin/apt-get", "update"]
• Status: ✅ matches
• Notes: DEBIAN_FRONTEND=noninteractive suppresses all debconf prompts. NEEDRESTART_MODE=a auto-restarts services post-install without prompting. The env wrapper is the standard way to inject these into a sudo invocation because sudo strips most environment variables by default.

AptUpgrade — Full system upgrade

• Canonical: sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get dist-upgrade -y
• SysKnife argv: ["sudo", "env", "DEBIAN_FRONTEND=noninteractive", "NEEDRESTART_MODE=a", "/usr/bin/apt-get", "dist-upgrade", "-y"]
• Status: ✅ matches
• Notes: dist-upgrade resolves dependency changes by removing packages where necessary. upgrade (without dist-) is safer but may not complete all upgrades. The choice of dist-upgrade is intentional and documented in the code. Exit code 0 = success; 100 = error.
• Recommendation: Consider adding --no-install-recommends to reduce blast radius; not strictly required but commonly advised for headless/server use.

AptInstall — Install package

• Canonical: sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get install -y <pkg>
• SysKnife argv: ["sudo", "env", "DEBIAN_FRONTEND=noninteractive", "NEEDRESTART_MODE=a", "/usr/bin/apt-get", "install", "-y", "<pkg>"]
• Status: ✅ matches
• Notes: Exit code 0 = success; 100 = error (package not found, broken deps, etc.). -y / --assume-yes required for non-interactive use.

AptRemove — Remove package (keep config files)

• Canonical: sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get remove -y <pkg>
• SysKnife argv: ["sudo", "env", "DEBIAN_FRONTEND=noninteractive", "NEEDRESTART_MODE=a", "/usr/bin/apt-get", "remove", "-y", "<pkg>"]
• Status: ✅ matches
• Notes: Config files in /etc are preserved. Use purge to also remove them.

AptPurge — Remove package and its config files

• Canonical: sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get purge -y <pkg>
• SysKnife argv: ["sudo", "env", "DEBIAN_FRONTEND=noninteractive", "NEEDRESTART_MODE=a", "/usr/bin/apt-get", "purge", "-y", "<pkg>"]
• Status: ✅ matches

AptAutoremove — Remove orphaned dependency packages

• Canonical: sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get autoremove -y
• SysKnife argv: ["sudo", "env", "DEBIAN_FRONTEND=noninteractive", "NEEDRESTART_MODE=a", "/usr/bin/apt-get", "autoremove", "-y"]
• Status: ✅ matches

AptHold — Pin package at current version

• Canonical: sudo apt-mark hold <pkg>
• SysKnife argv: ["sudo", "apt-mark", "hold", "<pkg>"]
• Status: ✅ matches
• Notes: apt-mark hold does not require DEBIAN_FRONTEND; it only writes to the dpkg hold file (/var/lib/dpkg/info/<pkg>.list) without invoking debconf or needrestart.

AptUnhold — Remove version pin

• Canonical: sudo apt-mark unhold <pkg>
• SysKnife argv: ["sudo", "apt-mark", "unhold", "<pkg>"]
• Status: ✅ matches

AptSearch — Search repositories

• Canonical: apt-cache search <term>
• SysKnife argv: ["apt-cache", "search", "<term>"] (no sudo)
• Status: ✅ matches
• Notes: apt-cache is read-only and does not require sudo. Output format is <pkg-name> - <short description>, one line per match.

AptListInstalled — List installed packages

• Canonical (SysKnife uses): dpkg -l
• Alternative (better for parsing): dpkg-query -W -f='${Package}\t${Version}\t${Status}\n'
• SysKnife argv: ["dpkg", "-l"]
• Status: ⚠ functional but suboptimal for machine parsing
• Notes: dpkg -l output format:
  • Header line: Desired=Unknown/Install/Remove/Purge/Hold
  • Data lines: <desired><status><error-flag> <pkg> <version> <arch> <description>
  • Two-letter prefix meaning: ii = installed+ok, rc = removed+config-files-present, hi = on-hold+installed, un = not installed.
  • The output is human-oriented with variable-width columns padded to align. dpkg-query -W is cleaner for programmatic parsing but is not a bug — dpkg -l is well-understood and widely used.
• No sudo required.

AptShow — Show package metadata

• Canonical: apt-cache show <pkg>
• SysKnife argv: ["apt-cache", "show", "<pkg>"] (no sudo)
• Status: ✅ matches
• Notes: Output is RFC 822 style (key: value). No sudo required.

AptListUpgradable — List packages with available upgrades

• Canonical: apt list --upgradable
• SysKnife argv: ["bash", "-c", "apt list --upgradable 2>/dev/null"] (no sudo)
• Status: ✅ matches
• Notes: Read-only. 2>/dev/null suppresses apt's "WARNING: apt does not have a stable CLI interface" notice on stderr. No sudo required.

AptHistoryList — Show recent apt transaction history

• Canonical: grep -A 4 '^Start-Date' /var/log/apt/history.log | tail -n 80
• SysKnife argv: ["bash", "-c", "grep -A 4 '^Start-Date' /var/log/apt/history.log | tail -n 80"] (no sudo)
• Status: ✅ matches
• Notes: Read-only file inspection of /var/log/apt/history.log. Returns the 80 most recent log lines, covering the last several Start-Date transaction blocks, for auditing what was installed/removed/upgraded.

ConfigureUnattendedUpgrades — Enable or disable automatic security updates

• Canonical: no single canonical CLI — SysKnife ships a helper script
• SysKnife argv: ["sudo", "/usr/lib/sysknife/unattended-upgrades-edit", "--enable"] or --disable — param: enabled (bool)
• Status: ✅ implemented via bundled helper
• Notes: the helper (packaging/sysknife-unattended-upgrades-edit) takes no free-form input — it writes one of two fixed file contents to /etc/apt/apt.conf.d/20auto-upgrades (no injection surface), and when enabling, first ensures unattended-upgrades is installed. Risk: High — toggles whether the host auto-applies security updates unattended.

snap

SnapInstall — Install snap (with auto-hold)

• Canonical (with hold): sudo sh -c "snap install --channel=<ch> <name> && snap refresh --hold <name>"
• Canonical (auto-update allowed): sudo snap install --channel=<ch> <name>
• SysKnife argv (auto_update=false): ["sudo", "sh", "-c", "snap install --channel=<ch> <name> && snap refresh --hold <name>"]
• SysKnife argv (auto_update=true): ["sudo", "snap", "install", "--channel=<ch>", "<name>"]
• Status: ✅ matches
• Notes: --channel format is <track>/<risk> (e.g., latest/stable, beta). Default channel when none specified by SysKnife: stable (passed as --channel=stable). The hold-after-install pattern is correct: snap refresh --hold <name> pins the snap indefinitely. Exit code 0 = success.
• Classic confinement: --classic is required for snaps with classic confinement (e.g., VS Code: snap install --classic code). SnapInstall itself does not expose a classic flag — that case is handled by a separate action, SnapClassicInstall (see below), rather than a flag on this one.

SnapRemove — Remove snap

• Canonical: sudo snap remove <name>
• Canonical (purge user data too): sudo snap remove --purge <name>
• SysKnife argv: ["sudo", "snap", "remove", "<name>"]
• Status: ✅ matches (basic remove)
• Notes: Without --purge, user data in ~/snap/<name>/ is archived as a snapshot, not deleted. SysKnife does not expose --purge; this is a deliberate conservative choice (data is recoverable). Not a bug.

SnapRefresh — Update snap(s)

• Canonical (one snap): sudo snap refresh <name>
• Canonical (all snaps): sudo snap refresh
• SysKnife argv: ["sudo", "snap", "refresh", "<name>"] or ["sudo", "snap", "refresh"]
• Status: ✅ matches

SnapHold — Prevent auto-refresh

• Canonical: sudo snap refresh --hold <name>
• SysKnife argv: ["sudo", "snap", "refresh", "--hold", "<name>"]
• Status: ✅ matches
• Notes: --hold without a =<duration> argument defaults to forever. Confirmed in Snapcraft docs: "If no duration is specified, the time duration defaults to forever." This is correct for the SysKnife intent.

SnapUnhold — Re-enable auto-refresh

• Canonical: sudo snap refresh --unhold <name>
• SysKnife argv: ["sudo", "snap", "refresh", "--unhold", "<name>"]
• Status: ✅ matches
• Notes: Both --hold and --unhold are flags on snap refresh, not separate subcommands. SysKnife correctly uses this form.

SnapList — List installed snaps

• Canonical: snap list
• SysKnife argv: ["snap", "list"] (no sudo)
• Status: ✅ matches
• Output columns: Name Version Rev Tracking Publisher Notes
  • Notes field can contain: classic, disabled, -, or hold info.
  • No --json flag exists in snapd as of 24.04. Parsing is tab/space aligned.

SnapInfo — Show snap details

• Canonical: snap info <name>
• SysKnife argv: ["snap", "info", "<name>"] (no sudo)
• Status: ✅ matches
• Output: Multi-section human-readable; includes name, summary, publisher, available channels, installed revision, and tracking channel.

SnapRevert — Revert to the previous revision

• Canonical: sudo snap revert [--revision=<rev>] <name>
• SysKnife argv: ["sudo", "snap", "revert", "<name>"] — param: name
• Status: ✅ matches (basic revert, no explicit --revision)
• Notes: Rolls the snap back one revision; the reverted-from revision is preserved on disk and the revert itself can be undone with SnapRefresh. Risk: Medium.

SnapClassicInstall — Install with classic confinement

• Canonical: sudo snap install --classic <name>
• SysKnife argv: ["sudo", "snap", "install", "--classic", "<name>"] — param: name
• Status: ✅ matches
• Notes: Separate action from SnapInstall rather than a flag on it (see the classic-confinement note above). Classic-confined snaps get full system access with no sandbox, so this carries more risk than a sandboxed install — Risk: Medium.

ufw

UfwEnable — Enable firewall

• Canonical: sudo ufw --force enable
• SysKnife argv: ["sudo", "ufw", "--force", "enable"]
• Status: ✅ matches
• Notes: --force suppresses the interactive "Proceed with operation (y|n)?" prompt. Required for daemon/non-interactive invocation. The man page confirms: "SSH administrators should use --force enable when enabling remotely." Exit code 0 = success.

UfwDisable — Disable firewall

• Canonical: sudo ufw disable
• SysKnife argv: ["sudo", "ufw", "disable"]
• Status: ✅ matches
• Notes: disable does not prompt; --force is not needed here (only enable and reset have interactive prompts in the default ufw implementation).

UfwAllow — Allow traffic

• Canonical: sudo ufw allow <port_or_service>
• Extended canonical: sudo ufw allow <port>/<proto> or sudo ufw allow from <ip> to any port <port>
• SysKnife argv: ["sudo", "ufw", "allow", "<port_or_service>"]
• Status: ✅ matches (simple form)
• Notes: The simple form (ufw allow 22, ufw allow 22/tcp, ufw allow OpenSSH) covers the most common cases. SysKnife does not currently expose direction (in/out) or source IP filtering — not a bug for the current scope.

UfwDeny — Deny traffic

• Canonical: sudo ufw deny <port_or_service>
• SysKnife argv: ["sudo", "ufw", "deny", "<port_or_service>"]
• Status: ✅ matches

UfwReset — Reset to defaults

• Canonical: sudo ufw --force reset
• SysKnife argv: ["sudo", "ufw", "--force", "reset"]
• Status: ✅ matches
• Notes: --force suppresses the "Proceed with operation?" prompt. reset disables ufw AND removes all rules. Backup copies of old rules are written to /etc/ufw/*.rules.YYYYMMDD_HHMMSS — SysKnife does not document this side effect but it is not a functional issue.

UfwStatus — Show firewall status

• Canonical: sudo ufw status verbose
• SysKnife argv: ["sudo", "ufw", "status", "verbose"]
• Status: ✅ matches
• Notes: verbose adds default policies (incoming/outgoing/routed) to the output. Without verbose, ufw status shows only active rules and the Enabled/Disabled state. status numbered adds rule numbers, which is how a user finds the rule_number argument for UfwDeleteRule.
• Output format: Plain text, columns: To Action From with directional qualifiers (ALLOW, ALLOW IN, ALLOW OUT). No JSON output available.
• sudo required: Yes — ufw reads privileged iptables state.

UfwDeleteRule — Delete a rule by number

• Canonical: sudo ufw --force delete <rule_number>
• SysKnife argv: ["sudo", "ufw", "--force", "delete", "<rule_number>"] — param: rule_number (positive integer from ufw status numbered)
• Status: ✅ matches
• Notes: --force suppresses the confirmation prompt, same as UfwEnable / UfwReset. Risk: High — a mistaken deletion can expose services or drop needed traffic. Rejects rule_number == 0 (ufw rule numbers are 1-based).

UfwLimit — Rate-limit connections

• Canonical: sudo ufw limit <port_or_service>
• SysKnife argv: ["sudo", "ufw", "limit", "<target>"] — param: target (port number, port/proto, or app profile name, e.g. ssh)
• Status: ✅ matches
• Notes: Blocks IPs making more than 6 connections within 30 seconds — the standard ufw brute-force mitigation, commonly applied to SSH (port 22). Risk: High — can inadvertently rate-limit legitimate traffic under high-connection workloads.

netplan

NetplanGetConfig — Read current configuration

• SysKnife argv: ["find", "/etc/netplan", "-maxdepth", "1", "-name", "*.yaml", "-print", "-exec", "cat", "{}", "+"]
• Canonical alternative: sudo netplan get (merges all YAML from /etc/netplan/, /lib/netplan/, and /run/netplan/ into a single output)
• Status: ⚠ functional divergence — not wrong, but differs from canonical
• Notes: SysKnife shells out to find directly (no bash -c), printing each matched file's path followed by its contents. netplan get returns a single merged representation instead. The find approach:
  1. Does not merge configs across /lib/netplan/ and /run/netplan/ overlay paths.
  2. May include YAML comments; netplan get output is stripped and canonical.
  3. Surfaces three distinct states instead of collapsing them: a missing/ permission-denied /etc/netplan makes find exit non-zero with a stderr diagnostic, while a directory with no *.yaml files exits 0 with empty stdout. (An earlier version shelled out to bash -c "cat /etc/netplan/*.yaml 2>/dev/null || echo 'no netplan files found'", which collapsed all three states into one fake-success exit 0 — fixed by moving to find directly.)
  • For read-only inspection, this is harmless and does not require sudo for files readable by the daemon user. netplan get requires sudo on most Ubuntu installs because /etc/netplan/ is root-owned mode 600.
  • Not a blocking bug. Document as a known divergence.

NetplanApply — Apply configuration

• Canonical: sudo netplan apply
• SysKnife argv: ["sudo", "netplan", "apply"]
• Status: ✅ matches
• Notes: netplan apply immediately reconfigures network interfaces. Can terminate SSH sessions if IP changes. netplan try (with a rollback timeout) is safer for interactive use but requires a TTY and is intentionally not implemented as a daemon action (documented in the source). Exit code 0 = success; non-zero on YAML parse or backend errors.

NetplanSet — Set a single netplan key

• Canonical: sudo netplan set <key>=<value>
• SysKnife argv: ["sudo", "netplan", "set", "<key>=<value>"] — params: key (e.g. ethernets.eth0.dhcp4), value
• Status: ✅ matches
• Notes: key=value is passed as a single argument with no shell involved; validated_safe_arg (executor boundary) rejects spaces in value, so quoting a multi-word value would only inject literal quote bytes. Risk: High — modifies the active netplan configuration in-memory. Run NetplanApply afterward to activate the change.

NetplanGenerate — Regenerate backend config without applying

• Canonical: sudo netplan generate
• SysKnife argv: ["sudo", "netplan", "generate"]
• Status: ✅ matches
• Notes: Risk: Medium. Regenerates systemd-networkd / NetworkManager backend config files from the current netplan YAML but does not reload interfaces — safe to use as a dry-run check before NetplanApply.

Not implemented — netplan get

• netplan get: Returns merged netplan config as YAML across /etc/netplan/, /lib/netplan/, and /run/netplan/. More canonical than the find-based NetplanGetConfig, which only reads /etc/netplan/. See the NetplanGetConfig divergence above — this is a coverage gap, not a bug in existing code.

distrobox

DistroboxList — List containers

• Canonical: distrobox list
• SysKnife argv: ["distrobox", "list"]
• Status: ✅ matches
• Notes: No sudo required (user-namespace containers). --no-color flag is available for cleaner parsing but not used — not a bug. Output includes: container name, status (Up/Exited), image. No JSON output.

DistroboxCreate — Create container

• Canonical: distrobox create --yes --name <name> --image <image>
• SysKnife argv: ["distrobox", "create", "--yes", "--name", "<name>", "--image", "<image>"]
• Status: ✅ matches (was missing --yes until 2026-04-26 — fixed in PR that lands distrobox_create_includes_yes_flag test)
• Why --yes matters: Without --yes / -Y, distrobox-create prompts the user to confirm pulling the image if it is not already cached locally. In a daemon context with no TTY, this prompt hangs indefinitely. The canonical non-interactive form requires --yes (documented as "non-interactive, pull images without asking").
  • Fix: Add "--yes" to the argv: ["distrobox", "create", "--name", "<name>", "--image", "<image>", "--yes"]

DistroboxRemove — Remove container

• Canonical: distrobox rm --force <name>
• SysKnife argv: ["distrobox", "rm", "--force", "<name>"]
• Status: ✅ matches
• Notes: --force / -f sets both force=1 and non_interactive=1 in distrobox internals — no confirmation prompt. The flag name --force (not -f) is correct and matches the official docs. No sudo required unless the container was created with --root.

Bug list

Divergences (not bugs, but worth knowing)

Live validation checklist

Run these commands inside an Ubuntu 24.04 VM to verify each action family's preconditions and post-state. All commands that need root are shown with sudo.

apt — checklist

# Precondition: verify apt-get is available and lock is free
which apt-get && apt-get --version
sudo fuser /var/lib/dpkg/lock 2>/dev/null && echo "LOCKED" || echo "free"

# AptUpdate
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get update
echo "exit=$?"

# AptInstall (use a small safe package)
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get install -y curl
dpkg -l curl | grep '^ii'    # should print one ii line

# AptHold / AptUnhold
sudo apt-mark hold curl
apt-mark showhold | grep curl
sudo apt-mark unhold curl
apt-mark showhold | grep curl && echo "BUG: still held" || echo "unhold ok"

# AptRemove / AptPurge (install something small first)
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get install -y hello
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get remove -y hello
dpkg -l hello | grep '^rc'   # rc = config files remain
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get purge -y hello
dpkg -l hello | grep '^un'   # un = not installed, no config files

# AptAutoremove
sudo env DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a apt-get autoremove -y
echo "exit=$?"

# AptSearch
apt-cache search curl | head -5

# AptShow
apt-cache show curl | grep '^Package:'

# AptListInstalled
dpkg -l | head -10
dpkg -l | grep '^ii' | wc -l   # count installed packages

snap — checklist

# Precondition: snapd running
snap version
systemctl is-active snapd

# SnapList
snap list   # columns: Name  Version  Rev  Tracking  Publisher  Notes

# SnapInstall (use hello-world, tiny and always available)
sudo snap install hello-world
snap list | grep hello-world   # should appear with Notes: -

# SnapHold
sudo snap refresh --hold hello-world
snap list hello-world | awk '{print $NF}'   # Notes column should show "held"

# SnapUnhold
sudo snap refresh --unhold hello-world
snap list hello-world

# SnapRefresh
sudo snap refresh hello-world
echo "exit=$?"

# SnapInfo
snap info hello-world | grep '^name:'

# SnapRemove
sudo snap remove hello-world
snap list | grep hello-world && echo "BUG: still listed" || echo "removed ok"

ufw — checklist

# Precondition: ufw installed
which ufw && ufw --version

# UfwStatus (initial state)
sudo ufw status verbose
echo "exit=$?"

# UfwEnable
sudo ufw --force enable
sudo ufw status | grep 'Status: active'

# UfwAllow
sudo ufw allow 8080/tcp
sudo ufw status verbose | grep 8080

# UfwDeny
sudo ufw deny 8081/tcp
sudo ufw status verbose | grep 8081

# UfwDisable
sudo ufw disable
sudo ufw status | grep 'Status: inactive'

# UfwReset (destructive — do last)
sudo ufw --force reset
sudo ufw status | grep 'Status: inactive'
# Verify backup files were created
ls /etc/ufw/*.rules.* 2>/dev/null || echo "no backup files (expected on fresh VM)"

netplan — checklist

# Precondition: netplan installed and config present
which netplan && netplan --version
ls /etc/netplan/

# NetplanGetConfig (SysKnife approach)
find /etc/netplan -maxdepth 1 -name '*.yaml' -print -exec cat {} +

# Canonical alternative
sudo netplan get

# NetplanGenerate (dry run — regenerates backend config, does not apply)
sudo netplan generate
echo "exit=$?"

# NetplanSet (in-memory change — run NetplanApply after to activate)
sudo netplan set ethernets.eth0.dhcp4=true

# NetplanApply (safe: only apply if no config change)
# WARNING: can disconnect SSH — run only on a console or via OOB access if changing IP
sudo netplan apply
echo "exit=$?"
# Verify network is still up
ip addr show

distrobox — checklist

# Precondition: distrobox and podman/docker installed
which distrobox && distrobox --version
which podman || which docker

# DistroboxList
distrobox list

# DistroboxCreate (always pass --yes — without it the daemon hangs without a TTY).
# The canonical form is:
distrobox create --name test-sysknife --image ubuntu:24.04 --yes
echo "exit=$?"

# Verify created
distrobox list | grep test-sysknife

# DistroboxRemove
distrobox rm --force test-sysknife
echo "exit=$?"

# Verify removed
distrobox list | grep test-sysknife && echo "BUG: still listed" || echo "removed ok"

SysKnife vs. the alternatives

Letting an AI operate a Linux host is now a real category, and several tools address it. This page is an honest map of that space and where SysKnife sits in it. The short version: most tools either filter shell strings or watch after the fact. SysKnife removes the shell string entirely, gates execution with a hard interlock, and produces an audit trail anyone can verify with a public key.

At a glance

The two things nobody else pairs: public-key-verifiable audit and automatic rollback. Everything below explains why those matter.

Typed actions beat string allowlists

The common design is to let the model produce a shell command and then gate that string with an allowlist or regex. Published red-team work ("GuardFall") found that the large majority of agents can talk their way past such guards — the guard is filtering a language rich enough to hide intent.

SysKnife takes the string out of the loop. The model proposes typed actions from a fixed catalog; the daemon builds the command internally and is the sole executor. There is no shell string to smuggle a payload into. See Typed Actions for the mechanism.

Public-key audit beats symmetric audit

An audit trail is only worth what its verification proves. SysKnife signs the hash chain with Ed25519: verification needs only the public key, so a third party — an auditor, a customer, a court — can confirm the log is intact and was produced by the holder of the private key, and the signer cannot later deny it. See The Audit Chain.

Symmetric schemes (HMAC-SHA256, as used by AIShell-Gate) use one shared secret for both signing and verifying. Whoever can verify can also forge, so a "valid" log convinces no one who did not already hold the secret. It is a good integrity check for yourself; it is not non-repudiable evidence for anyone else.

Automatic rollback

On atomic hosts (Fedora/Silverblue via rpm-ostree), a failed change is automatically reverted to the prior deployment. No other tool in this space does this. Be aware of the scope limit: on Ubuntu/apt there is no automatic rollback yet — SysKnife still gates and audits, but does not undo package changes. That scope limit is enforced, not just documented: every catalogued action's rollback_available flag is tested against the actual rollback mechanism, so a Debian-family action (PPAs, netplan, GRUB kargs, Ubuntu Pro attach/detach included) cannot claim automatic rollback it doesn't have, even when it happens to have an obvious manual inverse. See Automatic Rollback.

Open beats closed — especially here

A security and audit tool you cannot read is a contradiction: you are asked to trust the thing whose entire job is to be trustworthy. SysKnife is MIT, the LACS protocol it implements is CC0. You can read the gate, self-host it, fork it, and verify every claim on this page against the source.

The field, honestly

• AIShell-Gate — the closest peer: typed JSON commands, a hash-chain audit, a confirmation model, and an executor-separation design much like SysKnife's daemon boundary. But it is proprietary and license-required, and its audit is HMAC-SHA256 (symmetric; by its own documentation, with no persistent key set, post-hoc verification is not possible). No automatic rollback.
• gate-oc-audit (Apache-2.0) — a genuinely good audit-only layer: tamper-evident records via a Merkle tree, optionally anchored to an external network. It validates that this category matters. It does not gate or execute and does not roll back — it observes.
• MCP traffic gateways (agentgateway, MCPX, Obot, mcp-firewall, and others) — these govern the protocol stream: routing, authn/z, threat detection, and audit at the proxy. They are complementary to SysKnife, not competitive: none execute typed actions on the host or roll changes back. You can run SysKnife behind one.
• generic mcp-shell servers — hand the model a shell with an allowlist. Lowest friction, weakest guarantee (see the GuardFall result above).

When SysKnife is the wrong tool

Being honest about the fit:

• You want the AI to run arbitrary commands and accept the risk — a generic mcp-shell server is lower friction. SysKnife's typed catalog is finite by design; operations outside it are not (yet) available.
• You only need observability, not enforcement — a pure audit layer like gate-oc-audit is lighter.
• Your fleet is Ubuntu/apt and you require automatic rollback today — that is roadmap, not shipped (gate + audit work now).

If you want the AI to change a Linux box while a human stays in control and every action is provably recorded, that is exactly what SysKnife is for.

Accepted cargo audit advisories

cargo audit is run in CI (.github/workflows/ci.yml, security-audit job). It fails the build on any vulnerability. The advisories listed here are the accepted unmaintained and unsound warnings — every one of them enters the dependency graph only through the Tauri desktop shell (sysknife-shell) and its GTK3 / webview stack.

None of these crates are reachable from the release-published crates — sysknife-proto, sysknife-core, sysknife-types, sysknife-brain, sysknife-daemon, or sysknife-cli. The privileged daemon and the CLI (the security-sensitive trust boundary) do not link any of them. Verify with:

cargo tree -e no-dev -i <crate>        # shows the path is via tauri/gtk only
cargo audit                             # full current report

Unsound (informational)

RUSTSEC-2026-0097 (rand 0.7.3, reached via phf → selectors → kuchikiki → tauri-utils) used to be listed here and --ignored in CI. The crate has since left the lockfile — only rand 0.9.4 and 0.10.2 are built — so cargo audit now runs with no ignore at all. The ignore was removed rather than left in place: an ignore for a crate that is not built protects nothing, and would silently re-accept the advisory if a future dependency bump pulled 0.7.3 back in.

Unmaintained

Policy

• Vulnerabilities (RUSTSEC advisories of type vulnerability) are never accepted here — they fail CI and must be fixed or the dependency dropped.
• Unmaintained advisories are reported as non-fatal warnings by default; they are left visible (not globally ignored) so the team notices if the set grows. This file is the record of the currently-accepted set and why.
• Re-evaluate the GTK3 entries when Tauri migrates its Linux backend off the GTK3 binding stack; re-evaluate rand when the webview dep chain bumps it.

ADR 0001: System Boundaries

Status

Accepted.

Context

SysKnife exists to let an agent perform real Linux administration without granting it arbitrary shell access or root.

Decision

SysKnife uses three separate roles:

• sysknife-brain plans
• sysknife-shell presents and collects approval
• sysknife-daemon executes

The daemon is the only privileged executor. The brain is never allowed to mutate the system directly.

Consequences

• The trust boundary is simple and auditable.
• The shell remains a client instead of a second privileged runtime.
• The daemon can enforce policy, preview, transaction logging, and rollback in one place.
• The project can grow without collapsing into an unsafe command runner.

ADR 0002: Brain Provider Layer

Status

Accepted.

Context

sysknife-brain needs to talk to an LLM to plan system actions. Different users will have different LLM setups: some will use the Anthropic API, others will run a local model via Ollama, and future contributors may want to add other providers.

Additionally, the API key must never be visible outside sysknife-brain to prevent accidental logging or exposure through sysknife-shell.

Decision

sysknife-brain exposes a single LlmProvider trait:

#![allow(unused)]
fn main() {
#[async_trait]
pub trait LlmProvider: Send + Sync {
    async fn complete(
        &self,
        system: &str,
        messages: &[Message],
        tools: &[ToolDefinition],
        max_tokens: u32,
    ) -> Result<Completion, ProviderError>;
}
}

Two implementations ship with the crate:

• AnthropicProvider — POST /v1/messages with input_schema tool definitions (Anthropic wire format).
• OllamaProvider — POST /v1/chat/completions with OpenAI function-calling format (parameters field, role: "tool" messages).

LlmPlanner::from_config(BrainConfig, Box<dyn StateClient>) builds the provider from environment-derived config, keeping the API key inside sysknife-brain. ProviderConfig and BrainConfig.provider are pub(crate) so the shell cannot read credentials.

BrainConfig::from_env() validates inputs at the boundary:

• SYSKNIFE_BRAIN_MAX_TURNS must be a positive integer when set.
• ANTHROPIC_API_KEY must be non-empty when provider is anthropic.

Alternatives Considered

• Single hard-coded Anthropic client — ruled out; excludes local models and makes the project inaccessible to contributors without API keys.
• Runtime plugin loading (.so / WASM) — too complex for v0; the two-implementation trait covers the main use cases.

Consequences

• Adding a new provider requires implementing LlmProvider and a new ProviderConfig variant; no changes to LlmPlanner.
• The shell layer cannot accidentally read or log API keys.
• Integration tests use MockProvider — no network calls in CI.

Amendment (2026-07-23)

The original decision above described the two providers that shipped at the time (Anthropic, Ollama). The LlmProvider trait design anticipated more — see "Consequences" — and the provider set has since grown to eight: anthropic, ollama, openai, gemini, groq, deepseek, mistral, and xai (see crates/sysknife-brain/src/config.rs). Each new provider was a pure addition — a new ProviderConfig variant and a require_api_key-backed from_env() arm — with no change to LlmPlanner or the trait itself, confirming the extensibility this ADR designed for. The original Decision and Alternatives Considered sections above are left as written for historical context; this amendment records the outcome.

ADR 0003: IPC Wire Protocol

Status

Accepted.

Context

The daemon and shell need a local IPC protocol. The codebase already has a .proto file (sysknife-proto) with message definitions for RequestEnvelope, PreviewEnvelope, ResultEnvelope, and TransactionRecord, and a bind_unix_listener helper. The question is what framing and encoding to use over the Unix domain socket.

Candidates considered:

1. gRPC over Unix socket — uses the proto definitions directly with tonic. Full service interface. Requires HTTP/2 and a service block in the proto.
2. Length-prefixed protobuf — binary, compact. Requires a custom dispatcher (no HTTP/2). Harder to inspect without tooling.
3. Length-prefixed JSON — 4-byte LE u32 length + UTF-8 JSON body. Uses the same Rust structs (sysknife-types) already tested for serialization. Human-readable, easy to debug with socat.

Decision

Use length-prefixed JSON (option 3) for the Unix socket protocol.

Each message carries a "type" discriminant string so the dispatcher can route without a full decode. The sysknife-types structs are reused as-is — no additional generated code or build-time proto compilation beyond what already exists.

The proto definitions in sysknife-proto are kept for potential future use (e.g., a networked or gRPC-based control plane) but are not the primary on-wire encoding for local IPC.

Consequences

• No new build-time dependencies for the daemon or shell.
• Messages are inspectable with standard tools (socat, nc, jq).
• A framing crate is not needed — tokio::io::AsyncReadExt::read_exact is sufficient.
• If a binary protocol is needed later (performance, cross-language clients), the socket path changes and both sides are updated together. There are no external clients to coordinate with.

ADR 0004: Per-distro Prompt Dispatch

Status

Accepted (merged in PR #203, 2026-04-25).

Context

SysKnife gained Ubuntu/Debian action support in Phase 2b. The action catalogue split into two mostly non-overlapping sets:

• Fedora-family: AddLayeredPackage, RemoveLayeredPackage, rpm-ostree deployment controls, flatpak, toolbox, firewalld, …
• Debian-family: AptInstall, AptRemove, AptUpdate, snap, distrobox, ufw, netplan, …

The original single-distro prompt listed all action names in one block. After adding Debian actions this caused two problems:

1. Prompt confusion: the model saw both AddLayeredPackage and AptInstall for every intent and had to infer which to use from context. In practice, GPT-4o and Claude sometimes proposed the wrong family — AptInstall on Silverblue, or AddLayeredPackage on Ubuntu — especially for compound intents that did not name the package manager explicitly.

2. Context window waste: every planning call carried ~40% action names that could never legally be executed on the current host.

The E2E story suite (recorded at the time as 65/65 passing on Ubuntu 24.04 with gpt-4.1 — see the correction note at the end of this ADR) validated that per-distro isolation fixes (1) and reduces prompt size meaningfully.

Decision

build_system_prompt dispatches to one of three pure render functions based on distro_hint.family:

distro_hint.family == "fedora"  →  render_fedora_prompt
distro_hint.family == "debian"  →  render_debian_prompt
_                               →  render_generic_prompt

Each render function concatenates:

1. Shared const blocks: role, the single-planning-rule, cross-distro action catalogue, and the six worked examples (A–F).
2. Per-distro const blocks: distro header (with detected version), distro-specific action catalogue, risk overrides, selection rules, and action parameter reference.

The FEDORA_ONLY_ACTIONS and DEBIAN_ONLY_ACTIONS string-slice constants back safety-fence unit tests that assert no cross-contamination at the type level.

The DistroHint is provided by sysknife-core's detect_distro() function which reads /etc/os-release at startup. If detection fails, distro_hint is None and render_generic_prompt is used.

Consequences

• The model cannot propose a Fedora-specific action on a Debian host (or vice versa) because the action name simply does not appear in its context window.
• Adding a new distro family requires: a new render_* function, a new DISTRO_FAMILY_* constant, a dispatch arm in build_system_prompt, and a *_ONLY_ACTIONS slice for the safety fence. This is intentionally mechanical — each step has a test to confirm it.
• Prompt size decreases by roughly 20–25% for Fedora hosts and 15–20% for Debian hosts compared to the pre-dispatch single-prompt.
• The generic fallback deliberately omits all distro-specific actions. A host with an unrecognised /etc/os-release gets cross-distro-only planning until the user adds a manual distro_hint override in ~/.config/sysknife/config.toml.

References

• PR #203 — "refactor(prompt): per-distro dispatch"
• crates/sysknife-brain/src/prompt.rs — implementation
• docs/architecture.md — prompt construction overview
• docs/research/prompt-composition-patterns.md — survey of dynamic prompt patterns that informed this design (dynamic middleware, template composition, conditional blocks)

Correction (2026-08-05)

The "65/65" figure quoted above was never backed by a run: no story set of that size has existed, and the Ubuntu family contains 50 stories. Later runs were observed at 46/50 on 22.04 and 45/50 on 24.04 with openai/gpt-oss-120b, but those numbers come from run logs that were never committed, so they are not artifact-backed either and are recorded here only as history. Figures fit to publish live in tests/evidence/story-runs/, written by the run itself. The decision this ADR records — structural per-distro prompt isolation — is unaffected, and was independently confirmed later: the two Fedora actions that still reached Ubuntu plans came from the tool schema, which this ADR did not cover, not from the prompt. Published figures now derive from tests/evidence/ and are enforced by scripts/check_evidence_claims.py.

Follow-up (2026-08-13)

The "50 stories" above was the family size at the time. It is 79 now: GetHostState — the action this ADR's fence required somebody to introduce, because naming Fedora's GetSystemState on an apt host is what the fence forbids — had no story at all, and neither did 28 other Debian-only actions. Every Debian-only action now has one, and the family runs 79/79 on all three LTSes.

Two of the new stories say something about this decision that the fence itself does not:

• The four Ubuntu*Flatpak actions build byte-identical argv to their un-prefixed twins, which action_family.rs deliberately leaves unfenced. So the Debian prompt names only the prefixed set while the tool schema offers both, and the choice has no effect on the host. Measured: the model picks the un-prefixed name 3 times in 4. Those stories accept either and log which, because asserting a distinction the host cannot observe is not a gate.
• ConfigureUnattendedUpgrades is described in the tool schema and named in none of the Debian prose blocks. Story 126 is therefore a measurement of whether the schema description alone is enough to steer. It is.

Developer Guide

Welcome to SysKnife. This guide gets you from zero to a running dev environment and covers everything you need to contribute confidently.

Read First

• Architecture overview — understand the four-crate structure and the trust boundary before writing code
• ADR 0001: System boundaries
• ADR 0002: Brain provider layer
• ADR 0003: IPC wire protocol

Prerequisites

Required. Without these nothing builds:

Required to reproduce the docs-and-hygiene job, which is one of the five checks main requires:

Required only for the jobs that name them:

Install cargo-nextest before you trust a green run. scripts/ci-local.sh reports WARN ... SKIPPED rather than failing when it is missing, so the whole Rust test step silently does not run and the summary still ends ci-local: PASS.

This project uses npm. apps/sysknife-shell carries a package-lock.json and CI runs npm ci against it. There is no pnpm or yarn lockfile in the tree.

No API key is required to get started. SysKnife auto-detects a local Ollama instance (http://localhost:11434) when no cloud API key is set. If you do not have Ollama installed, you can still run all unit and integration tests without it.

Clone and Set Up

git clone https://github.com/lacs-project/sysknife
cd sysknife

# Install the git hooks (run once). This sets core.hooksPath to .githooks,
# which is what wires up the real pre-commit and pre-push gates.
scripts/ci-local.sh --install-hooks

# Install frontend dependencies
npm ci --prefix apps/sysknife-shell

Do not use pip install pre-commit && pre-commit install. This repository drives its hooks through core.hooksPath, so anything written into .git/hooks is ignored by Git and you would end up with no gate at all. .pre-commit-config.yaml is left over from an earlier setup and is not used.

Building

The workspace builds native code (TLS via aws-lc-sys/ring, SQLite via libsqlite3-sys), so a C compiler and linker are required on top of rustup: sudo apt-get install -y build-essential on Debian/Ubuntu. Without one the build stops at error: linker cc not found. cmake is not required.

# Build all Rust crates (fast, no linking of the Tauri app)
cargo build --workspace

# Build the Tauri app (includes the GUI)
npm run tauri --prefix apps/sysknife-shell -- build

Running Tests

These run in under 15 seconds and are required before every push:

# Rust unit + integration tests
cargo nextest run --workspace --locked

# TypeScript / React tests
npm test --prefix apps/sysknife-shell
npm exec --prefix apps/sysknife-shell -- tsc --noEmit

See docs/contributing/testing.md for the full test pyramid, including how to run the LLM-driven E2E stories on your workstation and in a Fedora Atomic VM.

Reproducing the Required Checks

main requires five checks. scripts/ci-local.sh runs most of them in one go and is the command to reach for first:

scripts/ci-local.sh          # everything it can run here
scripts/ci-local.sh --fast   # what the pre-push hook runs

Read its summary rather than its exit code. A tool it cannot find becomes a WARN ... SKIPPED line and the run still ends ci-local: PASS, so an absent cargo-nextest means the Rust suite never ran at all.

Use cargo nextest, not cargo test

CI and every script here run cargo nextest run --workspace --locked. Plain cargo test runs the whole binary's tests as threads in one process, and a few tests here set process-global environment variables, so it fails intermittently on main through no fault of your branch. nextest gives each test its own process and does not have that problem. If you have no choice, cargo test -- --test-threads=1 is stable.

The live Postgres contract

postgres-contract is the only required check that needs a server, and it is the one people assume they cannot run. You can. CI starts postgres:17-alpine as a service container; do the same locally:

podman run -d --name sysknife-pg -p 55987:5432 \
  -e POSTGRES_USER=sysknife -e POSTGRES_PASSWORD=sysknife \
  -e POSTGRES_DB=sysknife_test docker.io/library/postgres:17-alpine

export SYSKNIFE_TEST_POSTGRES_URL=postgres://sysknife:sysknife@127.0.0.1:55987/sysknife_test
export SYSKNIFE_REQUIRE_POSTGRES=1
cargo test -p sysknife-daemon --test postgres_store --locked -- --include-ignored

podman rm -f sysknife-pg

Those tests are #[ignore] by default, so a normal run reports them as ignored rather than passing over a database that was never there. SYSKNIFE_REQUIRE_POSTGRES=1 without a URL panics on purpose: a misnamed variable used to report success.

Podman works rootless and needs no daemon. Docker works too; on some hosts its published ports reset the connection, in which case use podman.

The hygiene guards

docs-and-hygiene runs about two dozen host-side scripts under tests/release/ and scripts/. They are plain shell and Python, they need no build, and each one runs on its own in well under a second:

bash tests/release/public-claims.test.sh
bash scripts/check_release_versions.sh

If one goes red with no output, that is #347, not you.

Running a contributor's branch safely

Reviewing someone else's PR means running their code. Do it in a container with no network and nothing from your home directory mounted:

podman run --rm --network=none -v "$PWD:/repo:z" -w /repo \
  docker.io/library/python:3-slim bash -c 'bash tests/release/public-claims.test.sh'

For Rust, give the container a throwaway copy of your cargo home with an overlay mount so a build script cannot write to yours, and point CARGO_HOME at it. The official rust image sets CARGO_HOME=/usr/local/cargo, so a mount alone is ignored:

podman run --rm --network=none -v "$PWD:/repo:z" -v "$HOME/.cargo:/cargo:O" \
  -w /repo -e CARGO_HOME=/cargo -e CARGO_TARGET_DIR=/repo/.container-target \
  -e CARGO_NET_OFFLINE=true docker.io/library/rust:1-slim \
  cargo test -p sysknife-cli --bins --offline

sysknife-cli is a binary crate, so its unit tests need --bins; --lib finds no target.

Running the Full Stack Locally

You need two terminals.

Terminal 1 — daemon

# Binds $SYSKNIFE_LISTEN_URI, else $XDG_RUNTIME_DIR/sysknife/daemon.sock
# (last resort /tmp/sysknife-$UID.sock). The CLI resolves the same default,
# so `sysknife doctor` works without setting a socket env var.
# Privileged system actions (rpm-ostree, useradd, etc.) require root.
# For development you can run without root — read-only queries still work.
cargo run -p sysknife-daemon

Terminal 2 — shell (GUI)

cd apps/sysknife-shell
npm run tauri --prefix apps/sysknife-shell -- dev

The shell opens as a desktop window. Type an intent and the daemon responds. The LLM is auto-detected from your environment.

Running the E2E Stories on Your Dev Machine

tests/e2e/dev-stories.sh runs the 7 read-only user stories without a VM. It validates that the LLM proposes the correct typed plan — it does not execute the actions against your host.

# With an Anthropic key
ANTHROPIC_API_KEY=sk-ant-... tests/e2e/dev-stories.sh

# With an OpenAI key
OPENAI_API_KEY=sk-proj-... tests/e2e/dev-stories.sh

# With local Ollama (must have a tool-capable model pulled)
tests/e2e/dev-stories.sh

# Specific stories only
OPENAI_API_KEY=sk-... tests/e2e/dev-stories.sh 3 6 7

Run this tier after any change to crates/sysknife-brain/src/prompt.rs or the planning tools. See the testing guide for full details.

Inspecting the IPC Protocol

The daemon speaks length-prefixed JSON over a Unix socket. You can poke it manually without the GUI:

cargo run -p sysknife-daemon &
socat - UNIX-CONNECT:"$XDG_RUNTIME_DIR/sysknife/daemon.sock"

Type or paste a JSON message (with a 4-byte LE length prefix). This is useful for debugging the dispatcher or previewing action output.

Pre-commit Hooks

Pre-commit runs on every git commit. Run all hooks manually before pushing:

pre-commit run --all-files

Hooks included:

Intentionally excluded from pre-commit (they run in CI instead): cargo clippy (20–30 s), cargo nextest run (minutes), vitest (minutes).

Configuration

Config file: ~/.config/sysknife/config.toml (created manually, optional):

[daemon]
socket   = "/run/sysknife/daemon.sock"    # raw path, not URI
database = "/var/lib/sysknife/daemon.sqlite"

[llm]
provider   = "ollama"                 # ollama | anthropic | openai | gemini | groq | deepseek | mistral | xai
model      = "llama3.2:3b"
ollama_url = "http://localhost:11434"
max_turns  = 10

Config file values act as defaults. Environment variables always win.

User Preferences

SysKnife remembers user preferences in ~/.config/sysknife/prefs.md. The planner injects them at the start of each plan_intent() call.

Preferences are user-stated intentions that inform planning decisions. Do not store system facts as preferences — those are queried live.

Manage preferences through natural language:

• "Remember that I always prefer vim-enhanced over vim"
• "Forget my vim preference"

Or edit ~/.config/sysknife/prefs.md directly. Maximum 10 KB; SysKnife rejects passwords, API keys, and tokens automatically.

Transaction History

The ListJobHistory action and query_job_history planning tool expose the daemon's SQLite transaction log. Ask "what has SysKnife done recently?" or "did my update succeed?" and the planner queries the log directly.

ListJobHistory is Observer-level (read-only, no approval required).

Repository Layout

crates/
  sysknife-brain/     LLM planner, provider adapters, safety fence
  sysknife-types/     Shared domain types (CallerRole, RiskLevel, JobState, …)
  sysknife-core/      Config loading, shared constants
  sysknife-daemon/    Privileged executor, 189 actions with an `ActionSpec`,
                      IPC, rollback, SQLite
  sysknife-proto/     Protobuf definitions (future use)
apps/
  sysknife-shell/     Tauri + React GUI
tests/
  e2e/
    dev-stories.sh  Run E2E stories on any Linux host (uses sysknife --dry-run --json)
    atomic-vm.sh  Manage a Silverblue QEMU/KVM VM for full E2E
docs/
  adr/            Architectural decision records
  contributing/   Testing guide

CI

CI runs on every pull request and push to main.

Run the Rust checks locally before pushing:

cargo fmt --all --check
cargo clippy --workspace --all-features --locked -- -D warnings
cargo nextest run --workspace --locked

Running CI locally

scripts/ci-local.sh mirrors the runnable jobs from .github/workflows/ci.yml (rust, frontend, hygiene, security, and the optional postgres-contract job) so you catch failures before pushing, without spending GitHub Actions minutes:

# Full run — everything CI runs, including the optional Postgres contract
# test if docker/podman is available (or SYSKNIFE_TEST_POSTGRES_URL is set)
scripts/ci-local.sh

# Fast subset — rust fmt/clippy/nextest + frontend tsc/vitest only
scripts/ci-local.sh --fast

# Skip the postgres-contract job even if a container runtime is available
scripts/ci-local.sh --no-postgres

It detects which tools are installed first: cargo and node are required (missing either is a hard failure with an install link); an optional linter that's missing (cargo-nextest, cargo-audit, markdownlint-cli2, markdown-link-check, yamllint, shellcheck) just prints a warning with an install hint and skips that one check. Every check still runs even after an earlier one fails — a PASS/FAIL/WARN/SKIP summary prints at the end, and the script exits non-zero only if something in the summary actually failed.

Pre-push hook

scripts/ci-local.sh --install-hooks runs git config core.hooksPath .githooks, which wires up .githooks/pre-push to run scripts/ci-local.sh --fast automatically before every git push and block the push on failure. Bypass a single push with git push --no-verify; undo the hook entirely with git config --unset core.hooksPath.

.githooks/ already ships a pre-commit hook (cargo fmt --all --check + cargo nextest run --workspace --locked) alongside the new pre-push one. Both are opt-in via the same core.hooksPath setting. Note that core.hooksPath is a single switch: pointing it at .githooks means Git stops looking in .git/hooks, so it supersedes hooks installed by the pre-commit framework (see Pre-commit Hooks above) — use one mechanism or the other, not both, per clone.

Full workflow replay

ci-local.sh runs the same commands as CI, but not inside the same container/runner image, so it cannot catch environment-specific breakage. For an exact, Docker-based replay of the GitHub Actions workflow (all jobs, the real ubuntu-latest image), use act instead.

Working Style

• keep changes small and reviewable
• keep behavior typed and explicit
• keep the daemon as the only privileged executor
• update docs when user-facing behavior changes
• add or update tests for every behavior change
• write the failing test first; verify it fails before writing code

Quality Bar

Before merging, a change should be:

• understandable without reading every dependency
• covered by deterministic tests
• documented if it changes user-visible behavior
• safe by default (fail closed, not open)
• consistent with the trust boundary (daemon is the only executor)

Testing SysKnife

This guide explains how to run SysKnife tests at every level — from the fast unit tests you run locally on every change to the VM-based end-to-end validation before a release.

Test pyramid

No single layer is enough on its own. Use the ones that match your change.

Running unit and integration tests (required)

These run on every CI build and must pass before merge:

# Rust
cargo fmt --all --check
cargo clippy --workspace --all-features --locked -- -D warnings
cargo nextest run --workspace --locked

# TypeScript / React
cd apps/sysknife-shell
npm ci
npm test
npm exec -- tsc --noEmit

Running stories on your dev machine (no VM required)

tests/e2e/dev-stories.sh runs the 7 read-only user stories directly on your workstation. It builds the daemon and test CLI, starts the daemon on /tmp/sysknife-daemon.sock, runs the stories, and then stops the daemon.

What it validates: the LLM proposes the correct plan (right action names, risk levels, parameters). It does not execute the actions — it tests plan structure only. This works on any Linux host because the story scripts check the JSON plan output, not the results of df, ps, etc.

LLM provider: the harness mirrors the product's BrainConfig::from_env, which auto-detects exactly two providers:

The other six providers are supported but never auto-detected: setting the key alone does nothing, because from_env only consults ANTHROPIC_API_KEY. Each needs SYSKNIFE_LLM_PROVIDER and its key:

This table listed three auto-detected providers and omitted four entirely, which is how a run with only GROQ_API_KEY exported silently fell through to the uninstalled Ollama fallback and failed all fifty stories with an error naming the wrong provider. tests/e2e/provider-parity.test.sh now derives the provider list and the default models from config.rs and fails when this table drifts again.

Override the model with SYSKNIFE_LLM_MODEL (leave it unset rather than empty: an empty value wins over the per-provider default).

# Run the default read-only stories with an Anthropic key
ANTHROPIC_API_KEY=sk-ant-... tests/e2e/dev-stories.sh

# Run with OpenAI
OPENAI_API_KEY=sk-proj-... tests/e2e/dev-stories.sh

# Run with local Ollama (must have qwen3:8b or llama3.2:3b pulled)
tests/e2e/dev-stories.sh

# Run specific stories
OPENAI_API_KEY=sk-... tests/e2e/dev-stories.sh 3 6 7

# Stories 8-10 require SYSKNIFE_ALLOW_DESTRUCTIVE=1 — they will fail on a
# non-Fedora-Atomic host because query_packages / query_authorized_keys
# call rpm-ostree and SSH tools that are not installed. This is expected.
# Run them on a provisioned VM for real coverage.
SYSKNIFE_ALLOW_DESTRUCTIVE=1 OPENAI_API_KEY=sk-... tests/e2e/dev-stories.sh

When to use this tier:

• After any change to crates/sysknife-brain/src/prompt.rs
• After adding or changing a query_* tool or Get*/List* action
• As a quick sanity check for brain/planner changes before pushing

Expected results on a dev machine:

• Stories 1-7: all pass
• Story 8 (install vim): fails — model calls query_packages, daemon can't run rpm-ostree, model escalates to get_system_state → StateUnavailable. Passes on a real VM where rpm-ostree is present.
• Story 9 (create toolbox): may pass plan-structure check (plan structure only, no toolbox is created).
• Story 10 (add SSH key): fails for the same reason as story 8 — query_authorized_keys fails without the SSH tooling. Passes on a real VM.

For full story 8 and 10 coverage, use the VM path.

Running the CI smoke test (opt-in)

The smoke test boots Ollama and the daemon directly in a GitHub Actions runner (no VM, no real atomic desktop), pulls a small tool-capable model (llama3.2:3b), and runs the 7 read-only user stories.

Triggers:

1. Label a PR with e2e — the workflow runs automatically
2. Manual dispatch via Actions → e2e → Run workflow

Results appear as the container-smoke job. Story logs are uploaded as build artifacts.

What the smoke test covers:

• Daemon startup, IPC framing, policy enforcement
• Brain ↔ Ollama integration, tool-use loop, safety fence
• All 7 read-only query tools and read-only action stories (1–7)

What it does NOT cover (that's the VM path below):

• rpm-ostree actions, real systemd host management
• Reboot / kernel-argument flows, rollback execution
• Tauri GUI rendering

Running the full E2E suite in a Fedora Atomic VM

This is the high-fidelity path. The VM is a real Fedora Atomic Desktop (Silverblue, Kinoite, Sway Atomic, Budgie Atomic, or COSMIC Atomic) install with rpm-ostree, systemd, flatpak, podman, and toolbox. All 54 user stories — including destructive ones — execute authentically.

Linux and macOS hosts (recommended)

We use quickemu to download the official Fedora ISO and boot it in QEMU/KVM with SSH forwarding pre-configured. One-time setup, then a reproducible VM you can snapshot and restore.

Install quickemu:

You also need qemu-system-x86_64, qemu-utils (for qemu-img), rsync, netcat, and ssh — these are all standard packages on every supported distro.

# Fedora Atomic 44 (default repos have a current quickemu)
sudo dnf install quickemu qemu qemu-img

# Fedora Atomic Desktops
sudo rpm-ostree install quickemu qemu qemu-img
# Reboot to activate, then proceed.

# Ubuntu 24.04 / Debian — the version in default Ubuntu repos may be too
# old (missing the Nov 2024 .ociarchive fix for Fedora Atomic). Use the PPA:
sudo add-apt-repository -y ppa:flexiondotorg/quickemu
sudo apt-get update
sudo apt-get install -y quickemu qemu-system-x86 qemu-utils \
    qemu-system-modules-spice rsync netcat-openbsd

# macOS (Homebrew)
brew install --cask quickemu

After installing, verify your user can access KVM (Linux only):

ls -l /dev/kvm           # should exist
test -r /dev/kvm \
    || sudo usermod -aG kvm "$USER"   # then log out and back in
                                       # (or use ACL: setfacl -m u:$USER:rw /dev/kvm)

You also need libguestfs-tools for the offline disk patches we apply between Anaconda's install and the first SSH login (set passwords, install our SSH key, enable sshd). Ubuntu 24.04 keeps kernel images at mode 0600 by default, which prevents libguestfs from running unprivileged — fix once with sudo chmod +r /boot/vmlinuz-*.

sudo apt-get install -y libguestfs-tools
sudo chmod +r /boot/vmlinuz-*

One-time VM setup:

# From the repo root.

# 1. Generate a passphrase-less SSH key dedicated to the VM (you do
#    not want to reuse your personal id_ed25519 — rsync/non-interactive
#    ssh cannot prompt for a passphrase). Idempotent.
./tests/e2e/atomic-vm.sh keygen

# 2. Download the Silverblue 44 ISO (~2.5 GB, cached under tests/e2e/vm/).
./tests/e2e/atomic-vm.sh download

# 3. Run the Fedora installer interactively (GUI window opens).
#    Just click through it — you don't need to set a password or create
#    a user; we patch all of that into the disk image afterwards via
#    libguestfs. Shut the VM down when the installer finishes (close
#    the QEMU window or pick "Power Off" in the post-install screen —
#    DO NOT click "Reboot": the ISO will re-mount as CD-ROM).
./tests/e2e/atomic-vm.sh install

# 4. Patch the disk image with our test user, password, sudoers, sshd,
#    and SSH key. (Implemented via guestfish so it works offline,
#    bypassing Silverblue's interactive first-boot wizard which has
#    gnome-initial-setup quirks on some hosts.)
./tests/e2e/atomic-vm.sh bootstrap

Why no enable-ssh step? Earlier versions of this script tried to boot the VM visibly so the contributor could enable sshd by hand. That ran into Fedora 42's gnome-initial-setup crashing on the third-party-repo screen with virgl/Wayland bugs. The current flow sidesteps the GUI entirely by configuring the VM offline via libguestfs. The enable-ssh subcommand is still there as a fallback if your Anaconda install did create a usable user.

What bootstrap does, in one shot: create user lacsdev, set the password (lacsdev), set root password (sysknife), install your VM SSH key, NOPASSWD-sudoers lacsdev, enable sshd, set SELinux to permissive, and pre-mark gnome-initial-setup as done. Idempotent — safe to re-run after install.

Run the tests:

# Boot the VM headlessly (in the background)
./tests/e2e/atomic-vm.sh start

# First-ever provision: rsyncs the repo into the VM, layers build tools
# via rpm-ostree, reboots the VM, then runs again to build SysKnife and
# pull the Ollama model. Re-run after the auto-reboot (the script tells
# you when). Expect 30-60 minutes total on first run (mostly waiting
# for Ollama tarball + Rust deps download). ~2 minutes on subsequent
# provisions.
./tests/e2e/atomic-vm.sh provision

# RECOMMENDED: take a "baseline" snapshot now, before any test run.
# Future test runs can `restore baseline` instead of re-provisioning.
./tests/e2e/atomic-vm.sh stop
./tests/e2e/atomic-vm.sh snapshot baseline
./tests/e2e/atomic-vm.sh start

# Run the read-only stories (non-destructive default)
./tests/e2e/atomic-vm.sh run

# Run ALL 54 stories including destructive — restore the baseline afterwards.
SYSKNIFE_ALLOW_DESTRUCTIVE=1 ./tests/e2e/atomic-vm.sh run

# Roll back to the clean baseline so the next run is fast
./tests/e2e/atomic-vm.sh stop
./tests/e2e/atomic-vm.sh restore baseline

Other useful commands:

./tests/e2e/atomic-vm.sh ssh            # interactive shell in the VM
./tests/e2e/atomic-vm.sh stop           # clean shutdown
./tests/e2e/atomic-vm.sh destroy        # delete VM disk (ISO kept)
./tests/e2e/atomic-vm.sh help

Try a different atomic variant:

SYSKNIFE_VM_VARIANT=kinoite ./tests/e2e/atomic-vm.sh download
SYSKNIFE_VM_VARIANT=kinoite ./tests/e2e/atomic-vm.sh install
# ... all management commands respect SYSKNIFE_VM_VARIANT.

Supported variants (these are the names quickget uses):

Windows hosts

quickemu does not support Windows as a host. Contributors on Windows should use WSL2 (with KVM nested virtualization) or VirtualBox with a manual ISO install:

1. Download the Silverblue ISO from fedoraproject.org/atomic-desktops/silverblue
2. Create a VirtualBox VM (4 GB RAM, 2 vCPUs, 20 GB disk) with SSH port forwarded from host 22220 → guest 22
3. Attach the ISO, boot, and run the Fedora installer. Create user lacsdev. Enable sshd during install.
4. SSH into the VM: ssh -p 22220 lacsdev@127.0.0.1
5. Clone the repo into /home/lacsdev/sysknife and run sudo bash tests/e2e/provision.sh inside the VM
6. Run stories with sudo -E tests/e2e/run-stories.sh

The atomic-vm.sh helper does not automate VirtualBox — that's a follow-up if Windows contributor interest warrants it.

Running the Ubuntu 24.04 E2E suite

Ubuntu 24.04 support is validated by the live-VM story suite. The pass rate and the model that produced it are recorded in tests/evidence/story-runs/, written by the run itself. The ubuntu-vm.sh script mirrors the atomic-vm.sh workflow but uses a Ubuntu 24.04 cloud image instead of a Fedora Atomic ISO.

See docs/contributing/ubuntu-vm-testing.md for the full setup and daily-use instructions. Quick reference:

# One-time setup
./tests/e2e/ubuntu-vm.sh download
./tests/e2e/ubuntu-vm.sh install

# Daily use
./tests/e2e/ubuntu-vm.sh start
./tests/e2e/ubuntu-vm.sh provision
./tests/e2e/ubuntu-vm.sh run

# Snapshot / restore for fast subsequent runs
./tests/e2e/ubuntu-vm.sh stop
./tests/e2e/ubuntu-vm.sh snapshot baseline
./tests/e2e/ubuntu-vm.sh start

When to use this tier: after any change to Ubuntu-specific actions (AptInstall, AptRemove, UfwAllow, …), the render_debian_prompt function, or the Ubuntu story scripts.

Running individual stories

Inside the VM (or on any provisioned Fedora Atomic Desktop):

cd /home/lacsdev/sysknife

# Run a specific story by number
sudo -E tests/e2e/run-stories.sh 3

# Run multiple specific stories
sudo -E tests/e2e/run-stories.sh 1 4 7

Per-story logs are written to tests/e2e/logs/story-N.log.

Before opening a PR

1. cargo nextest run --workspace && npm test --prefix apps/sysknife-shell — required, fast
2. cargo clippy --workspace --all-features --locked -- -D warnings
3. cargo fmt --all --check
4. For changes to the brain, daemon, IPC, or action catalogue:
   • Run the VM tests locally (atomic-vm.sh flow)
   • Add the e2e label to trigger the CI smoke test on your PR

Before a release

The maintainer runs these in order:

1. All automated tests green on main
2. VM tests (at least Silverblue + one other atomic variant) pass locally
3. Manual QA on real Silverblue hardware using docs/testing/user-stories.md as the checklist — all 54 stories including destructive ones
4. Record the demo video on real hardware (issue #32)

Troubleshooting

quickget fedora 43 silverblue fails

Check the quickemu wiki for current supported editions. Older or newer Silverblue releases may also be available; adjust SYSKNIFE_VM_RELEASE.

VM boots but atomic-vm.sh ssh times out

The Fedora installer doesn't enable sshd by default. Either enable sshd during the interactive install, or boot the VM's GUI console once to run:

sudo systemctl enable --now sshd

provision step fails during rpm-ostree install

rpm-ostree install requires a reboot. The provision script auto-reboots and asks you to re-run it. If it got stuck, just run provision again — it's idempotent.

Ollama download or model pull is very slow

Two distinct downloads can be slow:

1. The Ollama tarball itself (~1.5 GB, downloaded by install.sh from ollama.com/download/ollama-linux-amd64.tgz). On some networks / geos / times of day this CDN serves at <100 KB/s. There's no workaround inside the script — wait it out, or pre-stage the binary on the host and copy it in via SSH if you're going to re-provision often.

2. The model pull (~2 GB for the default llama3.2:3b, or ~5 GB for qwen3:8b if you override). Happens after Ollama is installed, via ollama pull. Goes through Ollama's registry (usually faster than the ollama.com CDN).

Override the model size with SYSKNIFE_TEST_MODEL:

SYSKNIFE_TEST_MODEL=llama3.2:3b ./tests/e2e/atomic-vm.sh provision  # default, CPU-only friendly
SYSKNIFE_TEST_MODEL=qwen2.5:3b  ./tests/e2e/atomic-vm.sh provision  # alt tool-capable 3B
SYSKNIFE_TEST_MODEL=qwen3:8b    ./tests/e2e/atomic-vm.sh provision  # GPU passthrough only