Omarchy-Stream/docs/HEADLESS-PRIMARY.md

# Headless-primary streaming (JARVIS as-built)

How **JARVIS** streams its desktop to Moonlight so that it works **whether or not
the physical monitor is on** — the machine's normal remote-use case.

> **Companion note — keep in sync.** This document is mirrored in the Obsidian
> vault at `Documents/Sync Vault/Network/Network — Sunshine + Moonlight (JARVIS).md`.
> They are **not** auto-linked. **If you edit one, update the other.**

---

## TL;DR

- The desktop permanently lives on a **persistent virtual `HEADLESS-1` output**.
  Sunshine captures it. It exists independent of the physical monitor.
- The physical **`DP-1` mirrors `HEADLESS-1`** when present, so the at-desk view
  equals the stream.
- When `DP-1` is off (remote), nothing is orphaned and the stream keeps working.
- Managed by `~/.local/share/omarchy-moonlight/bin/sunshine-headless-primary.sh`.

---

## Why this design (the three iterations)

1. **Headless-move (stock omarchy-moonlight headless mode)** — created a
   client-sized `HEADLESS-1` and *moved the active workspace onto it* via the
   `global_prep_cmd` hooks. Not a clone; and because a permanent off-screen
   output existed, apps that remember their monitor (OBS) reopened invisibly on
   it. Rejected.

2. **Mirror-clone (`HEADLESS-1` mirrors `DP-1`)** — a true clone while the
   monitor is on, and mirrored outputs are excluded from the layout so nothing
   strands on them. **But** when the physical monitor fully powers off, `DP-1`
   disappears from Hyprland, the mirror collapses to a standalone empty output,
   and every workspace bound to `DP-1` is orphaned off-screen. Fatal for a
   remote-first host whose monitor is usually off.

3. **Headless-primary (current)** — invert the mirror. The **headless output is
   the source of truth** (always present); `DP-1` mirrors *it*. The capture
   target is never absent, so nothing is ever orphaned. This is the only model
   that satisfies "the stream is a clone AND survives the monitor being off."

**Tradeoff:** the stream is the full `5120x1440` ultrawide, letterboxed on 16:9
clients. Inherent to cloning an ultrawide.

---

## Components

| Path | Role |
|---|---|
| `bin/sunshine-headless-primary.sh` | The manager. `apply` establishes/repairs state (idempotent); `rescue` pulls off-screen floating windows back on-screen; `watch` re-applies on monitor hotplug AND auto-rescues windows that open off-screen. |
| `bin/sunshine-prestart.sh` | Sunshine `ExecStartPre` — thin wrapper that runs `sunshine-headless-primary.sh apply` before the encoder probe. |
| `~/.config/hypr/monitors.conf` | `DP-1` kept **normal** (safety fallback); `HEADLESS-1..4` default to `0x0`. The manager applies the `DP-1 → mirror HEADLESS-1` on top. |
| `~/.config/hypr/autostart.conf` | `exec-once` runs `apply` at login and starts the `watch` daemon. |
| `~/.config/sunshine/sunshine.conf` | `capture = wlr`, `output_name = HEADLESS-1`, `encoder = nvenc`, `global_prep_cmd = []` (no workspace moving). |
| systemd drop-in `…Sunshine.service.d/headless-prestart.conf` | wires `ExecStartPre` to prestart. |

### What `apply` does (idempotent)

1. Ensures exactly one `HEADLESS` output (creates if none; **disables** extras —
   `hyprctl output remove` is unreliable for these, returns "output not found").
2. Sizes it `5120x1440@60` at `0x0`.
3. If `DP-1` is present → `hyprctl keyword monitor "DP-1,…,mirror,HEADLESS-1"`.
4. Rescues any **orphaned** workspace (`monitorID == -1`) onto the headless
   primary — deliberately narrow, so a healthy second monitor (DP-2) is left
   alone.
5. **Rescues off-screen floating windows** — recenters any floating, mapped
   window whose rectangle doesn't intersect the monitor its workspace lives on
   (stranded by a topology change, or opened off-screen).
6. Syncs `sunshine.conf`'s `output_name` to the live headless name.

### Self-healing (the watcher)

`watch` subscribes to Hyprland's event socket and reacts:

- `monitoradded` / `monitorremoved` → full `apply` (re-impose the mirror,
  re-consolidate, rescue) — debounced 2s to coalesce the v1+v2 event pair.
- `openwindow` → targeted `rescue` of just that window if it opened off-screen
  (0.3s settle delay first).

This is what makes stray windows self-heal: a monitor swap or an app that opens
off-screen (OBS, `bluetui`, kdenlive have all done this) gets pulled back into
view automatically, no manual hunting.

### Critical implementation notes

- **Always `hyprctl monitors all`**, never plain `hyprctl monitors` — a mirrored
  output is excluded from the plain list, i.e. invisible to exactly the outputs
  we manage. Plain form → the dedup loop spawns a new headless every run.
- **`DP-1` must stay `normal` in `monitors.conf`.** It is the safety fallback: if
  the manager never runs, the physical screen still displays. Never hard-code
  `DP-1 … mirror,HEADLESS-1` statically — at cold boot `HEADLESS-1` doesn't exist
  yet and the screen could blank. The mirror is only ever added at runtime.
- **Hotplug:** Omarchy's own `omarchy-hyprland-monitor-watch` only handles
  `monitorremoved`; our `watch` handles `monitoradded` to re-impose the mirror
  when the monitor is plugged back in.

---

## The network side (this bit is what actually blocks connections)

Streaming failing is usually **not** Sunshine. Two real blockers hit on JARVIS:

### 1. ufw rules pinned to the OLD subnet (the real outage)

The omarchy-moonlight installer opens the Sunshine ports in **ufw scoped to the
LAN subnet**. After the LAN was renumbered `192.168.1.0/24 → 10.0.0.0/24`, the
rules still said `192.168.1.0/24`, so every client on `10.0.0.x` was silently
rejected (default-deny input) while `ssh` — rule `Anywhere` — kept working.

```bash
# symptom: from another LAN host, tcp/22 OPEN but 47984/47989/48010 BLOCKED,
# yet the ports listen on 0.0.0.0 and answer locally.
sudo ufw status verbose
# fix — re-scope to the current subnet:
sudo ufw allow from 10.0.0.0/24 to any port 47984,47989,47990,48010 proto tcp
sudo ufw allow from 10.0.0.0/24 to any port 47998,47999,48000,48010 proto udp
sudo ufw reload
# then delete the stale 192.168.1.0/24 rules (ufw status numbered; ufw delete N)
```

> **Rule of thumb: re-scope the omarchy-moonlight ufw rules after any network
> renumber.** The ports are TCP `47984/47989/47990/48010` + UDP
> `47998/47999/48000/48010`. `47990` (web UI) can stay closed — it's
> localhost-locked anyway.

### 2. mDNS resolves to the Docker bridge

With Docker running, avahi advertises on the docker interfaces and
`JARVIS.local` resolves to `172.17.0.1` (docker0) or an IPv6 link-local address —
not `10.0.0.13`. Moonlight auto-discovery then targets an unreachable address.
**Workaround: add the host in Moonlight by IP `10.0.0.13`.** Proper fix: restrict
avahi to the real NIC(s):

```bash
# /etc/avahi/avahi-daemon.conf, under [server]:
#   allow-interfaces=enp7s0,wlan0
#   deny-interfaces=docker0,br-8482440f28f0
sudo systemctl restart avahi-daemon
avahi-resolve -4 -n JARVIS.local     # should print 10.0.0.13
```

---

## Verify

```bash
# headless primary at 0x0, workspaces on it, DP-1 (if present) mirroring it
hyprctl monitors all -j | jq -r '.[] | "\(.name) pos=\(.x)x\(.y) mirrorOf=\(.mirrorOf // "-")"'
hyprctl workspaces -j  | jq -r '.[] | "ws \(.id) -> \(.monitor)"'

# manager is idempotent (run twice; stays one headless)
~/.local/share/omarchy-moonlight/bin/sunshine-headless-primary.sh apply

# hotplug watcher alive
pgrep -af 'sunshine-headless-primary.sh watch'

# from another LAN host: all Sunshine ports reachable
for p in 47984 47989 48010; do nc -vz 10.0.0.13 $p; done
```

Then connect Moonlight (add host by IP `10.0.0.13`) → you should see your desktop
whether or not the physical monitor is on.

---

## Gotchas & recovery

- **Stuck on the wrong/empty workspace after a monitor state change** — re-run
  the manager, or nudge manually:
  ```bash
  ~/.local/share/omarchy-moonlight/bin/sunshine-headless-primary.sh apply
  # or, targeted:
  hyprctl dispatch moveworkspacetomonitor "1 HEADLESS-1"; hyprctl dispatch workspace 1
  ```
- **`hyprctl output remove HEADLESS-N`** returns "output not found" and exits 0
  for these — use `hyprctl keyword monitor "HEADLESS-N,disable"` or reboot.
- **Re-running the omarchy-moonlight `install.sh`** regenerates `sunshine.conf`
  (re-enabling `global_prep_cmd`) and re-adds subnet-scoped ufw rules. Re-apply
  the headless-primary `sunshine.conf` settings and re-check ufw after any
  reinstall.
- **Admin UI** is localhost-only (`origin_web_ui_allowed = pc`). Reach it via
  `ssh -L 47990:localhost:47990 lwoodard@10.0.0.13` then `https://localhost:47990`
  (note: `-L` local forward, not `-R`).

See also `docs/ARCHITECTURE.md`, `docs/TROUBLESHOOTING.md`, `docs/FOLLOWUPS.md`.