Add headless-primary streaming mode + off-screen window self-heal

Desktop lives on a persistent HEADLESS output that Sunshine captures; DP-1
mirrors it when present, so streaming works whether or not the physical monitor
is on. Fixes workspace/window orphaning when the monitor powers off (the old
'HEADLESS mirrors DP-1' design collapsed when DP-1 disappeared).

- bin/sunshine-headless-primary.sh: apply (idempotent topology + rescue),
  rescue (recenter off-screen floating windows), watch (self-heal on monitor
  add/remove and on windows that open off-screen).
- bin/sunshine-prestart.sh: delegates to the manager's apply.
- docs/HEADLESS-PRIMARY.md: full rationale + the ufw stale-subnet gotcha.

Not yet wired into install.sh; configured directly on JARVIS via monitors.conf
and autostart.conf (tracked in the dotfiles repo).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Levi Woodard
2026-07-29 16:47:22 -06:00
parent 836d775d3d
commit 0d6da8c24a
3 changed files with 372 additions and 79 deletions

187
docs/HEADLESS-PRIMARY.md Normal file
View File

@@ -0,0 +1,187 @@
# Headless-primary streaming (JARVIS as-built)
How **JARVIS** streams its desktop to Moonlight so that it works **whether or not
the physical monitor is on** — the machine's normal remote-use case.
> **Companion note — keep in sync.** This document is mirrored in the Obsidian
> vault at `Documents/Sync Vault/Network/Network — Sunshine + Moonlight (JARVIS).md`.
> They are **not** auto-linked. **If you edit one, update the other.**
---
## TL;DR
- The desktop permanently lives on a **persistent virtual `HEADLESS-1` output**.
Sunshine captures it. It exists independent of the physical monitor.
- The physical **`DP-1` mirrors `HEADLESS-1`** when present, so the at-desk view
equals the stream.
- When `DP-1` is off (remote), nothing is orphaned and the stream keeps working.
- Managed by `~/.local/share/omarchy-moonlight/bin/sunshine-headless-primary.sh`.
---
## Why this design (the three iterations)
1. **Headless-move (stock omarchy-moonlight headless mode)** — created a
client-sized `HEADLESS-1` and *moved the active workspace onto it* via the
`global_prep_cmd` hooks. Not a clone; and because a permanent off-screen
output existed, apps that remember their monitor (OBS) reopened invisibly on
it. Rejected.
2. **Mirror-clone (`HEADLESS-1` mirrors `DP-1`)** — a true clone while the
monitor is on, and mirrored outputs are excluded from the layout so nothing
strands on them. **But** when the physical monitor fully powers off, `DP-1`
disappears from Hyprland, the mirror collapses to a standalone empty output,
and every workspace bound to `DP-1` is orphaned off-screen. Fatal for a
remote-first host whose monitor is usually off.
3. **Headless-primary (current)** — invert the mirror. The **headless output is
the source of truth** (always present); `DP-1` mirrors *it*. The capture
target is never absent, so nothing is ever orphaned. This is the only model
that satisfies "the stream is a clone AND survives the monitor being off."
**Tradeoff:** the stream is the full `5120x1440` ultrawide, letterboxed on 16:9
clients. Inherent to cloning an ultrawide.
---
## Components
| Path | Role |
|---|---|
| `bin/sunshine-headless-primary.sh` | The manager. `apply` establishes/repairs state (idempotent); `rescue` pulls off-screen floating windows back on-screen; `watch` re-applies on monitor hotplug AND auto-rescues windows that open off-screen. |
| `bin/sunshine-prestart.sh` | Sunshine `ExecStartPre` — thin wrapper that runs `sunshine-headless-primary.sh apply` before the encoder probe. |
| `~/.config/hypr/monitors.conf` | `DP-1` kept **normal** (safety fallback); `HEADLESS-1..4` default to `0x0`. The manager applies the `DP-1 → mirror HEADLESS-1` on top. |
| `~/.config/hypr/autostart.conf` | `exec-once` runs `apply` at login and starts the `watch` daemon. |
| `~/.config/sunshine/sunshine.conf` | `capture = wlr`, `output_name = HEADLESS-1`, `encoder = nvenc`, `global_prep_cmd = []` (no workspace moving). |
| systemd drop-in `…Sunshine.service.d/headless-prestart.conf` | wires `ExecStartPre` to prestart. |
### What `apply` does (idempotent)
1. Ensures exactly one `HEADLESS` output (creates if none; **disables** extras —
`hyprctl output remove` is unreliable for these, returns "output not found").
2. Sizes it `5120x1440@60` at `0x0`.
3. If `DP-1` is present → `hyprctl keyword monitor "DP-1,…,mirror,HEADLESS-1"`.
4. Rescues any **orphaned** workspace (`monitorID == -1`) onto the headless
primary — deliberately narrow, so a healthy second monitor (DP-2) is left
alone.
5. **Rescues off-screen floating windows** — recenters any floating, mapped
window whose rectangle doesn't intersect the monitor its workspace lives on
(stranded by a topology change, or opened off-screen).
6. Syncs `sunshine.conf`'s `output_name` to the live headless name.
### Self-healing (the watcher)
`watch` subscribes to Hyprland's event socket and reacts:
- `monitoradded` / `monitorremoved` → full `apply` (re-impose the mirror,
re-consolidate, rescue) — debounced 2s to coalesce the v1+v2 event pair.
- `openwindow` → targeted `rescue` of just that window if it opened off-screen
(0.3s settle delay first).
This is what makes stray windows self-heal: a monitor swap or an app that opens
off-screen (OBS, `bluetui`, kdenlive have all done this) gets pulled back into
view automatically, no manual hunting.
### Critical implementation notes
- **Always `hyprctl monitors all`**, never plain `hyprctl monitors` — a mirrored
output is excluded from the plain list, i.e. invisible to exactly the outputs
we manage. Plain form → the dedup loop spawns a new headless every run.
- **`DP-1` must stay `normal` in `monitors.conf`.** It is the safety fallback: if
the manager never runs, the physical screen still displays. Never hard-code
`DP-1 … mirror,HEADLESS-1` statically — at cold boot `HEADLESS-1` doesn't exist
yet and the screen could blank. The mirror is only ever added at runtime.
- **Hotplug:** Omarchy's own `omarchy-hyprland-monitor-watch` only handles
`monitorremoved`; our `watch` handles `monitoradded` to re-impose the mirror
when the monitor is plugged back in.
---
## The network side (this bit is what actually blocks connections)
Streaming failing is usually **not** Sunshine. Two real blockers hit on JARVIS:
### 1. ufw rules pinned to the OLD subnet (the real outage)
The omarchy-moonlight installer opens the Sunshine ports in **ufw scoped to the
LAN subnet**. After the LAN was renumbered `192.168.1.0/24 → 10.0.0.0/24`, the
rules still said `192.168.1.0/24`, so every client on `10.0.0.x` was silently
rejected (default-deny input) while `ssh` — rule `Anywhere` — kept working.
```bash
# symptom: from another LAN host, tcp/22 OPEN but 47984/47989/48010 BLOCKED,
# yet the ports listen on 0.0.0.0 and answer locally.
sudo ufw status verbose
# fix — re-scope to the current subnet:
sudo ufw allow from 10.0.0.0/24 to any port 47984,47989,47990,48010 proto tcp
sudo ufw allow from 10.0.0.0/24 to any port 47998,47999,48000,48010 proto udp
sudo ufw reload
# then delete the stale 192.168.1.0/24 rules (ufw status numbered; ufw delete N)
```
> **Rule of thumb: re-scope the omarchy-moonlight ufw rules after any network
> renumber.** The ports are TCP `47984/47989/47990/48010` + UDP
> `47998/47999/48000/48010`. `47990` (web UI) can stay closed — it's
> localhost-locked anyway.
### 2. mDNS resolves to the Docker bridge
With Docker running, avahi advertises on the docker interfaces and
`JARVIS.local` resolves to `172.17.0.1` (docker0) or an IPv6 link-local address —
not `10.0.0.13`. Moonlight auto-discovery then targets an unreachable address.
**Workaround: add the host in Moonlight by IP `10.0.0.13`.** Proper fix: restrict
avahi to the real NIC(s):
```bash
# /etc/avahi/avahi-daemon.conf, under [server]:
# allow-interfaces=enp7s0,wlan0
# deny-interfaces=docker0,br-8482440f28f0
sudo systemctl restart avahi-daemon
avahi-resolve -4 -n JARVIS.local # should print 10.0.0.13
```
---
## Verify
```bash
# headless primary at 0x0, workspaces on it, DP-1 (if present) mirroring it
hyprctl monitors all -j | jq -r '.[] | "\(.name) pos=\(.x)x\(.y) mirrorOf=\(.mirrorOf // "-")"'
hyprctl workspaces -j | jq -r '.[] | "ws \(.id) -> \(.monitor)"'
# manager is idempotent (run twice; stays one headless)
~/.local/share/omarchy-moonlight/bin/sunshine-headless-primary.sh apply
# hotplug watcher alive
pgrep -af 'sunshine-headless-primary.sh watch'
# from another LAN host: all Sunshine ports reachable
for p in 47984 47989 48010; do nc -vz 10.0.0.13 $p; done
```
Then connect Moonlight (add host by IP `10.0.0.13`) → you should see your desktop
whether or not the physical monitor is on.
---
## Gotchas & recovery
- **Stuck on the wrong/empty workspace after a monitor state change** — re-run
the manager, or nudge manually:
```bash
~/.local/share/omarchy-moonlight/bin/sunshine-headless-primary.sh apply
# or, targeted:
hyprctl dispatch moveworkspacetomonitor "1 HEADLESS-1"; hyprctl dispatch workspace 1
```
- **`hyprctl output remove HEADLESS-N`** returns "output not found" and exits 0
for these — use `hyprctl keyword monitor "HEADLESS-N,disable"` or reboot.
- **Re-running the omarchy-moonlight `install.sh`** regenerates `sunshine.conf`
(re-enabling `global_prep_cmd`) and re-adds subnet-scoped ufw rules. Re-apply
the headless-primary `sunshine.conf` settings and re-check ufw after any
reinstall.
- **Admin UI** is localhost-only (`origin_web_ui_allowed = pc`). Reach it via
`ssh -L 47990:localhost:47990 lwoodard@10.0.0.13` then `https://localhost:47990`
(note: `-L` local forward, not `-R`).
See also `docs/ARCHITECTURE.md`, `docs/TROUBLESHOOTING.md`, `docs/FOLLOWUPS.md`.