headless-primary: make the watcher always-converge

Add a self-healing backstop so recovery no longer depends on an event firing:
- periodic reconcile (every 8s): read-only drift check that runs apply ONLY when
  broken (no headless / DP-1 not mirroring / workspace trapped on a mirror /
  orphan / off-screen window) — no steady-state flicker.
- delayed reconcile 1.5s after a monitor event to catch late workspace
  reassignment races.
- socat auto-reconnect; exit cleanly when the session socket vanishes.
New 'reconcile' subcommand. Verified: reconcile heals a deliberately broken
mirror and no-ops on a healthy state.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

bin/sunshine-headless-primary.sh

@@ -20,8 +20,8 @@
 # Subcommands:
 #   apply      (default) establish/repair state — idempotent (topology + rescues)
 #   rescue     just pull off-screen floating windows back on-screen
-#   watch    long-running: re-apply on monitor hotplug AND rescue windows that
+#   reconcile  read-only drift check; runs apply ONLY if something is wrong
-#            open off-screen (this is what makes stray windows self-heal)
+#   watch      long-running self-healing daemon (events + periodic backstop)
 
 set -uo pipefail
 
@@ -31,6 +31,7 @@ WIDTH=5120
 HEIGHT=1440
 RATE=60
 POS="0x0"
+RECONCILE_SECS=8          # periodic backstop cadence (self-heal if an event is missed)
 CONF="$HOME/.config/sunshine/sunshine.conf"
 
 ensure_hypr_sig() {
@@ -52,28 +53,27 @@ headless_names() {
     | jq -r '.[] | select(.name | startswith("HEADLESS")) | .name' | sort -V
 }
 
-# Recenter any FLOATING, mapped window whose rectangle does not intersect the
+# Addresses of FLOATING, mapped windows whose rectangle does not intersect the
-# monitor its workspace lives on (i.e. it's stranded off-screen after a monitor
+# monitor their workspace lives on (stranded off-screen). Optional single-addr arg.
-# topology change, or it opened off-screen). Pass a single window address to
+offscreen_addrs() {
-# check just that window (used on the openwindow event); no arg = sweep all.
+  local only="${1:-}" mons clients
-rescue_offscreen_windows() {
-  local only="${1:-}" mons clients addrs a
-  have_tools || return 0
   mons="$(hyprctl monitors all -j 2>/dev/null)" || return 0
   clients="$(hyprctl clients -j 2>/dev/null)" || return 0
-  addrs="$(printf '%s' "$clients" | jq -r --argjson mons "$mons" --arg only "$only" '
+  printf '%s' "$clients" | jq -r --argjson mons "$mons" --arg only "$only" '
     ($mons | map({key:(.id|tostring), value:{x:.x,y:.y,w:.width,h:.height}}) | from_entries) as $M
     | .[]
     | select(.mapped == true and .floating == true)
     | select($only == "" or .address == $only)
     | . as $c | ($M[$c.monitor|tostring]) as $m
     | select($m != null)
-    | select( ($c.at[0]+$c.size[0]) <= $m.x
+    | select( ($c.at[0]+$c.size[0]) <= $m.x or  $c.at[0] >= ($m.x+$m.w)
-           or  $c.at[0] >= ($m.x+$m.w)
+           or  ($c.at[1]+$c.size[1]) <= $m.y or  $c.at[1] >= ($m.y+$m.h) )
-           or  ($c.at[1]+$c.size[1]) <= $m.y
+    | .address' 2>/dev/null
-           or  $c.at[1] >= ($m.y+$m.h) )
+}
-    | .address' 2>/dev/null)"
+
-  for a in $addrs; do
+rescue_offscreen_windows() {
+  local a
+  for a in $(offscreen_addrs "${1:-}"); do
     hyprctl dispatch focuswindow "address:$a" >/dev/null 2>&1
     hyprctl dispatch centerwindow >/dev/null 2>&1
     log "rescued off-screen window $a"
@@ -85,8 +85,8 @@ apply() {
   have_tools || { log "hyprctl/jq missing."; return 0; }
 
   # 1. Exactly one headless output. Create if none; disable extras (remove is
-  #    unreliable for mirrored/persistent headless — it returns "output not
+  #    unreliable for mirrored/persistent headless — returns "output not found"
-  #    found" and exits 0). Keep the lowest-numbered.
+  #    and exits 0). Keep the lowest-numbered.
   mapfile -t hs < <(headless_names)
   local head="${hs[0]:-}"
   if [[ -z "$head" ]]; then
@@ -113,11 +113,11 @@ apply() {
     log "DP-1 mirroring $head"
   fi
 
-  # 4. Relocate any workspace that lives on a MIRRORED output onto the headless
+  # 4. Relocate any workspace stranded on a MIRRORED output onto the primary.
-  #    primary. A mirror is excluded from the layout, so a workspace bound to it
+  #    On swap-back Hyprland brings DP-1 up normal, binds a workspace to it, then
-  #    is trapped and unreachable — exactly what happens on swap-back when
+  #    we mirror it — trapping that workspace on a layout-excluded mirror. (You
-  #    Hyprland brings DP-1 up normal, binds a workspace to it, then we mirror
+  #    can move a workspace OFF a mirror but not ONTO one; tostring guards
-  #    it. (tostring guards mirrorOf being null | "None" | a numeric id.)
+  #    mirrorOf being null | "None" | a numeric id.)
   local mon ws
   for mon in $(hyprctl monitors all -j \
       | jq -r '.[] | select(((.mirrorOf // "None") | tostring | ascii_downcase) != "none") | .name'); do
@@ -150,39 +150,88 @@ apply() {
   log "headless-primary established on $head"
 }
 
+# Read-only drift check. Runs apply ONLY when something is actually wrong, so it
+# never causes steady-state flicker. This is the backstop that makes the setup
+# self-heal even if a Hyprland event is missed entirely.
+reconcile() {
+  ensure_hypr_sig || return 0
+  have_tools || return 0
+  local mons broken=0 m
+  mons="$(hyprctl monitors all -j 2>/dev/null)" || return 0
+
+  # a) exactly one enabled headless output
+  [[ "$(printf '%s' "$mons" | jq -r '[.[]|select((.name|startswith("HEADLESS")) and (.disabled|not))]|length')" == "1" ]] || broken=1
+
+  # b) if DP-1 is present it must be mirroring (never a stray normal output)
+  if printf '%s' "$mons" | jq -e '.[]|select(.name=="DP-1")' >/dev/null 2>&1; then
+    printf '%s' "$mons" | jq -e '.[]|select(.name=="DP-1" and (((.mirrorOf//"None")|tostring|ascii_downcase)=="none"))' >/dev/null 2>&1 && broken=1
+  fi
+
+  # c) a workspace stranded on a mirrored output
+  for m in $(printf '%s' "$mons" | jq -r '.[]|select(((.mirrorOf//"None")|tostring|ascii_downcase)!="none")|.name'); do
+    hyprctl workspaces -j | jq -e --arg m "$m" '.[]|select(.monitor==$m)' >/dev/null 2>&1 && broken=1
+  done
+
+  # d) orphaned workspace
+  hyprctl workspaces -j | jq -e '.[]|select(.monitorID==-1)' >/dev/null 2>&1 && broken=1
+
+  # e) off-screen floating window
+  [[ -n "$(offscreen_addrs)" ]] && broken=1
+
+  if [[ "$broken" == "1" ]]; then
+    log "reconcile: state drift detected -> apply"
+    apply
+  fi
+}
+
 watch() {
   ensure_hypr_sig || { log "Hyprland not running; watcher exiting."; return 0; }
   have_tools || { log "hyprctl/jq missing; watcher exiting."; return 0; }
   command -v socat >/dev/null || { log "socat missing; no watcher."; return 0; }
   local sock="${XDG_RUNTIME_DIR}/hypr/${HYPRLAND_INSTANCE_SIGNATURE}/.socket2.sock"
-  log "watching Hyprland events on $sock"
+  log "watching Hyprland events on $sock (reconcile every ${RECONCILE_SECS}s)"
 
-  # monitoradded/removed  -> full re-apply (re-impose mirror + rescue everything)
+  # Backstop: periodic reconcile self-heals even if an event never arrives.
-  # openwindow            -> targeted rescue if the NEW window opened off-screen
+  # Exits when the session socket disappears (Hyprland gone) so we don't linger.
+  ( while sleep "$RECONCILE_SECS"; do [[ -S "$sock" ]] || exit 0; reconcile; done ) &
+  local bg=$!
+  trap 'kill "$bg" 2>/dev/null' EXIT INT TERM
+
+  # Event loop with auto-reconnect (survives transient socat drops). Pinned to
+  # this session's socket; if it vanishes the session ended → exit cleanly and
+  # let the next login's exec-once start a fresh watcher.
+  while [[ -S "$sock" ]]; do
     socat -U - "UNIX-CONNECT:$sock" 2>/dev/null | {
       local last=0 now addr
       while read -r event; do
         case "$event" in
           monitoradded*|monitorremoved*)
             now=$(date +%s)
-          (( now - last < 2 )) && continue   # debounce event bursts (v1+v2)
+            (( now - last < 2 )) && continue     # coalesce the v1+v2 burst
             last=$now
-          log "monitor event (${event%%>*}) -> apply"
+            log "monitor event (${event%%>*}) -> apply (+delayed reconcile)"
             apply
+            ( sleep 1.5; reconcile ) &           # catch late workspace reassignment
             ;;
           openwindow*)
             addr="0x${event#openwindow>>}"; addr="${addr%%,*}"
-          sleep 0.3   # let the new window's geometry settle
+            sleep 0.3                             # let the new window settle
             rescue_offscreen_windows "$addr"
             ;;
         esac
       done
     }
+    [[ -S "$sock" ]] || break
+    log "event socket dropped; reconnecting in 1s"
+    sleep 1
+  done
+  log "watcher: session socket gone; exiting"
 }
 
 case "${1:-apply}" in
   apply) apply ;;
   rescue) rescue_offscreen_windows "${2:-}" ;;
+  reconcile) reconcile ;;
   watch) watch ;;
-  *) echo "Usage: $(basename "$0") {apply|rescue|watch}" >&2; exit 1 ;;
+  *) echo "Usage: $(basename "$0") {apply|rescue|reconcile|watch}" >&2; exit 1 ;;
 esac

docs/HEADLESS-PRIMARY.md

@@ -49,7 +49,7 @@ clients. Inherent to cloning an ultrawide.
 
 | Path | Role |
 |---|---|
-| `bin/sunshine-headless-primary.sh` | The manager. `apply` establishes/repairs state (idempotent); `rescue` pulls off-screen floating windows back on-screen; `watch` re-applies on monitor hotplug AND auto-rescues windows that open off-screen. |
+| `bin/sunshine-headless-primary.sh` | The manager. `apply` establishes/repairs state (idempotent); `rescue` pulls off-screen floating windows back on-screen; `reconcile` heals drift only if broken; `watch` the self-healing daemon (events + periodic reconcile backstop + auto-reconnect). |
 | `bin/sunshine-prestart.sh` | Sunshine `ExecStartPre` — thin wrapper that runs `sunshine-headless-primary.sh apply` before the encoder probe. |
 | `~/.config/hypr/monitors.conf` | `DP-1` kept **normal** (safety fallback); `HEADLESS-1..4` default to `0x0`. The manager applies the `DP-1 → mirror HEADLESS-1` on top. |
 | `~/.config/hypr/autostart.conf` | `exec-once` runs `apply` at login and starts the `watch` daemon. |
@@ -77,16 +77,28 @@ clients. Inherent to cloning an ultrawide.
 
 ### Self-healing (the watcher)
 
-`watch` subscribes to Hyprland's event socket and reacts:
+`watch` is designed so the setup **always converges**, not just when an event
+fires. Three layers:
 
-- `monitoradded` / `monitorremoved` → full `apply` (re-impose the mirror,
+1. **Events** (fast path) — subscribes to Hyprland's socket:
-  re-consolidate, rescue) — debounced 2s to coalesce the v1+v2 event pair.
+   - `monitoradded` / `monitorremoved` → `apply` immediately (debounced 2s to
-- `openwindow` → targeted `rescue` of just that window if it opened off-screen
+     coalesce the v1+v2 burst), then a `reconcile` 1.5s later to catch a
-  (0.3s settle delay first).
+     workspace Hyprland reassigns to `DP-1` *after* apply already ran.
+   - `openwindow` → targeted `rescue` if that window opened off-screen.
+2. **Periodic `reconcile` backstop** (every `RECONCILE_SECS`, default 8s) — a
+   read-only drift check that runs `apply` **only when something is actually
+   wrong** (no headless / DP-1 not mirroring / workspace trapped on a mirror /
+   orphaned workspace / off-screen window). This is the guarantee: even if an
+   event is missed entirely, the state self-heals within a few seconds. It does
+   nothing when healthy, so there's no steady-state flicker.
+3. **Auto-reconnect** — if `socat` drops, the loop reconnects; if the session
+   socket disappears (Hyprland gone) it exits cleanly so the next login's
+   `exec-once` starts a fresh watcher (no cross-session duplicates).
 
-This is what makes stray windows self-heal: a monitor swap or an app that opens
+So a monitor swap, a trapped workspace ("can't show workspace 1"), or an app
-off-screen (OBS, `bluetui`, kdenlive have all done this) gets pulled back into
+that opens off-screen (OBS, `bluetui`, kdenlive have all done this) gets fixed
-view automatically, no manual hunting.
+automatically — near-instant via the event, or within ~8s via the backstop.
+Force a check manually anytime with `sunshine-headless-primary.sh reconcile`.
 
 ### Critical implementation notes