Restart without reboot; retrievable logs incl. mpv exit reasons

Two related gaps on the headless kiosk: no way to restart the wall short
of a full reboot, and no way to see why a stream keeps flapping (the
daemon's stderr goes to sway on tty1, and mpv's stderr was discarded).

Restart in place:
- `rtsp-streamer restart` sends a control-socket command; the daemon
  tears down its mpv children and syscall.Exec's the on-disk binary.
  Same PID, same parent (sway), same env — keeps WAYLAND_DISPLAY/
  SWAYSOCK and comes back up on the new binary, no reboot. Handles the
  os.Executable() "(deleted)" sentinel from make install's rename.
- sway config now launches the daemon in a relaunch loop, so a crash (or
  the restart) auto-recovers instead of leaving a black screen.
- `make deploy` now does `install` + `restart` instead of restarting
  getty@tty1 (which left stale duplicate sessions and forced reboots).

Retrievable logs:
- New internal/logbuf ring; the daemon tees slog output into it and
  serves the tail over the socket via `rtsp-streamer logs [-n N]` —
  readable over SSH, no file wrangling, no reboot.
- Capture the tail of each mpv's stderr and log its last line when the
  process exits, so "mpv exited, will restart" now carries the reason
  (connection refused, unsupported codec, 401, ...). This is the
  diagnostic for a single tile going unhealthy repeatedly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Levi Woodard
2026-07-02 10:03:25 -05:00
parent 9985a93851
commit be1b53ae04
10 changed files with 308 additions and 33 deletions

View File

@@ -13,11 +13,13 @@ import (
"os"
"os/exec"
"path/filepath"
"strings"
"sync"
"syscall"
"time"
"github.com/lwoodard/rtsp-streamer/internal/config"
"github.com/lwoodard/rtsp-streamer/internal/logbuf"
)
// Player supervises a single mpv instance bound to one grid slot.
@@ -38,6 +40,7 @@ type Player struct {
mu sync.Mutex
cmd *exec.Cmd
started time.Time
stderr *logbuf.Writer // tail of the current process's stderr
}
// New creates a player. runDir is where the mpv IPC socket lives.
@@ -143,15 +146,19 @@ func (p *Player) start(ctx context.Context) error {
cmd.WaitDelay = 2 * time.Second
// Inherit the caller's environment (WAYLAND_DISPLAY etc. must be set).
cmd.Env = os.Environ()
// Discard mpv's chatty stdout/stderr; errors surface via exit code.
// Keep the tail of mpv's stderr so a crash's reason (bad codec, dropped
// connection, auth failure) is logged when the process exits, instead of
// vanishing. Cheap: a small ring, not the whole firehose.
stderr := logbuf.New(30)
cmd.Stdout = nil
cmd.Stderr = nil
cmd.Stderr = stderr
if err := cmd.Start(); err != nil {
return fmt.Errorf("starting mpv: %w", err)
}
p.mu.Lock()
p.cmd = cmd
p.started = time.Now()
p.stderr = stderr
p.mu.Unlock()
p.log.Info("mpv started", "pid", cmd.Process.Pid)
return nil
@@ -177,6 +184,7 @@ func (p *Player) Supervise(ctx context.Context) {
}
p.mu.Lock()
cmd := p.cmd
stderr := p.stderr
p.mu.Unlock()
err := cmd.Wait()
@@ -187,7 +195,10 @@ func (p *Player) Supervise(ctx context.Context) {
if ctx.Err() != nil {
return
}
p.log.Warn("mpv exited, will restart", "err", err, "after", backoff)
// Surface the last line(s) mpv printed — that's where the failure
// reason lives (connection refused, unsupported codec, 401, ...).
reason := lastLine(stderr)
p.log.Warn("mpv exited, will restart", "err", err, "after", backoff, "reason", reason)
if !sleep(ctx, backoff) {
return
}
@@ -271,6 +282,22 @@ func (p *Player) Healthy() bool {
return reply["error"] == "success"
}
// lastLine returns the most recent non-empty stderr line mpv emitted, trimmed,
// or "" if there was none. mpv prints its fatal error last, so the tail is the
// useful part.
func lastLine(w *logbuf.Writer) string {
if w == nil {
return ""
}
lines := w.Lines(0)
for i := len(lines) - 1; i >= 0; i-- {
if s := strings.TrimSpace(lines[i]); s != "" {
return s
}
}
return ""
}
func sleep(ctx context.Context, d time.Duration) bool {
t := time.NewTimer(d)
defer t.Stop()