Posted to tcl by juliannoble2 at Sun Jul 12 18:09:06 GMT 2026view pretty

## Summary

When the console *server* (conhost.exe) serving a Tcl 9 process dies without the client
being terminated, a script with a `chan event stdin readable` handler is never notified:
the driver detects the error internally, but the event it queues for the interp thread is
discarded before `Tcl_NotifyChannel`, so no readable event, no EOF, no read error ever
reaches script level. Meanwhile the driver's console reader thread busy-loops on the
persistent error with no wait, and its continuous watcher-nudging keeps the interp
thread's event loop cycling at zero block time. The process cannot even implement its own
defence via fileevents - only a poll (see Workaround) can see the condition.

## Real-world triggers (legitimate application scenarios)

The reproduction below force-kills a conhost to make the trigger deterministic, but the
same client-side state - console server gone, client alive - arises in ordinary use.
Exposed application profile: any long-running Tcl program that reads console stdin via a
fileevent - a REPL, a monitor accepting operator commands, a server with console control
input - since the trigger is entirely outside the application's control:

- **A user kills a hung console window via Task Manager.** Killing "Console Window Host"
  (conhost.exe) is common folk-advice for a stuck window; TerminateProcess delivers no
  CTRL_CLOSE_EVENT, so the client survives against a dead console. (Identical mechanism
  to the scripted repro - verified.)
- **conhost.exe crashes.** The host is an ordinary user-mode process; when it dies of its
  own bug the client is orphaned exactly as above.
- **A terminal emulator crashes.** A clean close runs ConPTY/tab teardown that terminates
  the client tree, but a *crashed* terminal (Windows Terminal or third-party) never runs
  that cleanup; its console server goes away ungracefully. (Plausible-by-mechanism;
  untested here - the classic-conhost case is the verified repro.)
- **Automation/CI harnesses that kill their console host.** A harness that launches work
  under a hidden console (Start-Process -WindowStyle Hidden, scheduled tasks, agent
  runners) and later "cleans up" by killing the launcher or host process - without a job
  object that covers the clients - leaves the worker tclsh orphaned. This is how the bug
  was first hit in production use: an interactive shell under a hidden console outlived
  its host and was found 37 minutes later with ~1550 cpu-seconds accumulated, discovered
  only because it polluted a test harness's process-liveness checks.
- **Session teardown edge cases.** Any path where the session infrastructure tears down
  the console host before (or without) terminating its clients produces the same state.

In every case the failure compounds: each event adds one immortal spinner, and the
Observability section explains why nobody notices them.

## Steps to reproduce

Two files; the driver is one command. The payload writes a 1-second heartbeat so liveness
of the process and its event loop is provable independently of CPU sampling.

repro.tcl:

```tcl
#stdin fileevent + 1s heartbeat. Optional argv 0: heartbeat file path.
if {[llength $argv]} {
    set hbf [lindex $argv 0]
} else {
    set hbf [file join [file dirname [info script]] heartbeat.txt]
}
chan configure stdin -blocking 0
chan event stdin readable {
    set data [read stdin]
    puts stderr "readable fired: [string length $data] bytes eof=[chan eof stdin]"
    flush stderr
    if {[chan eof stdin]} {
        puts stderr "eof - exiting"
        exit 0
    }
}
proc heartbeat {} {
    set f [open $::hbf a]
    puts $f "[clock format [clock seconds] -format %H:%M:%S] alive - pid [pid]"
    close $f
    after 1000 heartbeat
}
heartbeat
puts stderr "armed - pid [pid]"
flush stderr
vwait ::forever
```

run_repro.ps1 (PowerShell; run from the directory containing repro.tcl):

```powershell
param([string]$Tclsh = 'tclsh90.exe')   # a Tcl 9.0.x tclsh
$dir = Join-Path $env:TEMP ("deadconsole_" + (Get-Date -Format HHmmss_fff))
New-Item -ItemType Directory $dir | Out-Null
$err = Join-Path $dir err.txt
$hb  = Join-Path $dir heartbeat.txt
# 1. launch the payload under its OWN dedicated CLASSIC conhost (bypasses Windows Terminal)
$con = Start-Process -WindowStyle Hidden -PassThru conhost.exe -ArgumentList `
    "$env:SystemRoot\System32\cmd.exe /c `"$Tclsh $PWD\repro.tcl $hb 2> $err`""
# 2. wait for the payload to arm and report its pid
$tclpid = $null
foreach ($i in 1..20) {
    Start-Sleep -Milliseconds 500
    if ((Test-Path $err) -and ((Get-Content $err -Raw) -match 'armed - pid (\d+)')) {
        $tclpid = [int]$Matches[1]; break
    }
}
if (-not $tclpid) { throw "payload never armed (see $err)" }
"conhost $($con.Id), tclsh $tclpid armed; baselining..."
$p = Get-Process -Id $tclpid; $c0 = $p.CPU; Start-Sleep 3; $p.Refresh()
"idle baseline: $($p.CPU - $c0) cpu-sec over 3s (expect ~0)"
# 3. force-kill the console server (TerminateProcess - no CTRL_CLOSE_EVENT is generated)
Stop-Process -Force -Id $con.Id
Start-Sleep 2
$p = Get-Process -Id $tclpid -ErrorAction SilentlyContinue
if (-not $p) { "tclsh terminated - not reproduced"; Get-Content $err; exit 1 }
$c1 = $p.CPU; Start-Sleep 10; $p.Refresh()
"SURVIVED. cpu rate: $(($p.CPU - $c1) / 10) cores over 10s"
$p.Threads | Sort-Object {$_.TotalProcessorTime} -Descending |
    Select-Object -First 3 Id, ThreadState, WaitReason, TotalProcessorTime
"stderr trail (no 'readable fired'/'eof' will have appeared):"; Get-Content $err
"heartbeat tail (still ticking - event loop alive, timers fire, only the fileevent is lost):"
Get-Content $hb -Tail 3
"kill the specimen when done:  Stop-Process -Force -Id $tclpid"
```

### Important: kill methods that do NOT reproduce (triage note)

The bug needs the console server to die *without* anything terminating the client. All of
the following are healthy paths in which the client is deliberately terminated, so they
will NOT show the bug:

- Closing a console/terminal window manually: clients receive CTRL_CLOSE_EVENT and are
  force-terminated after the grace period.
- Closing or killing a Windows Terminal tab (or its OpenConsole/ConPTY host): the terminal
  tears down the client tree on pseudoconsole close.
- Running the repro from an interactive shell inside Windows Terminal and killing "a
  conhost": the tclsh shares the tab's ConPTY; it has no dedicated classic conhost, and
  killing the tab's host lands in the previous bullet.

The driver above forces the reproducing topology: a dedicated classic conhost.exe as the
console server, force-killed by pid.

## Observed

- The tclsh survives and consumes ~1.85 cores indefinitely when it is the only orphan
  (one thread permanently Running - the console reader thread, hammering SetEvent - plus
  the interp thread woken continuously).
- err.txt shows only "armed - pid NNN": the readable handler NEVER fires after the console
  dies. No eof, no error - script level is completely blind.
- heartbeat.txt keeps ticking every second: the process and its event loop (timer events)
  are fully alive; only the channel notification is lost.
- Orphans accumulate. Concurrent orphans serialize on the kernel wake/handoff syscalls, so
  per-orphan burn drops but total grows: measured 4 concurrent orphans at ~1.15 cores each
  (4.6 cores aggregate); 8 concurrent produced a clear thermal (fan) response.

## Expected

The dead console is reported to the script (readable event whose read then yields EOF or
an error), and no driver thread busy-waits. The process can then close the channel and
exit or continue headless, idling at ~0 CPU.

## Observability (why this ships unnoticed)

On a 32-logical-processor workstation a single orphan is ~3-6% total CPU with no pegged
core anywhere: the scheduler migrates the spinning threads across LPs, so per-LP graphs
show only a diffuse ripple, and per-process CPU displays that are frequency-normalized
(Windows 11 "Processor Utility") under-report a load that is almost entirely kernel
wait/wake handoff and never drives clocks up - the affected processes were observed
showing 0 in Task Manager's Details CPU column while Get-Process/TotalProcessorTime showed
them burning more than a core each. The bug was originally found only because an orphan
polluted process-liveness checks in a test harness, not because anyone saw CPU load.

## Analysis (win/tclWinConsole.c, 9.1b1 sources; same code in 9.0.x)

When the console host dies, ReadConsoleW fails (observed GetLastError: ERROR_BROKEN_PIPE)
and the reader thread records it in handleInfoPtr->lastError. Two defects then interact:

**Defect 1 - the error notification is produced but never delivered.**
ConsoleSetupProc and ConsoleCheckProc both treat `lastError != ERROR_SUCCESS` as "input
ready" (block time 0 / queue a ConsoleEvent). But ConsoleEventProc computes the notify
mask ONLY from ring-buffer state:

```c
if ((chanInfoPtr->watchMask & TCL_READABLE)
        && RingBufferLength(&handleInfoPtr->buffer)) {
    mask = TCL_READABLE;
}
```

With the buffer empty and only lastError set, mask stays 0 and Tcl_NotifyChannel is never
called. The queued event is discarded - and CheckProc immediately queues another. The
interp thread therefore spins through Tcl_DoOneEvent at zero block time, queueing and
discarding console events forever, and the registered fileevent script never runs.
(ConsoleInputProc would correctly surface the error/EOF if a read ever happened - it
never does, because the script is never notified.)

**Defect 2 - the reader thread's persistent-error path has no wait.**
In ConsoleReaderThread's main loop, the "notify clients" branch runs when
`inputLen > 0 || handleInfoPtr->lastError != 0`. For the data case the buffer drains and
the thread subsequently blocks in SleepConditionVariableSRW. For the persistent-error case
nothing resets lastError (ConsoleInputProc clears it only on a read, and for
ERROR_INVALID_HANDLE not even then), so every iteration takes the branch:
WakeAllConditionVariable + NudgeWatchers + `continue` - no wait anywhere on the path. The
thread hard-loops calling SetEvent (via Tcl_ThreadAlert in NudgeWatchers) until the channel
is closed (numRefs == 1) - which never happens, because of Defect 1. One full core, plus
the wakeups feeding the Defect-1 spin on the interp thread.

Related contributing quirk: ConsoleDataAvailable returns -1 on PeekConsoleInputW failure
and its caller treats the result as boolean, so a dead console reads as "data available",
which is what triggers the failing ReadConsoleChars and sets lastError in the first place.
That happens to be load-bearing for eventually noticing the error at all, but the -1/true
conflation looks unintended.

## Suggested fix

1. ConsoleEventProc: also set `mask = TCL_READABLE` when
   `(chanInfoPtr->watchMask & TCL_READABLE) && handleInfoPtr->lastError != ERROR_SUCCESS`
   (matching the condition SetupProc/CheckProc already use). The script then gets the
   readable event, the read surfaces EOF/error, and a well-behaved script closes the
   channel, which lets the reader thread exit.
2. ConsoleReaderThread: after notifying watchers of an error with an empty private buffer,
   wait on the condition variable (as the no-data path does) instead of continuing the
   loop - the error is persistent; re-announcing it in a hot loop serves nothing. This
   protects processes whose scripts never read/close (e.g. no fileevent registered) from
   the one-core spin as well.

Either half alone materially improves matters; both together make the failure mode clean:
console dies -> readable -> eof on read -> script closes/exits, no busy-wait meanwhile.

## Workaround (script level, until fixed)

Fileevents cannot see the condition, so poll: `chan configure stdin -inputmode` performs a
live GetConsoleMode and errors iff the console is gone; on failure close the console
channel(s) - closing is what stops the reader-thread spin - and exit or continue headless.
A 5-second poll made orphans of the reporting application exit within seconds of console
death in testing, with no observable effect on live-console interactive behaviour.