Posted to tcl by juliannoble2 at Sun Jul 12 18:09:06 GMT 2026view pretty
## Summary
When the console *server* (conhost.exe) serving a Tcl 9 process dies without the client
being terminated, a script with a `chan event stdin readable` handler is never notified:
the driver detects the error internally, but the event it queues for the interp thread is
discarded before `Tcl_NotifyChannel`, so no readable event, no EOF, no read error ever
reaches script level. Meanwhile the driver's console reader thread busy-loops on the
persistent error with no wait, and its continuous watcher-nudging keeps the interp
thread's event loop cycling at zero block time. The process cannot even implement its own
defence via fileevents - only a poll (see Workaround) can see the condition.
## Real-world triggers (legitimate application scenarios)
The reproduction below force-kills a conhost to make the trigger deterministic, but the
same client-side state - console server gone, client alive - arises in ordinary use.
Exposed application profile: any long-running Tcl program that reads console stdin via a
fileevent - a REPL, a monitor accepting operator commands, a server with console control
input - since the trigger is entirely outside the application's control:
- **A user kills a hung console window via Task Manager.** Killing "Console Window Host"
(conhost.exe) is common folk-advice for a stuck window; TerminateProcess delivers no
CTRL_CLOSE_EVENT, so the client survives against a dead console. (Identical mechanism
to the scripted repro - verified.)
- **conhost.exe crashes.** The host is an ordinary user-mode process; when it dies of its
own bug the client is orphaned exactly as above.
- **A terminal emulator crashes.** A clean close runs ConPTY/tab teardown that terminates
the client tree, but a *crashed* terminal (Windows Terminal or third-party) never runs
that cleanup; its console server goes away ungracefully. (Plausible-by-mechanism;
untested here - the classic-conhost case is the verified repro.)
- **Automation/CI harnesses that kill their console host.** A harness that launches work
under a hidden console (Start-Process -WindowStyle Hidden, scheduled tasks, agent
runners) and later "cleans up" by killing the launcher or host process - without a job
object that covers the clients - leaves the worker tclsh orphaned. This is how the bug
was first hit in production use: an interactive shell under a hidden console outlived
its host and was found 37 minutes later with ~1550 cpu-seconds accumulated, discovered
only because it polluted a test harness's process-liveness checks.
- **Session teardown edge cases.** Any path where the session infrastructure tears down
the console host before (or without) terminating its clients produces the same state.
In every case the failure compounds: each event adds one immortal spinner, and the
Observability section explains why nobody notices them.
## Steps to reproduce
Two files; the driver is one command. The payload writes a 1-second heartbeat so liveness
of the process and its event loop is provable independently of CPU sampling.
repro.tcl:
```tcl
#stdin fileevent + 1s heartbeat. Optional argv 0: heartbeat file path.
if {[llength $argv]} {
set hbf [lindex $argv 0]
} else {
set hbf [file join [file dirname [info script]] heartbeat.txt]
}
chan configure stdin -blocking 0
chan event stdin readable {
set data [read stdin]
puts stderr "readable fired: [string length $data] bytes eof=[chan eof stdin]"
flush stderr
if {[chan eof stdin]} {
puts stderr "eof - exiting"
exit 0
}
}
proc heartbeat {} {
set f [open $::hbf a]
puts $f "[clock format [clock seconds] -format %H:%M:%S] alive - pid [pid]"
close $f
after 1000 heartbeat
}
heartbeat
puts stderr "armed - pid [pid]"
flush stderr
vwait ::forever
```
run_repro.ps1 (PowerShell; run from the directory containing repro.tcl):
```powershell
param([string]$Tclsh = 'tclsh90.exe') # a Tcl 9.0.x tclsh
$dir = Join-Path $env:TEMP ("deadconsole_" + (Get-Date -Format HHmmss_fff))
New-Item -ItemType Directory $dir | Out-Null
$err = Join-Path $dir err.txt
$hb = Join-Path $dir heartbeat.txt
# 1. launch the payload under its OWN dedicated CLASSIC conhost (bypasses Windows Terminal)
$con = Start-Process -WindowStyle Hidden -PassThru conhost.exe -ArgumentList `
"$env:SystemRoot\System32\cmd.exe /c `"$Tclsh $PWD\repro.tcl $hb 2> $err`""
# 2. wait for the payload to arm and report its pid
$tclpid = $null
foreach ($i in 1..20) {
Start-Sleep -Milliseconds 500
if ((Test-Path $err) -and ((Get-Content $err -Raw) -match 'armed - pid (\d+)')) {
$tclpid = [int]$Matches[1]; break
}
}
if (-not $tclpid) { throw "payload never armed (see $err)" }
"conhost $($con.Id), tclsh $tclpid armed; baselining..."
$p = Get-Process -Id $tclpid; $c0 = $p.CPU; Start-Sleep 3; $p.Refresh()
"idle baseline: $($p.CPU - $c0) cpu-sec over 3s (expect ~0)"
# 3. force-kill the console server (TerminateProcess - no CTRL_CLOSE_EVENT is generated)
Stop-Process -Force -Id $con.Id
Start-Sleep 2
$p = Get-Process -Id $tclpid -ErrorAction SilentlyContinue
if (-not $p) { "tclsh terminated - not reproduced"; Get-Content $err; exit 1 }
$c1 = $p.CPU; Start-Sleep 10; $p.Refresh()
"SURVIVED. cpu rate: $(($p.CPU - $c1) / 10) cores over 10s"
$p.Threads | Sort-Object {$_.TotalProcessorTime} -Descending |
Select-Object -First 3 Id, ThreadState, WaitReason, TotalProcessorTime
"stderr trail (no 'readable fired'/'eof' will have appeared):"; Get-Content $err
"heartbeat tail (still ticking - event loop alive, timers fire, only the fileevent is lost):"
Get-Content $hb -Tail 3
"kill the specimen when done: Stop-Process -Force -Id $tclpid"
```
### Important: kill methods that do NOT reproduce (triage note)
The bug needs the console server to die *without* anything terminating the client. All of
the following are healthy paths in which the client is deliberately terminated, so they
will NOT show the bug:
- Closing a console/terminal window manually: clients receive CTRL_CLOSE_EVENT and are
force-terminated after the grace period.
- Closing or killing a Windows Terminal tab (or its OpenConsole/ConPTY host): the terminal
tears down the client tree on pseudoconsole close.
- Running the repro from an interactive shell inside Windows Terminal and killing "a
conhost": the tclsh shares the tab's ConPTY; it has no dedicated classic conhost, and
killing the tab's host lands in the previous bullet.
The driver above forces the reproducing topology: a dedicated classic conhost.exe as the
console server, force-killed by pid.
## Observed
- The tclsh survives and consumes ~1.85 cores indefinitely when it is the only orphan
(one thread permanently Running - the console reader thread, hammering SetEvent - plus
the interp thread woken continuously).
- err.txt shows only "armed - pid NNN": the readable handler NEVER fires after the console
dies. No eof, no error - script level is completely blind.
- heartbeat.txt keeps ticking every second: the process and its event loop (timer events)
are fully alive; only the channel notification is lost.
- Orphans accumulate. Concurrent orphans serialize on the kernel wake/handoff syscalls, so
per-orphan burn drops but total grows: measured 4 concurrent orphans at ~1.15 cores each
(4.6 cores aggregate); 8 concurrent produced a clear thermal (fan) response.
## Expected
The dead console is reported to the script (readable event whose read then yields EOF or
an error), and no driver thread busy-waits. The process can then close the channel and
exit or continue headless, idling at ~0 CPU.
## Observability (why this ships unnoticed)
On a 32-logical-processor workstation a single orphan is ~3-6% total CPU with no pegged
core anywhere: the scheduler migrates the spinning threads across LPs, so per-LP graphs
show only a diffuse ripple, and per-process CPU displays that are frequency-normalized
(Windows 11 "Processor Utility") under-report a load that is almost entirely kernel
wait/wake handoff and never drives clocks up - the affected processes were observed
showing 0 in Task Manager's Details CPU column while Get-Process/TotalProcessorTime showed
them burning more than a core each. The bug was originally found only because an orphan
polluted process-liveness checks in a test harness, not because anyone saw CPU load.
## Analysis (win/tclWinConsole.c, 9.1b1 sources; same code in 9.0.x)
When the console host dies, ReadConsoleW fails (observed GetLastError: ERROR_BROKEN_PIPE)
and the reader thread records it in handleInfoPtr->lastError. Two defects then interact:
**Defect 1 - the error notification is produced but never delivered.**
ConsoleSetupProc and ConsoleCheckProc both treat `lastError != ERROR_SUCCESS` as "input
ready" (block time 0 / queue a ConsoleEvent). But ConsoleEventProc computes the notify
mask ONLY from ring-buffer state:
```c
if ((chanInfoPtr->watchMask & TCL_READABLE)
&& RingBufferLength(&handleInfoPtr->buffer)) {
mask = TCL_READABLE;
}
```
With the buffer empty and only lastError set, mask stays 0 and Tcl_NotifyChannel is never
called. The queued event is discarded - and CheckProc immediately queues another. The
interp thread therefore spins through Tcl_DoOneEvent at zero block time, queueing and
discarding console events forever, and the registered fileevent script never runs.
(ConsoleInputProc would correctly surface the error/EOF if a read ever happened - it
never does, because the script is never notified.)
**Defect 2 - the reader thread's persistent-error path has no wait.**
In ConsoleReaderThread's main loop, the "notify clients" branch runs when
`inputLen > 0 || handleInfoPtr->lastError != 0`. For the data case the buffer drains and
the thread subsequently blocks in SleepConditionVariableSRW. For the persistent-error case
nothing resets lastError (ConsoleInputProc clears it only on a read, and for
ERROR_INVALID_HANDLE not even then), so every iteration takes the branch:
WakeAllConditionVariable + NudgeWatchers + `continue` - no wait anywhere on the path. The
thread hard-loops calling SetEvent (via Tcl_ThreadAlert in NudgeWatchers) until the channel
is closed (numRefs == 1) - which never happens, because of Defect 1. One full core, plus
the wakeups feeding the Defect-1 spin on the interp thread.
Related contributing quirk: ConsoleDataAvailable returns -1 on PeekConsoleInputW failure
and its caller treats the result as boolean, so a dead console reads as "data available",
which is what triggers the failing ReadConsoleChars and sets lastError in the first place.
That happens to be load-bearing for eventually noticing the error at all, but the -1/true
conflation looks unintended.
## Suggested fix
1. ConsoleEventProc: also set `mask = TCL_READABLE` when
`(chanInfoPtr->watchMask & TCL_READABLE) && handleInfoPtr->lastError != ERROR_SUCCESS`
(matching the condition SetupProc/CheckProc already use). The script then gets the
readable event, the read surfaces EOF/error, and a well-behaved script closes the
channel, which lets the reader thread exit.
2. ConsoleReaderThread: after notifying watchers of an error with an empty private buffer,
wait on the condition variable (as the no-data path does) instead of continuing the
loop - the error is persistent; re-announcing it in a hot loop serves nothing. This
protects processes whose scripts never read/close (e.g. no fileevent registered) from
the one-core spin as well.
Either half alone materially improves matters; both together make the failure mode clean:
console dies -> readable -> eof on read -> script closes/exits, no busy-wait meanwhile.
## Workaround (script level, until fixed)
Fileevents cannot see the condition, so poll: `chan configure stdin -inputmode` performs a
live GetConsoleMode and errors iff the console is gone; on failure close the console
channel(s) - closing is what stops the reader-thread spin - and exit or continue headless.
A 5-second poll made orphans of the reporting application exit within seconds of console
death in testing, with no observable effect on live-console interactive behaviour.