PowerShell exit codes: scheduled scripts that fail loudly
The nightly job has been green for eight months. Somebody finally opens the output and it says "Processed 0 records. Done." It has said that for eight months, because the credential expired in the spring and the script treated an authentication failure the same way it treats a quiet night.
Powershell exit codes are the difference between monitoring and the appearance of monitoring. A scheduler cannot read your output. It reads one integer, and if that integer is always zero then every alerting rule built on top of it is decoration.
Why powershell exit codes end up wrong
Three specific traps, all of which I have shipped.
#Requires -Modules with pwsh -File. If the module is missing, the
requires statement stops the script before it runs. Reasonable. What is not
reasonable is that invoking the file this way returns exit code 0 in that
situation, so the scheduler sees a clean run of a script that never executed a
line. The fix is to check for the module in code and exit deliberately, keeping
#Requires -Version for the things that behave correctly.
A populated context that is not a valid session. Get-AzContext returns a
fully populated context object when the cached token behind it has expired. So
the obvious connection check passes, the enumeration returns nothing because
every call is failing, and the script reports no findings. The only reliable
test is to acquire a token and treat failure as fatal:
function Assert-AzConnection {
$ctx = Get-AzContext
if (-not $ctx -or -not $ctx.Account) {
Write-Error 'Not connected. Run Connect-AzAccount.'
exit 1
}
# A context object is not proof of a live token. Force an acquisition.
try { $null = Get-AzAccessToken -ErrorAction Stop -WarningAction SilentlyContinue }
catch { Write-Error "Cannot acquire a token: $($_.Exception.Message)"; exit 1 }
return $ctx
}
Non-terminating errors, which are most of them. Many cmdlets report failure
without throwing, so a try block never triggers and $? is the only sign
anything went wrong. Setting $ErrorActionPreference = 'Stop' at the top of a
script converts these to exceptions, which is almost always what you want in
something scheduled and unattended.
The distinction that matters
Every scheduled script needs to distinguish three outcomes, and most distinguish two.
Did the work, found nothing. Exit 0. This is a legitimate success and it is what a healthy estate looks like.
Did the work, found things. Exit 0, with the findings in the output. Findings are not failures; a cost script that exits non-zero because it found orphaned disks will be muted within a week.
Could not do the work. Exit non-zero. Authentication failed, a module is missing, an API returned 403, the subscription list came back empty when it should not have.
The third case is the one that gets collapsed into the first, and it is the expensive one. A clean report from an empty search looks exactly like good news, which is why it survives for eight months.
Worth adding a fourth if the script covers multiple scopes: partial success. Enumerated seven of nine subscriptions, failed on two. That is not a success and it is not a total failure, and exiting non-zero with the two named in the output is more useful than either extreme.
Logging that is worth having
Console output is not a log, because nobody kept it.
Write structured records rather than sentences. A line per operation with a timestamp, a level, the target, and the outcome. It does not need to be JSON, though JSON costs nothing and makes the file greppable by field later.
function Write-Log {
param(
[ValidateSet('INFO','WARN','ERROR')] [string] $Level,
[string] $Message,
[string] $Target = ''
)
$line = [pscustomobject]@{
ts = (Get-Date).ToUniversalTime().ToString('o')
level = $Level
target = $Target
message = $Message
}
$line | ConvertTo-Json -Compress | Add-Content -Path $script:LogPath
if ($Level -eq 'ERROR') { Write-Error $Message } else { Write-Verbose $Message }
}
Three things that make a log useful six months later. Timestamps in UTC with an offset, because a scheduled job on a machine that observes daylight saving will otherwise produce an hour you cannot reason about. The target of each operation, so a failure names the resource rather than the step. And a record of what was skipped and why, because "skipped" is the outcome that most often needs explaining and least often gets recorded.
Do not log secrets. Obvious, and it happens anyway when somebody logs a whole request object for debugging and leaves it in.
Exit codes worth using
A single non-zero code is enough to be alerted. Distinct codes are enough to know what happened without opening the log, which matters at 7am.
The convention I use, and any consistent one beats none:
- 0 ran, whatever it found
- 1 could not authenticate, or the token was not usable
- 2 a prerequisite was missing, such as a module or a parameter
- 3 ran but could not cover everything asked of it, the partial case
- 4 the target was not in the expected state, such as an account that was already deleted before an offboarding started
Keep them under 125 and away from 0, and avoid the range above 128 on Unix-like hosts, where those values carry signal meanings.
The value of separating 1 from 2 is that they have different owners. An authentication failure is usually a credential that expired, which is a person to chase and often predictable from the credential expiry note. A missing prerequisite is a host that changed, which is a different fix. Getting that from the exit code rather than from reading output saves the first five minutes of every incident, and those five minutes are when you decide how worried to be.
Making the scheduler act on it
Exit codes only matter if something reads them.
In an Azure Automation runbook, a non-zero exit or an unhandled terminating error marks the job Failed, and you can alert on job status. In a pipeline, the step fails and the run goes red. In Task Scheduler, the last result code is recorded and can trigger an action.
The check worth doing on anything you already have scheduled: deliberately break it. Revoke the credential, or point it at a subscription it cannot see, and confirm the scheduler goes red. If it goes green, you have found a job whose monitoring has always been decorative, and it will not be the only one.
That test takes ten minutes and is more informative than reading the script.
Where this sits
This is the property behind the read-only tooling in the Azure notes: a check that cannot look must not report a clean result. It applies identically to leaver automation, where a partially completed offboarding reporting success is the worst available outcome.
Microsoft's documentation on PowerShell error handling covers the terminating and non-terminating distinction properly, which is worth reading once because it is the root of most of this.
Automation handed over as part of consulting work is written to this standard, because a script that lies is worse than no script and takes longer to discover.