Anachronaut
0a2965bc63
Four things SplitLint wanted, and they build on each other. EVERY WARNING NAMES ITS RULE, in brackets at the end the way a compiler names the flag that produced it. Twelve rules, listed by --help. That makes the other three possible: suppressions can name one rule and leave the line honest about the others, the harness can assert on a rule's identity rather than on the wording of its message, and --machine can print one tab separated line per warning - file, line, rule, message, help - so nothing downstream reads prose. This file's own output was parsed with regular expressions three times in one day before it had a shape to rely on. A CLEAN RUN SAYS SO: No style warnings: 121 files checked against 12 rules. It used to exit in silence, which does not tell you it found nothing - it tells you nothing at all, and from outside the two are identical. A MARKER THAT SILENCES NOTHING IS ITSELF REPORTED, as dead-suppression. An exception that outlived whatever made it necessary is the thing the required reason exists to prevent, and naming the wrong rule now gets you both the warning you meant to silence and a note that your suppression is doing nothing. AND THE CORPUS IS HELD TO A BASELINE. Sixty one warnings are left in it deliberately and nothing stopped a sixty second. Tests/lint-baseline.txt records how many of each rule each file should produce, so a new one fails make test while the sixty one stay quiet; confirmed by adding an INIA 0d0 to Say.asm and watching it name the file, the rule and the count. It counts per file and rule rather than recording line numbers, because line numbers would churn the whole baseline whenever anything was inserted above a warning - the same reason cycle counts are stripped from recorded output here. ./Tests/lint.sh --bless records it again. One thing to know for next time: the rule name was inserted before the line number at all twenty one call sites, and the signature was changed to match rather than the twenty one call sites being fixed. (path, rule, line) reads no worse than (path, line, rule) and one edit has fewer ways to go wrong than twenty one.
283 lines
11 KiB
Bash
Executable File
283 lines
11 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# Checks that SplitLint finds what it claims to find, one rule at a time.
|
|
#
|
|
# This used to compare a TOTAL. Twenty three warnings came out and twenty three were
|
|
# expected, which is a number that stays right while the thing behind it goes wrong: a
|
|
# change that stopped one rule firing and made another fire twice would pass without a
|
|
# murmur, and so would a rule that moved to the wrong line. What is checked here now is
|
|
# WHICH warning came out and AT WHICH LINE, so a rule that stops working says which one it
|
|
# was.
|
|
#
|
|
# The fixture is written here rather than kept as a file because every line of it exists
|
|
# to trip exactly one rule, and a reader wants the pattern and the expectation side by
|
|
# side rather than in two places.
|
|
#
|
|
# Written by Anachronaut
|
|
|
|
set -u
|
|
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
|
|
cd "$ROOT" || exit 1
|
|
|
|
LINT="$ROOT/SplitLint"
|
|
[ -x "$LINT" ] || { echo "SplitLint is not built."; exit 1; }
|
|
|
|
# Recording the corpus as it stands, for when warnings have been deliberately fixed or
|
|
# deliberately accepted. Same shape as run.sh's --bless, and for the same reason.
|
|
if [ "${1:-}" = "--bless" ]; then
|
|
"$LINT" --machine $(find Programs -name '*.asm' | sort) \
|
|
| awk -F'\t' '{print $1"\t"$3}' | sort | uniq -c \
|
|
| awk '{printf "%s\t%s\t%s\n", $2, $3, $1}' | sort > "$ROOT/Tests/lint-baseline.txt"
|
|
echo "Recorded $(wc -l < "$ROOT/Tests/lint-baseline.txt" | tr -d ' ') file and rule pairs."
|
|
exit 0
|
|
fi
|
|
|
|
mkdir -p Tests/build
|
|
fixture=Tests/build/lint.asm
|
|
output=Tests/build/lint.out
|
|
|
|
GREEN=$'\033[32m'; RED=$'\033[31m'; RESET=$'\033[0m'
|
|
[ -t 1 ] || { GREEN=""; RED=""; RESET=""; }
|
|
|
|
PASS=0
|
|
FAIL=0
|
|
MISSING=()
|
|
|
|
printf '%s\n' \
|
|
'#Program' \
|
|
' INIA 0d0' \
|
|
' INIB 0x00' \
|
|
' INIA 0d1' \
|
|
' DPUA.2' \
|
|
' INIA 0x01' \
|
|
' DPDA.3' \
|
|
' PSHQ' \
|
|
' POPA' \
|
|
' PSHQ ; Comments do not break adjacent instructions.' \
|
|
' POPB' \
|
|
' PSHA' \
|
|
' POPA' \
|
|
' PSHB' \
|
|
' POPB' \
|
|
' INIA 0d5' \
|
|
' RSTA' \
|
|
' LDB.2' \
|
|
' MVQB' \
|
|
' INA ; Device reads are not removable assignments.' \
|
|
' INIA 0d8' \
|
|
' BRI reachable' \
|
|
' NOP' \
|
|
' ; A comment does not make fallthrough possible.' \
|
|
' NOP' \
|
|
'reachable:' \
|
|
' NOP' \
|
|
' BNA alreadyHere' \
|
|
' ; Blank space and comments do not move an address.' \
|
|
'alreadyHere:' \
|
|
' NOP' \
|
|
'knownValues:' \
|
|
' INIA 0d1' \
|
|
' NOP' \
|
|
' DPUA.1' \
|
|
' RSTA' \
|
|
' INIB 0d1' \
|
|
' NOP' \
|
|
' DPUW.1' \
|
|
' INIA 0d7' \
|
|
' NOP' \
|
|
' INIA 0d7' \
|
|
'pointerState:' \
|
|
' SETD.2 Thing' \
|
|
' NOP' \
|
|
' SETD.2 Thing' \
|
|
' SETD.1 Other' \
|
|
' DPUP.1 0d3' \
|
|
' DPDN.1 0d3' \
|
|
' SETD.1 Other' \
|
|
'carryState:' \
|
|
' CCF' \
|
|
' CCF' \
|
|
' INIA 0xFF' \
|
|
' INCA' \
|
|
' BRC carryTaken' \
|
|
' NOP' \
|
|
'carryTaken:' \
|
|
' CCF' \
|
|
' BRC carryClear' \
|
|
' BNC carryClear' \
|
|
' NOP' \
|
|
'carryClear:' \
|
|
' NOP' \
|
|
' "INIA 0d0; DPUA.0"' \
|
|
' ; INIB 0d0' \
|
|
'acrossACall:' \
|
|
' SETD.2 Thing' \
|
|
' CALL somewhere' \
|
|
' SETD.2 Thing' \
|
|
' INIA 0d9' \
|
|
' CALL somewhere' \
|
|
' INIA 0d9' \
|
|
> "$fixture"
|
|
|
|
"$LINT" "$fixture" > "$output" 2>&1
|
|
|
|
# ---- What every line of the fixture is for ----
|
|
#
|
|
# One row per rule: the line it should be reported at, and enough of the message to name
|
|
# the rule without pinning its exact wording.
|
|
expect() {
|
|
local line="$1" want="$2"
|
|
local got
|
|
got=$(grep -c "^${fixture}:${line}: style: ${want}" "$output")
|
|
if [ "$got" = "1" ]; then
|
|
PASS=$((PASS + 1))
|
|
printf " [%sok %s] line %-3s %s\n" "$GREEN" "$RESET" "$line" "$want"
|
|
else
|
|
FAIL=$((FAIL + 1)); MISSING+=("line $line: $want")
|
|
printf " [%sFAIL%s] line %-3s %s\n" "$RED" "$RESET" "$line" "$want"
|
|
fi
|
|
}
|
|
|
|
echo "Checking that SplitLint still finds each thing it knows about."
|
|
|
|
expect 2 "loading zero into A takes two bytes"
|
|
expect 3 "loading zero into B takes two bytes"
|
|
expect 5 "pointer offset is known to be one"
|
|
expect 6 "INIA leaves A at its known value of 1"
|
|
expect 7 "pointer offset is known to be one"
|
|
expect 8 "moving Q to A through the stack takes two instructions"
|
|
expect 10 "moving Q to B through the stack takes two instructions"
|
|
expect 12 "pushing A and immediately restoring it leaves A unchanged"
|
|
expect 14 "pushing B and immediately restoring it leaves B unchanged"
|
|
expect 16 "INIA assigns A, but RSTA replaces it immediately"
|
|
expect 18 "LDB.2 assigns B, but MVQB replaces it immediately"
|
|
expect 23 "no ordinary fallthrough reaches this instruction"
|
|
expect 25 "no ordinary fallthrough reaches this instruction"
|
|
expect 28 "branch target is the next labeled address"
|
|
expect 35 "pointer offset is known to be one"
|
|
expect 39 "word-sized pointer offset is known to be one"
|
|
expect 42 "INIA leaves A at its known value of 7"
|
|
expect 46 "DP2 is already known to hold Thing"
|
|
expect 50 "DP1 is already known to hold Other"
|
|
expect 53 "carry is already known to be clear"
|
|
expect 56 "BRC is always taken because carry is known set"
|
|
expect 60 "BRC is never taken because carry is known clear"
|
|
expect 61 "BNC is always taken because carry is known clear"
|
|
|
|
# ---- And a claim does not survive a call ----
|
|
#
|
|
# CALL really does restore A, B and DP0 to DP2, so both repeats below are genuinely
|
|
# redundant and the linter used to say so. It no longer does, on purpose: that advice is
|
|
# correct only while the callee is reached with CALL, and RCAL exists precisely so hot
|
|
# helpers can stop being one. 122 of the 178 redundant SETDs in the corpus were of this
|
|
# kind, and removing them would have become wrong the day a helper was made faster.
|
|
#
|
|
# THE LINE NUMBERS ARE THE REPEATS, not the label above them. Checking the label instead
|
|
# passes for free, because a label is never warned about by anything.
|
|
quiet=0
|
|
for line in 70 73; do
|
|
if grep -q "^${fixture}:${line}: style:" "$output"; then
|
|
FAIL=$((FAIL + 1)); MISSING+=("line $line should be quiet across a CALL")
|
|
printf " [%sFAIL%s] line %-3s no claim survives a CALL\n" "$RED" "$RESET" "$line"
|
|
else
|
|
PASS=$((PASS + 1)); quiet=$((quiet + 1))
|
|
printf " [%sok %s] line %-3s no claim survives a CALL\n" "$GREEN" "$RESET" "$line"
|
|
fi
|
|
done
|
|
|
|
# ---- And nothing it does not know about ----
|
|
#
|
|
# The fixture carries four lines that must stay quiet for a different reason: a device read
|
|
# whose result is overwritten, a self-restoring push pair that is NOT one, a string that
|
|
# happens to spell instructions, and a commented-out instruction. A rule that started
|
|
# firing on any of those would be a rule reading source that is not code.
|
|
#
|
|
# Every pass so far is either a warning that was expected or a line that stayed quiet, so
|
|
# the warnings that came out should be the passes less the quiet ones.
|
|
total=$(grep -c "^${fixture}:[0-9]*: style:" "$output")
|
|
if [ "$total" = "$((PASS - quiet))" ] && [ "$FAIL" = "0" ]; then
|
|
printf " [%sok %s] and nothing else %s warnings, no more\n" "$GREEN" "$RESET" "$total"
|
|
PASS=$((PASS + 1))
|
|
else
|
|
printf " [%sFAIL%s] and nothing else %s warnings for %s expectations\n" \
|
|
"$RED" "$RESET" "$total" "$PASS"
|
|
FAIL=$((FAIL + 1))
|
|
grep "^${fixture}:[0-9]*: style:" "$output" | while read -r line; do
|
|
echo " $line"
|
|
done
|
|
fi
|
|
|
|
# ---- Saying a warning is deliberate ----
|
|
#
|
|
# The reason is required. A marker with nothing after it is refused rather than honoured,
|
|
# because a suppression nobody explained outlives whatever made it necessary.
|
|
printf '%s\n' '#Program' ' INIA 0d0 ; splitlint: on purpose, for the test' > Tests/build/lintSup.asm
|
|
supOut=$("$LINT" Tests/build/lintSup.asm 2>&1)
|
|
if [ -z "$(echo "$supOut" | grep 'style:')" ] && echo "$supOut" | grep -q "1 warning suppressed"; then
|
|
PASS=$((PASS + 1)); printf " [%sok %s] a reasoned suppression is honoured and counted\n" "$GREEN" "$RESET"
|
|
else
|
|
FAIL=$((FAIL + 1)); MISSING+=("a reasoned suppression")
|
|
printf " [%sFAIL%s] a reasoned suppression is honoured and counted\n" "$RED" "$RESET"
|
|
fi
|
|
|
|
printf '%s\n' '#Program' ' INIA 0d0 ; splitlint:' > Tests/build/lintBare.asm
|
|
if "$LINT" Tests/build/lintBare.asm >/dev/null 2>&1; then
|
|
FAIL=$((FAIL + 1)); MISSING+=("a bare suppression is refused")
|
|
printf " [%sFAIL%s] a suppression with no reason is refused\n" "$RED" "$RESET"
|
|
else
|
|
PASS=$((PASS + 1)); printf " [%sok %s] a suppression with no reason is refused\n" "$GREEN" "$RESET"
|
|
fi
|
|
|
|
# ---- And warnings can be made to fail a build ----
|
|
if "$LINT" --fatal-warnings "$fixture" >/dev/null 2>&1; then
|
|
FAIL=$((FAIL + 1)); MISSING+=("--fatal-warnings")
|
|
printf " [%sFAIL%s] --fatal-warnings fails on a warning\n" "$RED" "$RESET"
|
|
else
|
|
PASS=$((PASS + 1)); printf " [%sok %s] --fatal-warnings fails on a warning\n" "$GREEN" "$RESET"
|
|
fi
|
|
|
|
# A suppressed warning is not a warning, so it must not fail one either.
|
|
if "$LINT" --fatal-warnings Tests/build/lintSup.asm >/dev/null 2>&1; then
|
|
PASS=$((PASS + 1))
|
|
printf " [%sok %s] and does not fail on a suppressed one\n" "$GREEN" "$RESET"
|
|
else
|
|
FAIL=$((FAIL + 1)); MISSING+=("--fatal-warnings on a suppressed warning")
|
|
printf " [%sFAIL%s] and does not fail on a suppressed one\n" "$RED" "$RESET"
|
|
fi
|
|
|
|
# ---- And the corpus has not grown a new warning ----
|
|
#
|
|
# Sixty one warnings are left in the corpus ON PURPOSE - registers whose equal values mean
|
|
# different things, idioms whose redundancy is what makes them self contained, arms of
|
|
# comparison chains that get reordered. Nothing stopped a sixty second from appearing.
|
|
#
|
|
# THE BASELINE IS COUNTS PER FILE AND RULE RATHER THAN LINE NUMBERS. Recording lines would
|
|
# churn the whole file every time something was inserted above a warning, which is the same
|
|
# reason a cycle count is stripped from every recorded output here.
|
|
baseline="$ROOT/Tests/lint-baseline.txt"
|
|
current=Tests/build/lint-current.txt
|
|
"$LINT" --machine $(find Programs -name '*.asm' | sort) \
|
|
| awk -F'\t' '{print $1"\t"$3}' | sort | uniq -c \
|
|
| awk '{printf "%s\t%s\t%s\n", $2, $3, $1}' | sort > "$current"
|
|
|
|
if [ ! -f "$baseline" ]; then
|
|
FAIL=$((FAIL + 1)); MISSING+=("the baseline is missing")
|
|
printf " [%sFAIL%s] the corpus baseline is missing\n" "$RED" "$RESET"
|
|
elif diff -q "$baseline" "$current" >/dev/null; then
|
|
PASS=$((PASS + 1))
|
|
printf " [%sok %s] the corpus matches its baseline %s file and rule pairs\n" \
|
|
"$GREEN" "$RESET" "$(wc -l < "$current" | tr -d ' ')"
|
|
else
|
|
FAIL=$((FAIL + 1)); MISSING+=("the corpus baseline")
|
|
printf " [%sFAIL%s] the corpus has moved away from its baseline\n" "$RED" "$RESET"
|
|
diff "$baseline" "$current" | sed 's/^/ /'
|
|
printf " To accept these, run: ./Tests/lint.sh --bless\n"
|
|
fi
|
|
|
|
echo
|
|
if [ "$FAIL" -eq 0 ]; then
|
|
echo "All $PASS SplitLint checks passed."
|
|
exit 0
|
|
fi
|
|
echo "$PASS passed, $FAIL failed: ${MISSING[*]}"
|
|
exit 1
|