Where should AI sit inside industrial systems?

Research question

Given the sensor and error evidence just before a known breakdown, can an LLM name which of four components failed, and does it beat the classical tools used for this today?

Design & configuration

• Microsoft-simulated Azure Predictive Maintenance dataset; 213 held-out breakdowns, time-based split
• Three conditions per model: zero-shot, with train-period history, and a disguised-names contamination probe
• Models: gpt-5.4 · gpt-oss:120b · nemotron-3-super:120b
• Five baselines including a recent-error rule and a trained logistic-regression classifier
• Macro-F1 with 95% machine-cluster bootstrap confidence intervals

Key findings

• The trained classifier (0.995) and a one-line error rule (0.923) beat every model.
• Zero-shot LLMs trailed at 0.36–0.44 macro-F1 — near the trivial floors.
• A page of plant history lifted them to 0.86–0.91, but none exceeded the one-line rule.
• The disguised-names probe found no contamination signal for two of the three models.

Why it matters

For tabular sensor diagnosis of this shape, a cheap classical model is the right tool. A language model becomes competitive only once given the site’s own history, and even then should be measured against the simple rule that history implies. Result scoped to this simulated, same-fleet task.

Research question

Can a model improve supply-chain stability when it selects the exponential-smoothing parameter α while the deterministic formula calculates the order?

Design & configuration

• Three-tier serial supply-chain simulation; 25-month synthetic demand, deterministic lead times
• The AI selects α from {0.1, 0.3, 0.5, 0.7} each period; the formula places the order
• Models: gpt-4.1-mini · o4-mini · GPT OSS 120B, across blind, context, and stateful conditions (plus four debiased V6b conditions)
• Compared against a fixed α = 0.3 exponential-smoothing baseline
• 10 runs per primary condition; 5 per V6b debiasing condition

Key findings

• Every AI condition produced chain OVAR below 1.0 — damping variance, not amplifying it.
• Best condition: GPT OSS 120B blind at OVAR 0.535 ± 0.048 with 4.4 stockouts.
• The fixed α = 0.3 baseline reached OVAR 0.545 with 5 stockouts.
• The margin is small — evidence of matching the baseline, not of broad superiority.
• Context still hurt: models given context chose more reactive smoothing (α-inflation).

Why it matters

This supports a bounded role for AI inside industrial control loops: the model adapts one parameter while the deterministic method keeps execution discipline. It is the first positive result in my replenishment experiments.

Research question

If the Order-Up-To formula is handed perfect, ground-truth demand labels — no LLM at all — can it finally beat exponential smoothing?

Design & configuration

• Same three-tier replenishment simulation as the intent-classification studies
• Oracle (ground-truth) labels and causal rule-based classifiers — no LLM calls
• Fourteen architectural variants: multiplier maps, NEUTRAL redefinitions, order dampening, forecast oracle
• Compared against exponential smoothing (OVAR 1.193 in this setup) and the formula running alone
• 20 replications per variant

Key findings

• Oracle labels produced OVAR 1.776 — worse than the formula with no AI at 1.753.
• The best of the fourteen variants still sat 0.540 OVAR above exponential smoothing; the rest ran from 0.556 to 1.120 above it. Perfect labels did not close the gap.
• The ceiling is a structural property of the safety-stock formula, not a model-quality problem.
• The result applies to this Order-Up-To architecture; it says nothing about other control levers.

Why it matters

Perfect intelligence in the wrong position is worse than no intelligence. This closed the intent-classification line (V1–V5) and motivated moving the AI to a different control lever in V6.

Research question

Does confining the AI to a clean five-label classification task keep orders steadier than a standard forecasting formula?

Design & configuration

• Three-tier chain; 25-month no-disruption Indian automotive demand series, one-month lead times
• gpt-4.1-mini (Azure) picks one of five labels; a fixed label→multiplier table and an Order-Up-To formula compute the order
• Three information conditions: blind, context, stateful
• Compared against exponential smoothing at OVAR 0.545
• 20 replications per condition

Key findings

• Every condition amplified order swings: chain OVAR 3.268 ± 0.062 to 3.843 ± 0.129 — roughly 6–7× the baseline.
• Label compliance was perfect: 1.000, zero fallbacks across every run.
• The three information conditions barely separated — the fixed lookup table capped what any condition could do.
• Single model and single synthetic series; the numbers do not generalise beyond this setup.

Why it matters

A reliable interface is not a safe architecture. Structured output solved the format problem and left the control problem untouched.

Research question

Does giving the AI a clean five-label intent classification, instead of a raw number, let it lower order variance?

Design & configuration

• 36-month simulation with stochastic lead times and world events
• Four models from 14B to 120B: gpt-4.1-mini · o4-mini · phi4:14b · nemotron-super 120B
• Blind, context, and unstructured conditions; a hard-coded multiplier lookup replaces the AI’s continuous output
• Compared against the Order-Up-To formula baseline
• 10 runs per local-model cell, 20 per frontier-model cell, 100 per baseline

Key findings

• All four models produced OVAR within a 0.054-unit band (1.726–1.780) across 12 model-condition cells, clustering on the formula baseline.
• Context roughly doubled direction accuracy (0.41–0.48 → 0.72–0.84) without materially moving OVAR.
• A neutral-prior prompt moved mean OVAR by only −0.015 to +0.005 across six matched cells — small against within-cell SDs of 0.088–0.133, but not zero, and I ran no equivalence test.
• The equalisation is a property of this lookup-table architecture, not a general law.

Why it matters

Model capability and better information were discarded at the lookup table. You cannot prompt your way out of the bullwhip effect when the architecture caps what any answer can do.

Research question

If the formula executes the orders and the AI only adjusts the safety-stock buffer, does the hybrid beat the baseline?

Design & configuration

• Three-tier serial chain; blind, context, and stateful conditions
• Models: gpt-4.1-mini · o4-mini · nemotron-super-3:120b
• Compared against exponential smoothing at OVAR 0.5446 with 5 stockouts
• 20 replications per condition; 12,960 LLM calls

Key findings

• All four hypotheses rejected. Best AI condition (gpt-4.1-mini, blind): OVAR 2.3325 with 10.6 stockouts.
• Worst condition (o4-mini, stateful): OVAR 3.1211 — memory of past stockouts triggered violent over-correction.
• Context increased order variance for two of three models.
• A bounded role was not enough on its own: the safety-stock buffer was still the wrong control surface.

Why it matters

Hybrid is not safe by default. Which variable the model controls matters more than the fact that a formula executes the orders.

Research question

Can any LLM configuration — frontier or local, lightweight or reasoning — beat simple heuristic ordering rules in a serial supply chain?

Design & configuration

• Three-tier serial supply-chain simulation
• Four LLM configurations across cloud and local inference: gpt-4.1-mini · o4-mini · phi4:14b · GPT OSS 120B
• Compared against three deterministic heuristic baselines, including exponential smoothing
• 20 replications per cell across eight model-condition cells; 11,520 LLM calls

Key findings

• Exponential smoothing: chain OVAR 0.545 with 5 stockouts. Best LLM configuration: OVAR 4.33 with 41 stockouts.
• Every heuristic outperformed every LLM configuration on both metrics simultaneously; all seven hypotheses rejected.
• Reasoning-tier models generated over 1,000,000 reasoning tokens with no measurable improvement in ordering performance.
• The result applies to this controlled serial-chain setup; it is not a universal claim about LLMs.

Why it matters

More capable models cannot simply be dropped into operational decision loops and expected to beat the established method. This result anchors everything I have run since.

Research question

Does a model trained on Indian data read Indian seasonal demand better than a frontier reference — and can either beat the formula?

Design & configuration

• Synthetic Indian automotive demand series: monsoon slump, Diwali peak, financial year-end surge
• sarvam-30b (30B total / 2.4B active) vs a GPT OSS 120B reference from the baseline study (not co-run)
• Two conditions, 10 replications each; 1,440 LLM calls
• Compared against exponential smoothing

Key findings

• No measurable difference: sarvam-30b chain OVAR 4.504 vs GPT OSS 120B 4.52 — within noise.
• Neither model detected the Indian seasonal demand patterns.
• Exponential smoothing reached OVAR 0.545 — roughly 8× better than both models.
• The GPT OSS figure is a context reference from the earlier run, not a co-run comparison.

Why it matters

Model provenance did not change operational behaviour. Training-data locality is not a substitute for the right control architecture.

Research question

Does more domain context, or more reasoning capability, reduce how much an LLM amplifies demand swings?

Design & configuration

• Three-tier serial chain; the LLM places orders directly
• 2×2 design: gpt-4.1-mini × o1, blind × context
• 5 replications per condition; 720 LLM calls

Key findings

• Every configuration amplified demand swings.
• Context reduced chain OVAR for the lightweight model by 7.5% (3.149 → 2.914, four runs) and increased it for the reasoning model by 15.0% (3.835 → 4.412).
• The context + reasoning condition fully inverted the tier pattern (OEM OVAR 6.349, Component 2.698).
• Directional only (n = 5); this motivated the expanded baseline study.

Why it matters

My first evidence that context interacts with the decision role instead of simply improving it.

Research question

Can a model small enough to run locally — on a box beside the machines, no cloud — do real fault diagnosis, and what decides whether its output is usable?

Design & configuration

• IBM’s public FailureSensorIQ benchmark: 2,667 multiple-choice fault-diagnosis questions
• Four 4B-class local models: Nemotron-3 Nano, Gemma 3 4B, and Phi-4-mini (standard and reasoning)
• Local inference on workstation hardware — footprint evidence, not constrained-edge validation
• Reference points: ~27.5% blind-guessing floor, ~60.2% human-expert mean
• Scored for accuracy and output reliability separately

Key findings

• Best clean full-set result: Nemotron-3 Nano 4B at 51.8% with structured output on all 2,667 calls.
• Above the guessing floor, below the human-expert mean — assistance, not autonomy.
• Sampling temperature moved Phi-4-mini from 30.7% to 36.6% (0.8 → 0.3) on the same weights — a single-model contrast.
• One run degraded because of serving-runtime state, not model weights, and was excluded from the model comparison.

Why it matters

On the edge, the integration decides usability as much as the model: sampling settings, output format, and serving health. The evidence supports decision assistance with human oversight, not autonomous diagnosis.