Automation or AI?

Why This Confusion Matters

Picture this: three Product Controllers walk into a (virtual) status meeting. All three proudly announce they're "using AI to fix breaks."

One has lovingly crafted a spreadsheet macro that would make an Excel Olympian weep—complete with a heroic IF statement.

The second has fed two years of break history into a machine-learning model that actually sweats over the data.

And the third? They've simply strapped a chatbot to their break tracker and called it a day.

All three walk away with the same "AI" badge of honor on their quarterly slide deck—and that's where the fun (and the budget blow-ups) begin.

This linguistic laziness isn't just pedantic—it's expensive. Budgets get greenlit for Skynet-level infrastructure when all you really needed was a coffee-fueled macro. Governance gets tossed out the window because "it's just automation, folks," while a humble rule-based fix ends up cooling its heels in the model-risk queue alongside full-blown neural networks, waiting for an approval it never should have needed in the first place.

Enter this guide. We've taken the top five "AI" use cases that Product Control teams actually ask for and run them through a simple, three-question reality check:

The goal? A working vocabulary that lets you walk into any scoping meeting—whether with your tech partner, your model-risk guardian, or your slightly sceptical CFO—and confidently say, "This is what we're building, this is why, and here's exactly which governance track it belongs on." No more surprises. No more mislabelled macros. Just clear, professional, and mildly entertaining clarity for the right solution, with the right oversight, for the right problem.

A working definition

Before the five use cases, it helps to fix three terms that get used interchangeably but describe genuinely different things:

The five use cases, and which category each one really is

The five use cases below were chosen deliberately to span the full spectrum — one is pure automation, three are different flavours of machine learning (supervised classification, unsupervised anomaly detection, and forecasting), and one is generative AI built on top of the other four's outputs. Seeing them side by side should make the distinctions concrete rather than theoretical.

The Five Use Cases at a Glance

Note the pattern across the last column: governance weight rises as the system moves from deterministic rules, to learned-but-numeric predictions, to open-ended language generation. This is the single most useful takeaway for scoping any future use case — ask which category it falls into before deciding how much validation, documentation and human review it needs.

Use Case 1: Missing-Feed Detection

Problem Statement

P&L and reconciliation processes depend on dozens of upstream feeds — trade capture, market data, risk sensitivities, static data — landing on time and complete. A feed that is late, partial, or silently empty can cascade into wrong P&L or a missed control, and it is typically discovered hours later when someone notices the numbers 'look wrong' rather than at the moment the feed actually failed.

Is This Automation or AI? (And Why)

This is automation, not AI, and it is important to say so plainly. The logic needed — 'has this feed arrived by its expected time, and does its row count fall within a normal range for this day of the week' — can be written completely as explicit rules with named thresholds. There is no pattern being learned from data that could not be written down as a checklist; a human operations analyst, given the same SLA table, would make exactly the same call every time. Calling this 'AI' over-states what is happening and risks routing a simple, highly deterministic control through an AI model-governance process it does not need.

AI / Automation Implementation Approach

Build a rules engine that checks each feed's arrival time against its documented SLA and its row count against a statistically-derived 'normal range' (which itself can be calculated with simple historical averages, not machine learning).

• Maintain one central SLA table per feed: expected arrival time, grace period, and expected row-count range, owned by data governance.
• Run the check on a schedule (e.g. every 5 minutes from 06:00) rather than once at a fixed cut-off, so a late feed is caught as early as possible.
• Tier severity by the feed's downstream criticality (a regulatory-capital feed is higher severity than an ancillary reference feed) and route alerts accordingly.

How the Solution Works, Step by Step

Illustrative Code

# This is a rules engine, not a trained model - every line below is an
# explicit, human-defined threshold, not something "learned" from data.
sla_table = load_feed_sla_table()   # owned & version-controlled by data governance

for feed_name, sla in sla_table.items():
    arrival = get_feed_arrival_time(feed_name, business_date)
    if arrival is None or arrival > sla["expected_time"] + sla["grace_period"]:
        raise_alert(feed_name, severity=sla["severity"], reason="late_or_missing")
        if sla["severity"] == "high":
            hold_downstream_process(sla["downstream_processes"])
        continue

    row_count = get_row_count(feed_name, business_date)
    expected_low, expected_high = sla["expected_row_range"]
    if not (expected_low <= row_count <= expected_high):
        raise_alert(feed_name, severity=sla["severity"], reason="row_count_out_of_range")

Required Controls

• The SLA table is owned by a named data-governance function and sits under standard change control — not adjustable ad hoc by whoever is on call that day.
• High-severity feed failures trigger a hard gate that blocks downstream close processes automatically, rather than relying on someone remembering to hold the process manually.
• Every alert, its resolution time, and whether any override was applied is logged, feeding operational-resilience MI.
• Because this is deterministic automation, testing is straightforward: run the rule against a full year of historical feed-arrival data and confirm it would have caught every known historical incident.

Worked Example

Common Mistake to Avoid

Use Case 2: FOBO Break Classification

Problem Statement

Front-Office-to-Back-Office (FOBO) breaks arrive as a long, undifferentiated list every day. Analysts spend the first hour of every close simply figuring out which breaks are pricing differences, which are timing, which are static-data mismatches, and which are genuine errors, before any real investigation starts.

Is This Automation or AI? (And Why)

This is genuine AI/ML, specifically supervised classification. Unlike a feed-arrival check, there is no simple, complete rule an analyst could write down that reliably separates a 'pricing difference' break from a 'timing' break from a 'genuine error' — the real signal is a complex combination of break size, product type, desk, recurrence pattern and free-text description that a human classifies through years of accumulated pattern recognition, not through an explicit checklist. That is precisely the situation a trained ML model is suited to: it learns the pattern from thousands of past examples that a human analyst already labelled correctly.

AI / Automation Implementation Approach

Train a supervised classification model (gradient-boosted trees such as XGBoost/LightGBM are a strong, explainable default for this kind of structured/tabular problem) on historical, analyst-labelled breaks, using break size, product type, desk, age, recurrence, and text similarity to previously-labelled categories as features. The model outputs a category and a confidence score — never a final, unreviewable decision.

• Start with a clean, sufficiently large labelled dataset — this is the single biggest determinant of model quality.
• Use a model type that supports feature-importance explanations so a controller can see why a given category was suggested.
• Set a confidence threshold below which a break is routed straight to manual triage rather than auto-tagged with a low-confidence guess.

How the Solution Works, Step by Step

Illustrative Code

# This IS a trained model - it learns the classification pattern from labelled
# history rather than following rules a human wrote down in advance.
from lightgbm import LGBMClassifier

features = ["abs_break_amount", "pct_of_position", "product_type", "desk_id",
            "break_age_days", "recurred_yesterday", "source_system_pair",
            "text_embedding_similarity_to_labelled_set"]

model = LGBMClassifier(n_estimators=300, max_depth=6, random_state=7)
model.fit(X_train[features], y_train)   # y_train = analyst-confirmed categories

prediction = model.predict_proba(new_break[features])
# -> {"pricing_difference": 0.7, "timing": 0.8, "genuine_break": 0.06, ...}
if prediction.max() < 0.70:
    route_to_manual_triage(new_break)
else:
    suggest_category(new_break, category=prediction.idxmax(), confidence=prediction.max())

Required Controls

• Every break keeps a 'Controller Confirmed Category' field, distinct from the model's suggested category; only the confirmed field feeds MI, audit reports and any downstream automation.
• Confidence thresholds route low-confidence predictions to a human queue rather than auto-tagging them, and the threshold itself is set and owned by the control owner, not the model.
• Model precision/recall per category is tracked monthly against controller overrides; drift beyond an agreed tolerance triggers a retraining review.
• The model sits in the firm's model inventory with a named owner and a documented risk tier.

Worked Example

Common Mistake to Avoid

Use Case 3: Anomaly Detection

Problem Statement

Genuine P&L or balance-sheet errors are often statistically unusual before anyone manually notices them — a book that never has vega P&L suddenly showing a large vega number, a reserve that jumps 5× month-on-month. Waiting for a human to eyeball every line before flagging an issue is slow and inconsistent across analysts and desks.

Is This Automation or AI? (And Why)

This is AI/ML, but a different flavour from Use Case 2 — unsupervised learning rather than supervised classification. There is no labelled dataset of 'this was an anomaly, this was not' for every possible P&L pattern, because true errors are rare and often previously unseen in exactly that form. Instead, the model learns what 'normal' looks like for a given book from its own history, without being told in advance what an anomaly looks like, and flags anything that deviates meaningfully from that learned normal pattern.

AI / Automation Implementation Approach

Use an unsupervised model — an Isolation Forest or an Autoencoder are both reasonable, well-understood choices — trained on a rolling window of each book's own historical P&L components, scoring each new day's pattern against that learned baseline.

• Train per book or per book-family, not one global model — 'normal' for an exotics book looks nothing like 'normal' for a flow book.
• Set the model's sensitivity (contamination rate) deliberately and document the rationale — too sensitive creates alert fatigue, too loose misses genuine issues.
• Exclude known one-off events from the training window so a genuine one-off doesn't permanently widen what the model treats as 'normal'.

How the Solution Works, Step by Step

Illustrative Code

# This IS a trained model - it learns "normal" for THIS book from its own
# history; nobody wrote down in advance what an anomaly looks like.
from sklearn.ensemble import IsolationForest

X = pnl_history[["delta_pnl", "gamma_pnl", "vega_pnl", "theta_pnl", "new_trade_pnl"]]

model = IsolationForest(n_estimators=200, contamination=0.02, random_state=7)
model.fit(X)   # unsupervised - no "correct answer" labels used at all

today_score = model.decision_function(today_features)
if today_score < threshold:
    raise_anomaly_flag(book_id, today_features, today_score)

Required Controls

• Anomaly flags are risk-ranked, not binary pass/fail, and every flag shows the controller which specific feature(s) drove the score, so the flag is explainable rather than a black-box alert.
• Sensitivity parameters are set and documented by the control owner, with the rationale recorded.
• The model is retrained on a rolling window that excludes known one-off events.
• Anomaly detection supplements, and never replaces, existing hard-coded limit and tolerance checks already required by the control framework.

Worked Example

Common Mistake to Avoid

Use Case 4: Reserve Forecasting

Problem Statement

Reserves (bid-offer, close-out cost, model, liquidity) are typically calculated at month-end using a snapshot process, giving Product Control little advance warning of a large swing until it's already late in the close. Forecasting the likely direction and rough size of reserve movements earlier in the cycle would let the team pre-empt large surprises rather than react to them on close day.

Is This Automation or AI? (And Why)

This is AI/ML — specifically a forecasting or regression problem, the third distinct flavour in this guide alongside classification and anomaly detection. The model is asked to predict a number (the likely reserve movement) from a set of drivers (position size changes, bid-offer spread widening, volatility regime, seasonal patterns), a genuinely different task from either classifying a break into a category or flagging a pattern as unusual. It counts as AI because the relationship between the drivers and the eventual reserve outcome is learned from historical data rather than expressed as a fixed formula.

AI / Automation Implementation Approach

Train a regression model (gradient boosting, or a simpler time-series model such as Prophet/SARIMA per reserve type) on historical month-end reserve outcomes and the intra-month drivers available at the time, producing a forecast with a confidence interval rather than a single point estimate.

• Be explicit from day one that this forecast is an early-warning signal for planning, and is never a substitute for the official, independently-validated reserve calculation methodology.
• Include a confidence interval, not just a point forecast — a range communicates the genuine uncertainty far better than a single number.
• Track forecast accuracy against actual outcomes over time as a first-class monitoring metric.

How the Solution Works, Step by Step

Illustrative Code

# This IS a trained model - it learns the relationship between mid-month
# drivers and the eventual reserve outcome from historical examples.
from sklearn.ensemble import GradientBoostingRegressor

features = ["position_notional_change_mtd", "bid_offer_spread_bp",
            "vol_regime_flag", "days_to_quarter_end", "prior_3m_avg_reserve"]

model = GradientBoostingRegressor(n_estimators=250, max_depth=4, random_state=7)
model.fit(X_train[features], y_train)   # y_train = historical actual reserve deltas

forecast = model.predict(current_month_features)
lower, upper = compute_prediction_interval(model, current_month_features)
notify_stakeholders(book="EQ_Exotics", forecast=forecast, interval=(lower, upper))

Required Controls

• Forecasts are explicitly labelled as early-warning indicators for planning purposes; the actual month-end reserve figure continues to run through the approved, independently-validated methodology unchanged.
• A forecast that diverges materially from the eventual calculated reserve is investigated and logged.
• Forecasting outputs are shared as a heads-up only, with clear labelling that distinguishes them from the reported, governed reserve number.
• Model inputs and forecast accuracy are reviewed on a scheduled basis (e.g. quarterly) against actual outcomes.

Worked Example

Common Mistake to Avoid

Use Case 5: Automatic P&L Commentary

Problem Statement

Every desk-level P&L, at flash and at formal close, needs a written explanation of what moved and why — market moves, new trades, funding, corporate actions. Analysts currently write this by hand, cross-referencing trade blotters, risk reports and market data one desk at a time, which is 30–60 minutes of repetitive drafting before any real analysis begins.

Is This Automation or AI? (And Why)

This is generative AI, and it is the only use case in this guide that is. The distinguishing feature is the output itself: a piece of natural-language text, not a number, category or flag. That is precisely what large language models are built for, and precisely why this use case needs a different, and generally heavier, control approach than the previous four — a wrong number in a forecast is visibly a number you can sanity-check; a fluent, well-written but factually wrong sentence in a commentary is much easier for a busy reader to accept at face value.

AI / Automation Implementation Approach

Use a retrieval-augmented generation (RAG) pattern: pull structured P&L drivers (top trades by P&L impact, risk sensitivities times market moves, funding, flagged corporate actions) into a templated prompt, and instruct the model to narrate strictly from that data, explicitly stating 'unexplained: $X' for any residual it cannot attribute rather than inventing a plausible-sounding cause.

• Ground the model in a structured driver feed only — never the raw trade blotter or open internet/market knowledge.
• Require the model to state 'unexplained' explicitly for any residual rather than allowing it to force-fit a narrative.
• Add an automated post-generation check that extracts every number in the draft commentary and confirms it matches a number in the source driver feed.

How the Solution Works, Step by Step

Prompt + Validation Control

SYSTEM: You are a Product Control commentary assistant for an equity
derivatives desk. Write commentary using ONLY the structured data provided
below. Do not infer causes that are not present in the data. If P&L is
unexplained after matching to the driver list, state "unexplained: $X" and
stop - do not invent a plausible-sounding cause to fill the gap.

DATA:
{ "desk": "EQ Derivatives - Asia Flow", "date": "2026-07-24",
  "total_pnl": 482000, "drivers": [
    {"type": "delta", "amount": 30000, "detail": "HSI +.8%, net long delta 45mm"},
    {"type": "new_trade", "amount": 96000, "detail": "3 new autocallables booked"},
    {"type": "funding", "amount": -8000, "detail": "O/N funding cost, JPY book"},
    {"type": "vega", "amount": 44000, "detail": "implied vol -0.6pt, short vega book"}
  ], "unexplained": 50000 }

TASK: Produce a 4-6 sentence commentary suitable for the daily flash pack.

# Post-generation control (not the LLM's job - a separate, deterministic check):
def validate_commentary(generated_text, source_data):
    numbers_in_text = extract_numeric_claims(generated_text)
    source_numbers = flatten_numeric_values(source_data)
    unmatched = [n for n in numbers_in_text if not any(close(n, s) for s in source_numbers)]
    if unmatched:
        flag_for_human_review(generated_text, unmatched_numbers=unmatched)
        return False
    return True

Required Controls

• Commentary generation is grounded strictly in a structured driver feed — the model is never given free rein over the raw trade blotter or general market knowledge.
• Any residual P&L the driver feed cannot explain must be surfaced verbatim as 'unexplained' with the dollar amount; never force-fit a narrative.
• An automated, deterministic check (not the model itself) cross-verifies every number in the draft against the source data before a human ever sees it.
• A Product Controller reviews and signs off every commentary before it leaves the draft state; nothing is auto-published to traders, MI or regulators.
• Prompt and driver-feed versions are logged against each commentary so any later dispute can be traced to the exact inputs used that day.

Worked Example

Common Mistake to Avoid

Putting the Five Together: How They Actually Stack

In practice these five use cases are not five separate projects — they form a single pipeline where automation and different kinds of AI each do the part they are actually suited to, and generative AI sits at the very top narrating what the layers below it have already found.

Layer 1 — AUTOMATION (rules-based)
   Missing-Feed Detection: is today's data even reliable enough to proceed?
        |
        v
Layer 2 — AI / ML (supervised + unsupervised + forecasting)
   FOBO Break Classification | Anomaly Detection | Reserve Forecasting
        |  (structured outputs: categories, flags, forecasts — all numbers/labels)
        v
Layer 3 — GENERATIVE AI
   Automatic P&L Commentary: narrates Layers 1 and 2's structured outputs
   in plain language for a human reader — grounded strictly in what
   Layers 1 and 2 actually produced, nothing more

This layering is also the cleanest way to explain the governance framework to non-technical stakeholders: each layer only has to be as trustworthy as the layer above it needs it to be. Automation must be deterministic and testable. The ML layer must be accurate, monitored, and explainable enough for a human to sanity-check its confidence. The generative layer must be grounded in exactly what the layers below it produced, so it can add readability without adding risk.

A short decision checklist for your next use case

When a new AI idea comes up for your own team, work through these questions in order before committing to a build approach:

• Can the complete logic be written as explicit if-then rules a competent analyst could follow from a checklist? If yes, it is automation — build it as such, and do not over-engineer it with a model.
• Does the task require learning a pattern from historical examples that cannot be reduced to a simple rule, and does the output need to be a number, category, or flag? If yes, it is AI/ML — pick supervised classification, unsupervised anomaly detection, or forecasting depending on whether you have labels, are looking for outliers, or are predicting a future value.
• Does the output need to be a piece of written language — an explanation, a summary, an answer to a question? If yes, it is generative AI — and it needs grounding, an explicit 'say when you don't know' instruction, an automated fact-check step, and mandatory human sign-off before anything reaches a system of record.
• Does the answer combine more than one of the above? Most real Product Control use cases do — design it as a layered pipeline, rather than forcing a single technology to do a job better suited to a different layer.

Closing Note

None of the five use cases in this guide ask a Product Controller to hand over judgement to a system. The rules-based layer removes mechanical monitoring work. The machine-learning layer removes repetitive pattern-recognition work while keeping every prediction advisory and confidence-scored. The generative layer removes repetitive drafting work while remaining strictly grounded in what the layers beneath it have already established as fact. What changes for the Product Controller is not the standard of judgement required, but the volume of mechanical work standing between them and the point where that judgement actually gets applied.

The practical outcome of getting this distinction right is a much easier conversation with model risk, internal audit, and senior management: 'this part is a deterministic rule, tested against a year of history; this part is a monitored ML model with a confidence threshold and a named owner; this part is a generative draft that is grounded, fact-checked automatically, and never published without sign-off' is a far more defensible answer than 'we're using AI for this', however impressive the second phrase sounds in a steering committee deck.