This is my entry for the Faro Head of Data challenge: use Faro as an active trader, find one piece of data that is missing and would matter to a trader — not a nice-to-have dashboard stat — and take it from an idea to a working proof of concept. I started from a real trading question and then built the data to answer it, instead of starting from a dataset and looking for a use for it.
I also treated it as a product question, not just a deliverable. This page is a proposal for how a metric like this could live inside Faro — a detail view a trader opens from a signal to look at it properly (the price levels, the regime, how it has moved) instead of a single number on a dashboard. The same pattern would fit other Faro metrics; this is one worked example, running live on Hyperliquid BTC. The metric, the Airflow DAG and the full pipeline are in the repo: github.com/javiAI/faro-liquidation-pressure-map.
Where the open BTC leverage we track would be force-liquidated, by price.
Teal = longs (liquidate below price — a downside flush); red = shorts (liquidate above — a squeeze up). Gold line = current mark.
density is a smoothed profile (area ∝ notional); bars shows the exact per-level notional.
Hover for the amount; drag/scroll to zoom, ⤢ to expand.
The same liquidation field over time: rows are price levels, columns are time, brighter = more liquidable notional parked there.
The white line is the mark-price path.
Pick a resolution — 10-minute snapshots, hourly, or daily averages.
Left: the current Cascade Fragility Index as a 0–100 “fear gauge” — green calm, amber building, red fragile.
Right (top): that index (gold) against BTC price (white) over time, with regime bands — watch fragility build or bleed off as price moves.
Right (strip below): the long/short asymmetry on the same timeline. It is not a “go long / go short” signal — it shows which side of the book holds more liquidation fuel just past the current price:
· above 0 (red, “short-side”) — more short positions would be force-liquidated just above price; their forced buying can squeeze price up.
· below 0 (teal, “long-side”) — more long positions would be force-liquidated just below price; their forced selling can flush price down.
Near 0 means the two sides are roughly balanced.
Faro already shows Liquidation Volume — the liquidations that have already fired. The other half is missing: not what just got liquidated, but where the leverage that is still open would get liquidated next. That gap is the metric below.
Right now the Cascade Fragility Index reads 25.7/100 (elevated). The book is tilted to the long side — more longs would be liquidated just below price, which is downside-flush fuel, with about $236.76M liquidable within 5% of the $75,794 mark. Read: a real amount of fuel is gathering near the price, so it is worth watching for a trigger. This is built from the 35% of open interest we currently see.
The metric is a Liquidation Pressure Map for BTC perpetuals on Hyperliquid: a density of the price levels where currently-open leveraged positions would be force-liquidated, separated by side. Longs liquidate below the price; shorts liquidate above. On top of the map I compute two numbers a trader can actually watch:
Why this adds something Faro doesn't already have: it is the forward side of the Liquidation Volume Faro shows today. Volume is what already happened; this is risk that hasn't fired yet, with a price attached to it. They answer different questions, and a trader reads them together — one says the fuel is sitting there, the other says it caught.
It matters most for BTC perps, intraday to swing. In practice it changes where
you put a stop (don't leave it inside a cluster you could get run into), whether you fade or join a
move heading toward a cluster, and how big you go into crowded positioning. Source data
is all public and needs no key: metaAndAssetCtxs for the mark price (the price Hyperliquid
liquidates against), oracle, funding and open interest; clearinghouseState per wallet for
the signed size, the exchange's own liquidationPx, and the position notional. The catch:
no public endpoint returns the whole book, so the map is built from a sample of wallets (Appendix C).
The DAG (liquidation_pressure_dag.py) is thin on purpose: it sequences reusable
modules and does nothing clever itself. All the real logic — the API client, the metric, the storage
layer — lives in plain importable functions, so the same code runs whether Airflow drives it or the
small runner behind the live page does. That logic is unit-tested: pytest -q covers the
CFI and asymmetry maths (including that near-mark dust can't spike the index), the cleaning filters,
and the loss-resistant history — run it before any deploy. Flow:
refresh_universe → extract_market_context → extract_positions → validate → transform → load.
Scheduling cadence. Every ~10 minutes. Open positions and the mark price move continuously, so a stale map misleads; but positions don't turn over second to second and I won't hammer a free API, so 10 minutes is the balance between freshness and load. The wallet universe is refreshed on a far slower beat — once a day — because re-ranking the leaderboard is a ~30 MB download and the set of most active wallets barely changes within a day. The loop sleeps adaptively to hold the cadence even as a run takes longer.
Source extraction. Two sources, two cadences. The wallet universe comes from
Hyperliquid's public leaderboard feed (daily). Per run, clearinghouseState is queried
for every wallet. The /info endpoint is weight-limited to 1,200 weight per minute per
IP and that call costs 2, so the hard ceiling is 600 calls/min. I run a thread pool behind a shared
token bucket pinned at ~7.5 req/s (≈75% of the ceiling) — enough headroom that retries and the
occasional metaAndAssetCtxs (weight 20) never trip the limit. That fetches ~2,000
wallets in about 4.5 minutes with zero rate-limit errors.
Data validation. Two tiers. Hard checks fail the run and quarantine the data:
a non-positive mark, or mark/oracle drift beyond 5% (a sign something is wrong upstream). Soft
checks pass the data through but flag it. The per-position cleaning drops three things, each for a
reason I hit in the real data: liquidationPx = null (cross-margin positions, where the
trigger depends on the whole account and isn't placeable per position); dust below $10k notional
(which returns nonsense triggers — I watched a $1 position report a liquidation price of $1.8
trillion); and anything more than 60% away from the mark (numerically unreliable and irrelevant to
a near-term cascade). Every count is stored, never hidden.
Transformation logic. A pure function, so it's testable and the DAG calls it
directly: bucket each kept position by its liquidationPx, weight by notional, and take
the CFI as the notional-weighted average proximity using a smooth kernel K(d)=exp(−d/τ),
τ = 8%. I chose the exponential kernel over a raw 1/d precisely so a dust position
sitting on the mark can't send the index to infinity. Full formula in Appendix B.
Storage and output-table design. Three artifacts, each matched to its job:
The part worth calling out is how the series is stored: an append-only ledger with union semantics and atomic writes, never a read-modify-write. Each run appends one immutable line first, then rebuilds the CSV view as the union of (ledger ∪ existing CSV ∪ new row). A stale or empty read can therefore add a point but never drop one — which is exactly the failure you worry about in something that keeps accumulating for months. In production these become warehouse tables; the shapes are the same.
Failure handling and observability. Tasks retry with exponential backoff. Per-wallet errors are tolerated and counted rather than failing the whole map — a map missing three wallets out of two thousand beats no map — and the run only hard-fails if the wallet error rate clears 50%. Alerts fire on staleness beyond two cadences, a drop in coverage, or a spike in the error rate. Every published reading also carries its own provenance, freshness, coverage and a confidence label, so whatever reads it — a trader or an AI agent — can downgrade or skip it instead of quoting a number it can't stand behind.
Backfill. One limit, up front: the map can't be backfilled from the public API,
because clearinghouseState only returns each account's current state — there is
no “state as of last Tuesday”. So the CFI series builds forward from first deploy, and only the wallet
selection looks at history. The full backfill path — event-sourcing the open-position book
from the fills feed — is in Appendix C.
Strip out the engineering and here is the trade. Liquidation cascades feed on themselves: a forced sell pushes price lower, which trips the next forced sell. What makes a market dangerous isn't how much leverage is on — it's how close the triggers sit to the current price. The CFI measures that, so it rises before a fragile move, not after it.
How to read it:
What it is not. Not a forecast and not a timing trigger. Clusters can sit unlit for days, and the levels behave more like magnets than walls — price often drifts toward a big cluster before anything happens. Read it as a map of where the market is structurally fragile, not as a buy or sell signal. And it is a sample of the book, not the whole thing.
Pair it with, before you act: funding and OI (is the exposed side actually crowded, and paying to stay on?), spot aggressor flow / CVD (is anyone really pushing toward the cluster?), and the obvious one inside Faro — the realized Liquidation Volume, to confirm whether the fuel you're watching actually caught.
Putting it together — a worked example. Say BTC is grinding up toward a thick band of short liquidations about 2% above the mark. Fig.01 shows that as a red wall and the asymmetry sits positive. The heatmap (Fig.01b) shows the band has been building for hours, not just this minute, and the CFI has climbed into the amber regime while price went sideways. Add what Faro already shows: funding is positive and open interest is rising, so the crowd is leaning long and the shorts above are real money paying to stay short. Put together, that reads as: there is fuel above, it has been stacking, and the structure is getting fragile — a spot where a push through the level can squeeze, and where you would not want a stop sitting inside the cluster. If Faro's Liquidation Volume then spikes at that price, the fuel caught. That is the case for sitting this next to the metrics Faro already has, in one view.
How I'd put it in front of a trader (the live charts above are the working mock):
Time horizon: the map is a live snapshot; the history and heatmap read best over hours to days. Thresholds and annotations worth adding: the regime bands, the asymmetry sign, and a callout on the single nearest large cluster (“$X of short fuel at +2.1%”). Overlays: mark/oracle, funding sign, and the one I'd push hardest for — Faro's own realized Liquidation Volume on the same price axis, so a trader sees fuel and ignition in one view. Where it lives: the BTC perp / derivatives page, in a positioning-and-risk tab, and exposed to Faro's agents as a structured, confidence-tagged signal (regime, asymmetry, nearest cluster, coverage) so the AI can cite it — and hedge or refuse when coverage is thin.
Data quality is the part that has to be right, so here is the current run in full:
| Check | Value | How it is handled |
|---|---|---|
| Freshness | live · auto-checked | stale > 2 cadences → alert + banner |
| Coverage vs OI | 34.7% | reported as an explicit bound, never hidden |
| Positions / wallets | 272 / 2000 | thin sample → confidence downgraded |
| liquidationPx = null | 60 dropped | cross-margin; not placeable per position |
| Dust (< $10k) | 27 dropped | dust returns garbage liquidationPx |
| Far from the mark (> 60%) | 85 dropped | numerically unreliable |
Reconciliation. No endpoint returns the full book, so I reconcile what I
can: sampled $1.07B against OI $3.07B
— the coverage ratio above — and mark versus oracle drift, which flags a data-quality event when it
widens. What this is: a high-coverage sample of the most active wallets, not the full book.
History integrity: every CFI reading lands in the append-only ledger and is
snapshotted to git each run, and each snapshot keeps its full histogram for the heatmap — so nothing
is silently dropped, and the whole series can be rebuilt from git history via
rebuild_history.py.
Provenance & caveat. Approximate market coverage from a wallet sample; not the full order book. liquidationPx is exchange-provided. The leaderboard/activity feed used for wallet discovery is an undocumented Hyperliquid frontend endpoint — treated as a PoC stand-in for the production fills-WebSocket discovery described below.
For each qualifying position with liquidable notional N and fractional distance to its
trigger d = |liqPx − mark| / mark, the proximity weight is K(d) = exp(−d/τ)
with τ = 8%. Then CFI = 100 · Σ N·K(d) / Σ N — the notional-weighted average
proximity, bounded 0–100; the smooth kernel is what stops a single dust position on the mark from
blowing it up. Asymmetry = (short_pressure − long_pressure) / (short_pressure +
long_pressure), where side pressure is Σ N·K(d) on that side. The Fig.01 density curve is
gaussian-smoothed for legibility (area stays proportional to notional); hover reports the real
notional within ±1.25% of a level.
Two distances, two jobs. The keep filter drops any position whose trigger sits more than 60% from the mark — at that range the price is usually garbage, not real risk. The map is built over that same ±60%, but it opens zoomed to the mark ±30% (the band that matters for a near-term cascade); pan or zoom out on the chart to see the rest. Either way the kernel makes anything past roughly 20% count for almost nothing in the CFI.
On the regime bands. The cut-offs (calm < 25, elevated < 50, else fragile) are provisional. I set them by hand from the range the CFI takes in the readings so far — low in calm books, climbing as big clusters pull in toward the mark — not from a long history, because there isn't one yet: the series only accumulates forward and the map can't be backfilled from the public API. They should be recalibrated once enough history builds up, or off a proper backtest once the production backfill in Appendix C exists, rather than treated as fixed.
(1) Pull the public leaderboard feed (~38k addresses). (2) Rank by activity_score = week
volume + month volume and keep the top 2,000 — turnover is a good proxy for wallets that
are actually carrying perp risk (ranking by account value pulls in spot holders and idle vaults with
no perp exposure, which I checked early on). (3) Query their live state concurrently, within
the rate budget above. (4) Keep whichever currently hold a qualifying open BTC position (notional ≥
$10k, liquidationPx present, within 60% of mark) — a few hundred at any instant — and
aggregate them. The 2,000 is a daily, activity-ranked pick; the on-map subset is not a second
selection, just whichever of those 2,000 hold a live position this snapshot.
Liquidable open interest is concentrated in the most active wallets, so a high-activity sample captures what matters for a cascade — the large clusters near price — while the long tail is many tiny positions that barely move anything. This run reaches 35% of reported BTC open interest. (Exchange OI is one-sided, so total open-position notional is roughly twice the reported figure; even so, capturing ~half of the reported OI is a large, representative slice of what matters.) The universe is rebuilt daily and re-ranked by recent activity, so quiet wallets drop out and newly active ones come in on their own — the list won't drift into dead addresses.
clearinghouseState
per wallet. This scales to the whole market and removes the rate-limit ceiling on coverage,
approaching ~100% of OI.userFillsByTime plus an archival node or data export for the full feed)
from each wallet's first trade forward, reconstructing the open-position book — and therefore the
liquidation map and CFI — at any past timestamp. Caveat: per-wallet fill history is capped, so a
true genesis-to-now reconstruction needs the archival feed and a snapshot anchor for positions
opened before the available window.