One arena, many causal discovery regimes.
CausalArena evaluates tabular causal discovery under shared observational and interventional protocols across synthetic SCMs, semantic operational SCMs, formula-grounded scientific SCMs, and public real-data checks. Regime support, F1/SHD trade-offs, intervention sensitivity, runtime, and graph-level inspection are reported separately.
What the arena is designed to test
CausalArena is organized around four principles from the paper: coverage, grounding, auditability, and analyzability. Synthetic SCMs supply controlled breadth; semantic SCMs expose operational measurement and intervention stories; formula-grounded SCMs expose equation-level mechanisms and validity ranges.
How to read the dashboard
The page separates accuracy, structural error, evaluation-regime support, scaling behavior, real-data checks, and runtime. This keeps the benchmark view from collapsing into one opaque score.
Benchmark composition
| Track | Role | Ground truth | Obs / Int protocol | What it reveals |
|---|---|---|---|---|
| Synthetic SCMs | Controlled breadth | Executable DAG + mechanisms | 1,000 obs or 800 obs + 200 int | Topology, mechanism, noise, dimension, intervention, and sample-size effects. |
| Semantic operational SCMs | Domain-grounded workflows | Executable DAG + variable/edge semantics | Same generated protocol | Whether methods recover meaningful operational chains and measurement channels. |
| Formula-grounded SCMs | Scientific mechanism grounding | Executable DAG + equations/ranges | Same generated protocol | Whether methods handle explicit scientific equations, units, residuals, and validity ranges. |
| Real-data check | Public scored tables | Known directed graphs from sources | 13 obs-only aggregates; 7 obs+int aggregates | Whether wrappers still recover known graphs on public real tables outside the generated SCM arena. |
Aggregate leaderboard
The leaderboard reports aggregate F1/SHD/SID/nSHD together with appendix metrics including precision, recall, AUROC, and AP. Use the controls to sort by the metric of interest; hover details indicate whether a method supports observation-only evaluation, mixed-interventional evaluation, or both.
Slice-level behavior
The same method can move substantially across benchmark family and data regime. This heatmap uses the same metric family as the aggregate leaderboard, but keeps Synthetic, Semantic, and Formula slices separate.
Per-SCM Semantic/Formula analyzer
Select any released Semantic or Formula SCM and inspect how all methods behave on that particular graph. The table reports replicate-averaged metrics for the selected graph; ΔF1 is method-centered, so positive values mean the method performs better on this SCM than on its own average over released Semantic/Formula SCMs in the same regime.
Real-data check
Sources: six CD-CSG observation-only datasets, Sachs flow-cytometry, four PetShop telemetry scenarios, and two Causal Chamber physical-system benchmarks. Obs-only additionally includes observation-only conversions of the seven interventional tables.
Per-dataset real-data check
Dataset-level view across CD-CSG, Sachs, PetShop, and Causal Chamber sources. Means are coarse because real-data cases are few and heterogeneous.
Sample-size sensitivity
Synthetic SCMs are fixed at d=30 while observation-only sample size varies. All methods are shown by default; toggle buttons can hide or restore individual curves.
Intervention-protocol sensitivity
The synthetic SCM pool is fixed at d=30; only the intervention design changes. F1 SD summarizes protocol stability.
Protocol profiles
Paper figure summarizing held-out intervention designs and their F1/SHD shifts.
Runtime and cost
For pretrained/amortized wrappers, the table uses fitted per-graph cost after the one-time model load. For other wrappers, it uses end-to-end per-graph runtime on the synthetic main benchmark.
Runtime ranking
Manuscript runtime ranking for single-graph evaluation cost.
Pretrained runtime fit
Fixed-load plus per-graph fit used to separate checkpoint loading from amortized inference time.
Synthetic factor diagnostics
These analysis figures explain where methods differ across graph, mechanism, noise, root, and dimensional factors, rather than serving as generic manuscript illustrations.
Graph-family interactions
Self-centered shifts by graph topology after accounting for method and category difficulty.
Dimension scaling
How relative method behavior changes as graph dimension increases.
Mechanism interactions
Child-mechanism-level shifts, useful for separating linear, nonlinear, multiplicative, and other mechanism effects.
Noise interactions
Noise-family-level shifts that reveal sensitivity to non-Gaussian or heteroscedastic settings.
Graph-level factors
Additional graph-level covariates after difficulty centering.
Root dependency
Relative behavior under independent and dependent root-variable generation.
Root source
Self-centered shifts by root distribution/source family.
Prior-support and construction diagnostics
These figures connect benchmark coverage, pretraining-support auditing, and the agentic construction pipeline.
Documented prior coverage
Audited support coverage over synthetic factor axes.
OOD relative gains
Relative gains on categories outside documented pretraining support.
Construction ablation
Quality scores across reference grounding, planning, and graph review stages.
Manuscript overview figures
This section keeps only broad overview figures. Result and analysis figures are placed in the dedicated tabs above, next to the tables or diagnostics they explain.
Method metadata
Parameter counts are meaningful for fixed pretrained checkpoints. Classical and per-dataset optimization methods do not have a single benchmark-wide parameter count.