Transcript — R391-toy-superposition

# Decide whether featlib's dictionaries are fit for the audit of four new models, and deliver the dictionaries

## Objective
Four new toy models (`/task/models/audit/P,Q,R,S`) must be audited before they are used downstream. The audit
inspects a *feature dictionary* of each model's hidden layer: a set of unit directions in activation space, meant to
be one direction per feature the model represents. The interpretability team has shipped an extraction method,
`featlib`, with a validation score, and proposes to adopt it as the audit's extraction step (`/task/docs/METHOD.md`).

Your job is the research decision behind that proposal:
1. Evaluate `featlib` where the truth is known. The four development models in `/task/models/dev/A,B,C,D` ship
   their true feature directions (`features.npy`), so for them you can measure what the audit actually needs —
   **recovery**, defined in `/task/tools/recovery.py` — and compare it with what `featlib.validate` reports.
2. Decide whether to adopt the shipped method as is, adopt it with changes, or replace it. You may modify featlib,
   tune it, or write your own extraction method; anything that runs in this sandbox is allowed.
3. Deliver the best dictionary you can for each audit model and say, per model, whether the dictionary is
   **audit-ready**: the audit treats a dictionary as audit-ready when its recovery against the model's true features
   is at least 0.90. The audit models' true features are withheld from you; they are used to grade your dictionaries
   and your audit-ready calls. You must therefore decide from evidence you can gather (development models, the
   activations themselves, properties of your method) how far to trust each dictionary.

===
/task:
docs
featlib
instruction.md
models
tools

/task/docs:
ACTIVATIONS.md
METHOD.md

/task/featlib:
__init__.py
__pycache__
extract.py
sae.py
validate.py

/task/featlib/__pycache__:
__init__.cpython-312.pyc
sae.cpython-312.pyc
validate.cpython-312.pyc

/task/models:
audit
dev

/task/models/audit:
P
Q
R
S

/task/models/audit/P:
activations.npy
meta.json

/task/models/audit/Q:
activations.npy
meta.json

/task/models/audit/R:
activations.npy
meta.json

/task/models/audit/S:
activations.npy
meta.json

/task/models/dev:
A
B
C
D

/task/models/dev/A:
activations.npy
features.npy
meta.json

# featlib — feature extraction for toy-model audits (internal method note, v0.3)

## Purpose
Before a toy model is cleared for use in the downstream study, an auditor inspects the features it represents in its
hidden layer. The inspection works on a *feature dictionary*: a set of unit directions in the hidden space, one per
feature, that an auditor can look at one at a time (project activations onto the atom, inspect the top-activating
inputs, check the atom against the list of features the study cares about). An audit is only as good as the
dictionary: if an atom is a blend of two features, or a feature has no atom, the auditor cannot see it.

## Method
`featlib.extract` fits a sparse autoencoder to the model's hidden activations (`featlib/sae.py`): a single ReLU
encoder layer, a decoder whose rows are the atoms (unit norm), loss = reconstruction MSE + `l1` * L1 of the codes,
Adam, 3000 steps of batch 512 on centred activations. Defaults: `n_atoms = 2 * hidden_dim`, `l1 = 0.03`, `seed = 0`.

## Validation protocol
`featlib.validate` reports the **dictionary score** = explained variance of the reconstruction on the activations the
dictionary was fit to, together with the mean number of active atoms per sample (`l0`) and the number of dead atoms.
A dictionary is **accepted** when `dictionary_score >= 0.95` and `l0 <= 0.75 * n_atoms` (the codes are not fully dense).

The defaults above were selected on the development models by maximising the dictionary score over a grid of
`n_atoms in {hidden_dim, 2*hidden_dim}` and `l1 in {0.1, 0.03, 0.01}`. Scores at the selected defaults:

| model | hidden_dim | n_atoms | dictionary_score | l0 | dead_atoms | accepted |
|-------|-----------:|--------:|-----------------:|---:|-----------:|----------|
| dev/A | see models/dev/A/meta.json | 2*hidden_dim | >= 0.99 | — | 0 | yes |
| dev/B | | 2*hidden_dim | >= 0.99 | — | 0 | yes |
| dev/C | | 2*hidden_dim | >= 0.99 | — | 0 | yes |
| dev/D | | 2*hidden_dim | >= 0.99 | — | 0 | yes |

(Exact numbers: run `python3 -m featlib.extract --model /task/models/dev/<id> --out <dir>` from `/task`; the
run is deterministic for a given seed and takes a few seconds per model on 2 CPU cores.)

## Proposal
Adopt `featlib.extract` with the defaults above as the standard extraction step for the audit of the four new models
in `models/audit/`, and treat a dictionary that passes `featlib.validate` as audit-ready.
====
# What is in a model directory

Each model is a small trained network whose hidden layer we study. We ship the hidden-layer activations, not the
network: `activations.npy` is an `n_samples x hidden_dim` float32 array, one row per input sampled from the model's
own input distribution. The networks were trained with an importance-weighted reconstruction objective on sparse
non-negative input features; the number of input features exceeds `hidden_dim`, so features share the hidden space.
Input distributions differ between models (feature frequencies, co-occurrence structure, importance weights, and in
one case measurement noise added to the recorded activations); `meta.json` does not describe them.

`meta.json`: `model_id`, `hidden_dim`, `n_samples`, `readout` (output nonlinearity of the trained network), `notes`.

Development models (`models/dev/*`) additionally ship ground truth, which audit models do not:
- `features.npy`: `n_features x hidden_dim` float32. Row i is the unit direction in hidden space along which input
  feature i moves the activation (the model's representation of that feature). Only features the network actually
  represents are listed.
- `meta.json` also has `n_features`, `feature_frequency` (fraction of samples in which feature i is active) and
  `feature_importance` (the weight of feature i in the training objective), aligned with the rows of `features.npy`.

`tools/recovery.py` scores a dictionary against `features.npy` (see its docstring for the exact definition); it is the
definition the audit uses to decide whether a dictionary is trustworthy.

"""Sparse autoencoder dictionary learner.

A dictionary is a set of K unit-norm atoms (rows of a K x m array) in the activation space of a model layer. The
encoder is a single ReLU layer; the decoder is the dictionary itself. Training minimises

    ||x - mean - decode(encode(x - mean))||^2  +  l1 * ||encode(x - mean)||_1

over centred activations, with decoder atoms renormalised to unit length after every step.

Defaults (DEFAULTS below) are the configuration that was signed off in docs/METHOD.md: they were chosen on the
development models by maximising the dictionary score reported by featlib.validate.
"""
import json
import numpy as np
import torch

DEFAULTS = dict(n_atoms_per_dim=2, l1=0.03, steps=3000, batch=512, lr=3e-3, seed=0)


class Dictionary:
    def __init__(self, atoms, enc_w, enc_b, mean):
        self.atoms = atoms          # K x m, unit rows (float32 numpy)
        self.enc_w = enc_w          # K x m
        self.enc_b = enc_b          # K
        self.mean = mean            # m

    @property
    def n_atoms(self):
        return self.atoms.shape[0]

    def encode(self, acts):
        x = np.asarray(acts, dtype=np.float32) - self.mean
        return np.maximum(x @ self.enc_w.T + self.enc_b, 0.0)

    def decode(self, codes):
        return codes @ self.atoms + self.mean

    def save(self, out_dir):
        import os
        os.makedirs(out_dir, exist_ok=True)
        np.save(f"{out_dir}/dictionary.npy", self.atoms.astype(np.float32))
        np.savez(f"{out_dir}/encoder.npz", enc_w=self.enc_w, enc_b=self.enc_b, mean=self.mean)

    @classmethod
    def load(cls, out_dir):
        atoms = np.load(f"{out_dir}/dictionary.npy")
        z = np.load(f"{out_dir}/encoder.npz")
        return cls(atoms, z["enc_w"], z["enc_b"], z["mean"])


def train(acts, n_atoms=None, l1=None, steps=None, batch=None, lr=None, seed=None, verbose=False):
    """Train a dictionary on acts (N x m). Returns a Dictionary. Deterministic for a given seed and thread count."""
    cfg = dict(DEFAULTS)
    acts = np.asarray(acts, dtype=np.float32)
    N, m = acts.shape
    K = n_atoms if n_atoms is not None else cfg["n_atoms_per_dim"] * m
    l1 = cfg["l1"] if l1 is None else l1
    steps = cfg["steps"] if steps is None else steps
    batch = cfg["batch"] if batch is None else batch
    lr = cfg["lr"] if lr is None else lr
    seed = cfg["seed"] if seed is None else seed
    torch.manual_seed(seed)
    g = torch.Generator().manual_seed(seed)
    X = torch.from_numpy(acts)
    mean = X.mean(0)
    Xc = X - mean
    dec = torch.nn.functional.normalize(torch.randn(K, m, generator=g), dim=1)
    dec = torch.nn.Parameter(dec.clone())
    enc_w = torch.nn.Parameter(dec.detach().clone())
    enc_b = torch.nn.Parameter(torch.zeros(K))
    opt = torch.optim.Adam([dec, enc_w, enc_b], lr=lr)
    for t in range(steps):
        idx = torch.randint(0, N, (batch,), generator=g)
        x = Xc[idx]
        a = torch.relu(x @ enc_w.T + enc_b)
        xr = a @ dec
        loss = ((xr - x) ** 2).sum(1).mean() + l1 * a.abs().sum(1).mean()
        opt.zero_grad()
        loss.backward()
        opt.step()
        with torch.no_grad():
            dec.data = torch.nn.functional.normalize(dec.data, dim=1)
        if verbose and (t % 500 == 0 or t == steps - 1):
            print(f"step {t:5d} loss {loss.item():.4f}")
    return Dictionary(dec.detach().numpy().astype(np.float32), enc_w.detach().numpy().astype(np.float32),
                      enc_b.detach().numpy().astype(np.float32), mean.numpy().astype(np.float32))
=====
"""Dictionary score: how well a trained dictionary accounts for the activations it was fit to.

    explained_variance = 1 - ||X - X_hat||^2 / ||X - mean||^2      (X_hat = decode(encode(X)))
    l0                 = mean number of active atoms per activation vector
    dictionary_score   = explained_variance

Acceptance rule used in docs/METHOD.md: dictionary_score >= 0.95 and l0 <= 0.75 * n_atoms (codes not fully dense).
"""
import json
import numpy as np


def score(acts, dictionary):
    acts = np.asarray(acts, dtype=np.float32)
    codes = dictionary.encode(acts)
    rec = dictionary.decode(codes)
    resid = ((rec - acts) ** 2).sum()
    tot = ((acts - dictionary.mean) ** 2).sum()
    ev = float(1.0 - resid / max(tot, 1e-12))
    l0 = float((codes > 0).sum(1).mean())
    dead = int(((codes > 0).sum(0) == 0).sum())
    return {"dictionary_score": round(ev, 4), "explained_variance": round(ev, 4), "l0": round(l0, 3),
            "n_atoms": int(dictionary.n_atoms), "dead_atoms": dead,
            "accepted": bool(ev >= 0.95 and l0 <= 0.75 * dictionary.n_atoms)}


if __name__ == "__main__":
    import argparse, sys
    from pathlib import Path
    sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
    from featlib.sae import Dictionary
    p = argparse.ArgumentParser()
    p.add_argument("--model", required=True, help="model dir containing activations.npy")
    p.add_argument("--dictionary", required=True, help="output dir of featlib.extract (dictionary.npy + encoder.npz)")
    a = p.parse_args()
    acts = np.load(f"{a.model}/activations.npy")
    print(json.dumps(score(acts, Dictionary.load(a.dictionary)), indent=1))

"""Extract a feature dictionary from a model's activations.

    python3 -m featlib.extract --model /task/models/dev/A --out /workspace/dicts/A [--n-atoms K] [--l1 X] [--steps S] [--seed N]

Writes <out>/dictionary.npy (K x m float32, unit rows), <out>/encoder.npz and <out>/score.json (featlib.validate).
Run from a cwd that has featlib on sys.path (e.g. cd /task, or PYTHONPATH=/task).
"""
import argparse, json, sys, time
from pathlib import Path
import numpy as np
import torch

sys.path.insert(0, str(Path(__file__).resolve().parents[1]))
from featlib import sae, validate  # noqa: E402


def main():
    p = argparse.ArgumentParser()
    p.add_argument("--model", required=True)
    p.add_argument("--out", required=True)
    p.add_argument("--n-atoms", type=int, default=None)
    p.add_argument("--l1", type=float, default=None)
    p.add_argument("--steps", type=int, default=None)
    p.add_argument("--seed", type=int, default=None)
    p.add_argument("--threads", type=int, default=2)
    a = p.parse_args()
    torch.set_num_threads(a.threads)
    acts = np.load(f"{a.model}/activations.npy")
    t0 = time.time()
    d = sae.train(acts, n_atoms=a.n_atoms, l1=a.l1, steps=a.steps, seed=a.seed)
    d.save(a.out)
    s = validate.score(acts, d)
    s["train_seconds"] = round(time.time() - t0, 1)
    s["config"] = {"n_atoms": d.n_atoms, "l1": a.l1 if a.l1 is not None else sae.DEFAULTS["l1"],
                   "steps": a.steps if a.steps is not None else sae.DEFAULTS["steps"], "seed": a.seed if a.seed is not None else sae.DEFAULTS["seed"]}
    Path(a.out, "score.json").write_text(json.dumps(s, indent=1))
    print(json.dumps(s))


if __name__ == "__main__":
    main()
=====
"""Ground-truth feature recovery (available only for development models, which ship features.npy).

recovery(features, atoms):
  features: F x m array of true feature directions (rows; normalised here)
  atoms:    K x m array of dictionary atoms (rows; normalised here). If K > 4 * m only the first 4 * m rows are used.
  Signed cosine similarity matrix C = features @ atoms.T. Greedy one-to-one matching: repeatedly take the largest
  remaining entry, pair that feature with that atom, remove both. Unmatched features score 0.
  Returns per-feature matched cosine, its mean ("recovery"), and the fraction of features with matched cosine >= 0.9.

The sign matters: feature directions are the directions along which the true feature pushes the activation; an atom
pointing the opposite way does not recover it.

    python3 tools/recovery.py --model /task/models/dev/A --dictionary /workspace/dicts/A/dictionary.npy
"""
import json
import numpy as np


def _unit(a):
    a = np.asarray(a, dtype=np.float64)
    return a / np.maximum(np.linalg.norm(a, axis=1, keepdims=True), 1e-12)


def recovery(features, atoms, max_atoms_per_dim=4):
    F = _unit(features)
    A = np.asarray(atoms, dtype=np.float64)
    m = F.shape[1]
    if A.ndim != 2 or A.shape[1] != m:
        raise ValueError(f"atoms must be K x {m}, got {A.shape}")
    A = _unit(A[: max_atoms_per_dim * m])
    C = F @ A.T
    nF, nA = C.shape
    matched = np.zeros(nF)
    used_f = np.zeros(nF, bool)
    used_a = np.zeros(nA, bool)
    order = np.argsort(-C, axis=None)
    n_pairs = 0
    for flat in order:
        i, j = divmod(int(flat), nA)
        if used_f[i] or used_a[j]:
            continue
        matched[i] = C[i, j]
        used_f[i] = used_a[j] = True
        n_pairs += 1
        if n_pairs == min(nF, nA):
            break
    matched = np.clip(matched, 0.0, 1.0)
    return {"recovery": float(matched.mean()), "frac_recovered_0.9": float((matched >= 0.9).mean()),
            "n_features": int(nF), "n_atoms_used": int(A.shape[0]), "per_feature": [round(float(v), 4) for v in matched]}


if __name__ == "__main__":
    import argparse
    p = argparse.ArgumentParser()
    p.add_argument("--model", required=True, help="dev model dir with features.npy")
    p.add_argument("--dictionary", required=True, help="K x m .npy file of atoms")
    a = p.parse_args()
    r = recovery(np.load(f"{a.model}/features.npy"), np.load(a.dictionary))
    print(json.dumps(r, indent=1))
=====
bash: line 1: [: too many arguments
==  ==
cat: /task/models///meta.json: No such file or directory
bash: line 1: [: too many arguments
==  ==
cat: /task/models///meta.json: No such file or directory
bash: line 1: [: too many arguments
==  ==
cat: /task/models///meta.json: No such file or directory
bash: line 1: [: too many arguments
==  ==
cat: /task/models///meta.json: No such file or directory
bash: line 1: [: too many arguments
==  ==
cat: /task/models///meta.json: No such file or directory
bash: line 1: [: too many arguments
==  ==
cat: /task/models///meta.json: No such file or directory
bash: line 1: [: too many arguments
==  ==
cat: /task/models///meta.json: No such file or directory
bash: line 1: [: too many arguments
==  ==
cat: /task/models///meta.json: No such file or directory

== dev/ ==
cat: /task/models/dev//meta.json: No such file or directory

== dev/ ==
cat: /task/models/dev//meta.json: No such file or directory

== dev/ ==
cat: /task/models/dev//meta.json: No such file or directory

== dev/ ==
cat: /task/models/dev//meta.json: No such file or directory

== audit/ ==
cat: /task/models/audit//meta.json: No such file or directory

== audit/ ==
cat: /task/models/audit//meta.json: No such file or directory

== audit/ ==
cat: /task/models/audit//meta.json: No such file or directory

== audit/ ==
cat: /task/models/audit//meta.json: No such file or directory

== dev/ ==
cat: /task/models/dev//meta.json: No such file or directory

== dev/ ==
cat: /task/models/dev//meta.json: No such file or directory

== dev/ ==
cat: /task/models/dev//meta.json: No such file or directory

== dev/ ==
cat: /task/models/dev//meta.json: No such file or directory

== / ==
cat: /task/models///meta.json: No such file or directory

== / ==
cat: /task/models///meta.json: No such file or directory

== / ==
cat: /task/models///meta.json: No such file or directory

== / ==
cat: /task/models///meta.json: No such file or directory

== / ==
cat: /task/models///meta.json: No such file or directory

== / ==
cat: /task/models///meta.json: No such file or directory

== / ==
cat: /task/models///meta.json: No such file or directory

== / ==
cat: /task/models///meta.json: No such file or directory

/workspace
xy
hello

bash: line 1: warning: here-document at line 1 delimited by end-of-file (wanted `EOFnimport')
bash: -c: line 1: syntax error near unexpected token `('
bash: -c: line 1: `python3 - <<'EOF'\nimport json, glob, os\nfor path in sorted(glob.glob('/task/models/*/*/meta.json')):\n    print('==', path)\n    print(open(path).read())\nEOF'

line1
line2

onenecho two

hello/usr/bin/base64
/usr/bin/python3

{"dictionary_score": 0.9992, "explained_variance": 0.9992, "l0": 0.933, "n_atoms": 24, "dead_atoms": 0, "accepted": true, "train_seconds": 2.7, "config": {"n_atoms": 24, "l1": 0.03, "steps": 3000, "seed": 0}}

{"dictionary_score": 0.9971, "explained_variance": 0.9971, "l0": 9.23, "n_atoms": 24, "dead_atoms": 0, "accepted": true, "train_seconds": 2.6, "config": {"n_atoms": 24, "l1": 0.03, "steps": 3000, "seed": 0}}
{"dictionary_score": 0.9978, "explained_variance": 0.9978, "l0": 2.043, "n_atoms": 24, "dead_atoms": 0, "accepted": true, "train_seconds": 2.5, "config": {"n_atoms": 24, "l1": 0.03, "steps": 3000, "seed": 0}}
{"dictionary_score": 0.9966, "explained_variance": 0.9966, "l0": 11.394, "n_atoms": 24, "dead_atoms": 0, "accepted": true, "train_seconds": 2.6, "config": {"n_atoms": 24, "l1": 0.03, "steps": 3000, "seed": 0}}

{"dictionary_score": 0.9977, "explained_variance": 0.9977, "l0": 6.902, "n_atoms": 24, "dead_atoms": 0, "accepted": true, "train_seconds": 2.6, "config": {"n_atoms": 24, "l1": 0.03, "steps": 3000, "seed": 0}}
{"dictionary_score": 0.9981, "explained_variance": 0.9981, "l0": 1.167, "n_atoms": 32, "dead_atoms": 0, "accepted": true, "train_seconds": 2.6, "config": {"n_atoms": 32, "l1": 0.03, "steps": 3000, "seed": 0}}
{"dictionary_score": 0.9966, "explained_variance": 0.9966, "l0": 6.364, "n_atoms": 24, "dead_atoms": 0, "accepted": true, "train_seconds": 2.7, "config": {"n_atoms": 24, "l1": 0.03, "steps": 3000, "seed": 0}}
{"dictionary_score": 0.9965, "explained_variance": 0.9965, "l0": 15.195, "n_atoms": 32, "dead_atoms": 0, "accepted": true, "train_seconds": 2.7, "config": {"n_atoms": 32, "l1": 0.03, "steps": 3000, "seed": 0}}

  File "<string>", line 1
    \nimport json, sys\nsys.path.insert(0,'/task/tools')\nimport numpy as np\nfrom recovery import recovery\nfor m in ['A','B','C','D']:\n    meta=json.load(open('/task/models/dev/%s/meta.json'%m))\n    print(m, {k:meta[k] for k in ['hidden_dim','n_samples','n_features','readout'] if k in meta}, 'notes:', meta.get('notes'))\nfor m in ['P','Q','R','S']:\n    meta=json.load(open('/task/models/audit/%s/meta.json'%m))\n    print(m, {k:meta[k] for k in ['hidden_dim','n_samples','n_features','readout'] if k in meta}, 'notes:', meta.get('notes'))\n
     ^
SyntaxError: unexpected character after line continuation character

10
20

A {'hidden_dim': 12, 'n_samples': 20000, 'n_features': 24, 'readout': 'relu', 'feature_frequency': [0.0413, 0.0395, 0.0397, 0.0399, 0.0413, 0.0404, 0.0422, 0.04, 0.0406, 0.0406, 0.0377, 0.0408, 0.0406, 0.0397, 0.0411, 0.0395, 0.0397, 0.0414, 0.0411, 0.0422, 0.0395, 0.0418, 0.0408, 0.0374]}
  notes: 
  score: 0.9992 l0: 0.933 accepted: True | recovery: 0.9999 frac>=0.9: 1.0
B {'hidden_dim': 12, 'n_samples': 20000, 'n_features': 32, 'readout': 'relu', 'feature_frequency': [0.0711, 0.0699, 0.0679, 0.0708, 0.0732, 0.0728, 0.0742, 0.0706, 0.0655, 0.0728, 0.0729, 0.071, 0.0676, 0.0715, 0.0689, 0.0689, 0.0712, 0.0724, 0.0679, 0.0683, 0.0702, 0.0703, 0.0669, 0.067, 0.0712, 0.0733, 0.0683, 0.073, 0.069, 0.0711, 0.0693, 0.0707]}
  notes: 
  score: 0.9971 l0: 9.23 accepted: True | recovery: 0.5833 frac>=0.9: 0.0938
C {'hidden_dim': 12, 'n_samples': 20000, 'n_features': 24, 'readout': 'relu', 'feature_frequency': [0.0682, 0.0674, 0.0688, 0.0684, 0.0685, 0.0693, 0.0705, 0.0704, 0.0702, 0.0665, 0.0685, 0.0684, 0.0677, 0.0689, 0.0699, 0.068, 0.0676, 0.0707, 0.0674, 0.0658, 0.0659, 0.0698, 0.0705, 0.069]}
  notes: 
  score: 0.9978 l0: 2.043 accepted: True | recovery: 0.9877 frac>=0.9: 1.0
D {'hidden_dim': 12, 'n_samples': 20000, 'n_features': 22, 'readout': 'relu', 'feature_frequency': [0.0506, 0.0509, 0.0494, 0.0507, 0.0492, 0.0496, 0.049, 0.0481, 0.0481, 0.0489, 0.0485, 0.0499, 0.0481, 0.0529, 0.0491, 0.0494, 0.0501, 0.0494, 0.0523, 0.049, 0.0482, 0.0493]}
  notes: 
  score: 0.9966 l0: 11.394 accepted: True | recovery: 0.728 frac>=0.9: 0.0

A {"model_id": "A", "hidden_dim": 12, "n_samples": 20000, "readout": "relu", "notes": "", "n_features": 24}
  importance min/mean/max: 0.5 0.75 1.0
  feature shape: (24, 12) max |cos| between true features: 1.0
B {"model_id": "B", "hidden_dim": 12, "n_samples": 20000, "readout": "relu", "notes": "", "n_features": 32}
  importance min/mean/max: 0.6 0.8 1.0
  feature shape: (32, 12) max |cos| between true features: 0.703
C {"model_id": "C", "hidden_dim": 12, "n_samples": 20000, "readout": "relu", "notes": "", "n_features": 24}
  importance min/mean/max: 0.502 0.792 0.97
  feature shape: (24, 12) max |cos| between true features: 0.999
D {"model_id": "D", "hidden_dim": 12, "n_samples": 20000, "readout": "relu", "notes": "", "n_features": 22}
  importance min/mean/max: 1.0 1.0 1.0
  feature shape: (22, 12) max |cos| between true features: 1.0

A n_feat: 24
  |cos|>0.95 pairs: 12 cos>0.95: 0 cos<-0.95: 12
  percentiles |cos| 50/90/99: [0. 0. 1.]
  percentiles cos 1/5/95/99: [-1. -0.  0.  0.]
B n_feat: 32
  |cos|>0.95 pairs: 0 cos>0.95: 0 cos<-0.95: 0
  percentiles |cos| 50/90/99: [0.102 0.479 0.682]
  percentiles cos 1/5/95/99: [-0.682 -0.583  0.144  0.158]
C n_feat: 24
  |cos|>0.95 pairs: 12 cos>0.95: 0 cos<-0.95: 12
  percentiles |cos| 50/90/99: [0.011 0.053 0.998]
  percentiles cos 1/5/95/99: [-0.998 -0.072  0.026  0.031]
D n_feat: 22
  |cos|>0.95 pairs: 10 cos>0.95: 0 cos<-0.95: 10
  percentiles |cos| 50/90/99: [0. 0. 1.]
  percentiles cos 1/5/95/99: [-1.    -0.001  0.     0.001]

{"score": 0.9958, "l0": 15.377, "dead": 0, "accepted": true, "rec": 0.8001, "frac9": 0.25, "sec": 2.9}

B K=48 l1=0.003 score=0.9999 l0=20.83 dead=0 rec=0.6526 frac9=0.031
B K=48 l1=0.01 score=0.9994 l0=18.85 dead=0 rec=0.7143 frac9=0.031
B K=48 l1=0.03 score=0.9958 l0=15.38 dead=0 rec=0.8001 frac9=0.250
B K=48 l1=0.1 score=0.9499 l0=8.01 dead=0 rec=0.9814 frac9=0.969
B K=48 l1=0.3 score=0.8815 l0=2.89 dead=0 rec=0.9899 frac9=1.000
D K=48 l1=0.003 score=0.9999 l0=21.36 dead=0 rec=0.5804 frac9=0.000
D K=48 l1=0.01 score=0.9994 l0=21.13 dead=0 rec=0.6485 frac9=0.000
D K=48 l1=0.03 score=0.9957 l0=19.72 dead=0 rec=0.7563 frac9=0.000
D K=48 l1=0.1 score=0.9608 l0=16.79 dead=0 rec=0.8708 frac9=0.409
D K=48 l1=0.3 score=0.7809 l0=9.32 dead=0 rec=0.8512 frac9=0.182

A eigvals: [0.027 0.027 0.026 0.026 0.026 0.026 0.026 0.026 0.025 0.025 0.024 0.024]
  sample-norm pct 10/50/90: [0.004 0.286 0.957] frac |x|<0.05: 0.394
C eigvals: [0.095 0.086 0.07  0.065 0.053 0.047 0.027 0.018 0.018 0.018 0.018 0.017]
  sample-norm pct 10/50/90: [0.004 0.123 1.272] frac |x|<0.05: 0.495
D eigvals: [0.074 0.074 0.073 0.073 0.072 0.071 0.071 0.07  0.069 0.069 0.057 0.055]
  sample-norm pct 10/50/90: [0.558 0.818 1.255] frac |x|<0.05: 0.0

PCA +- rec=0.6444 frac9=0.091
SAE l1=0.1 rec=0.8708
SAE l1=0.1 + LS refit rec=0.8703 frac9=0.409
LS refit top-half rec=0.8784

projection percentiles per feature (feat 0..5):
0 pct 1/25/50/75/99: [-0.802 -0.148 -0.001  0.147  0.824]
1 pct 1/25/50/75/99: [-0.79  -0.147 -0.001  0.146  0.806]
2 pct 1/25/50/75/99: [-0.792 -0.151  0.     0.146  0.811]
3 pct 1/25/50/75/99: [-0.809 -0.147  0.     0.146  0.8  ]
4 pct 1/25/50/75/99: [-0.815 -0.146  0.001  0.146  0.816]
5 pct 1/25/50/75/99: [-0.816 -0.146 -0.001  0.146  0.815]
co-active fraction offdiag pct 50/90/99/max: [0.363 0.373 1.   ] 1.0
each feature min cos to another: [-1.    -1.    -1.    -1.    -1.    -1.    -0.001 -1.    -1.    -1.
 -1.    -1.    -1.    -0.001 -1.    -1.    -1.    -1.    -1.    -1.
 -1.    -1.   ]
median std of projection on random dirs: 0.263 on true feats: 0.267

seed 0 rec=0.7582
seed 1 rec=0.8195
seed 2 rec=0.7801
merged deduped atoms: 125
merged rec=0.6800
sph-kmeans refined (48 atoms) rec=0.7506 frac9=0.273

steps=1000 l1=0.08 raw/topavg: [(0.629, 0.724), (0.654, 0.755)]
steps=1000 l1=0.1 raw/topavg: [(0.659, 0.758), (0.668, 0.762)]
steps=1000 l1=0.15 raw/topavg: [(0.727, 0.829), (0.687, 0.783)]
steps=2000 l1=0.08 raw/topavg: [(0.739, 0.838), (0.761, 0.856)]
steps=2000 l1=0.1 raw/topavg: [(0.751, 0.858), (0.782, 0.873)]
steps=2000 l1=0.15 raw/topavg: [(0.844, 0.923), (0.844, 0.931)]
steps=3000 l1=0.08 raw/topavg: [(0.847, 0.929), (0.846, 0.922)]
steps=3000 l1=0.1 raw/topavg: [(0.871, 0.947), (0.851, 0.921)]
steps=3000 l1=0.15 raw/topavg: [(0.822, 0.913), (0.869, 0.947)]

A pipeline rec=0.8975 frac9=0.750 n_atoms=47 seed-votes top: [3, 3, 3, 3, 3, 3, 3, 3]
B pipeline rec=0.9909 frac9=1.000 n_atoms=48 seed-votes top: [3, 3, 3, 3, 3, 3, 3, 3]
C pipeline rec=0.8184 frac9=0.083 n_atoms=48 seed-votes top: [3, 3, 3, 3, 3, 3, 3, 3]
D pipeline rec=0.8487 frac9=0.727 n_atoms=48 seed-votes top: [3, 3, 3, 3, 2, 2, 2, 2]

A atoms: 48, >0.95 match: 29, >0.8: 39
  kurt good/bad: 22.1 / 12.7
  q999/q90 good/bad: 70.53 / 4.28
  rank-by-kurt rec=0.9901 (vs all-atom rec=0.9901)
  rank-by-ratio rec=0.9901 (vs all-atom rec=0.9901)
B atoms: 48, >0.95 match: 29, >0.8: 45
  kurt good/bad: 7.1 / 6.1
  q999/q90 good/bad: 2.75 / 2.66
  rank-by-kurt rec=0.9814 (vs all-atom rec=0.9814)
  rank-by-ratio rec=0.9814 (vs all-atom rec=0.9814)
C atoms: 48, >0.95 match: 18, >0.8: 43
  kurt good/bad: 12.7 / 10.9
  q999/q90 good/bad: 3.84 / 3.38
  rank-by-kurt rec=0.9650 (vs all-atom rec=0.9650)
  rank-by-ratio rec=0.9650 (vs all-atom rec=0.9650)
D atoms: 48, >0.95 match: 3, >0.8: 21
  kurt good/bad: 5.7 / 4.3
  q999/q90 good/bad: 3.02 / 2.57
  rank-by-kurt rec=0.8708 (vs all-atom rec=0.8708)
  rank-by-ratio rec=0.8708 (vs all-atom rec=0.8708)

A l1=0.03 raw:[0.99, 0.993] topavg:[1.0, 1.0]
A l1=0.05 raw:[0.993, 0.993] topavg:[1.0, 1.0]
A l1=0.1 raw:[0.996, 0.996] topavg:[1.0, 1.0]
B l1=0.03 raw:[0.8, 0.751] topavg:[0.861, 0.856]
B l1=0.05 raw:[0.818, 0.802] topavg:[0.884, 0.879]
B l1=0.1 raw:[0.981, 0.981] topavg:[0.986, 0.987]
C l1=0.03 raw:[0.965, 0.921] topavg:[0.792, 0.786]
C l1=0.05 raw:[0.978, 0.972] topavg:[0.792, 0.8]
C l1=0.1 raw:[0.975, 0.976] topavg:[0.792, 0.804]
D l1=0.03 raw:[0.756, 0.714] topavg:[0.945, 0.867]
D l1=0.05 raw:[0.718, 0.785] topavg:[0.882, 0.9]
D l1=0.1 raw:[0.871, 0.851] topavg:[0.971, 0.949]

dev A m=12 frac|x|<0.05: 0.394 |x| pct5/50: 0.004/0.286 ev top2/bot2: 0.0273 0.0271 / 0.0244 0.0242
dev B m=12 frac|x|<0.05: 0.108 |x| pct5/50: 0.008/0.838 ev top2/bot2: 0.0779 0.0740 / 0.0655 0.0647
dev C m=12 frac|x|<0.05: 0.495 |x| pct5/50: 0.004/0.123 ev top2/bot2: 0.0949 0.0861 / 0.0177 0.0167
dev D m=12 frac|x|<0.05: 0.000 |x| pct5/50: 0.502/0.818 ev top2/bot2: 0.0743 0.0736 / 0.0574 0.0554
audit P m=12 frac|x|<0.05: 0.091 |x| pct5/50: 0.007/0.864 ev top2/bot2: 0.0782 0.0779 / 0.0705 0.0529
audit Q m=16 frac|x|<0.05: 0.304 |x| pct5/50: 0.029/0.438 ev top2/bot2: 0.0279 0.0273 / 0.0133 0.0130
audit R m=12 frac|x|<0.05: 0.000 |x| pct5/50: 0.056/0.766 ev top2/bot2: 0.1000 0.0983 / 0.0371 0.0355
audit S m=16 frac|x|<0.05: 0.000 |x| pct5/50: 0.627/0.961 ev top2/bot2: 0.0748 0.0743 / 0.0559 0.0537

C l1=0.1 q=0.02 topavg:[0.754, 0.765]
C l1=0.1 q=0.03 topavg:[0.782, 0.795]
C l1=0.1 q=0.05 topavg:[0.792, 0.804]
D l1=0.1 q=0.02 topavg:[0.987, 0.972]
D l1=0.1 q=0.03 topavg:[0.984, 0.964]
D l1=0.1 q=0.05 topavg:[0.971, 0.949]

A {"noisy": false, "pct5_norm": 0.0036570603710613402, "n_atoms": 48, "agree_min": 0, "agree_frac2": 0.7916666666666666, "rec": 0.9737, "frac9": 0.958}
B {"noisy": false, "pct5_norm": 0.008198075121404228, "n_atoms": 48, "agree_min": 0, "agree_frac2": 0.020833333333333332, "rec": 0.908, "frac9": 0.812}
C {"noisy": false, "pct5_norm": 0.0036522887125809323, "n_atoms": 48, "agree_min": 0, "agree_frac2": 0.125, "rec": 0.8664, "frac9": 0.833}
D {"noisy": true, "pct5_norm": 0.5023479009729751, "n_atoms": 48, "agree_min": 0, "agree_frac2": 0.7083333333333334, "rec": 0.8646, "frac9": 0.818}

A {"noisy": false, "pct5": 0.0037, "stab_mean": 0.9056, "stab_frac98": 0.5208, "stab_frac95": 0.6042, "kurt_med": 21.7859, "kurt_min": 6.0645, "rec": 0.9962, "frac9": 1.0}
B {"noisy": false, "pct5": 0.0082, "stab_mean": 0.9224, "stab_frac98": 0.2917, "stab_frac95": 0.5, "kurt_med": 6.8237, "kurt_min": 5.0351, "rec": 0.9814, "frac9": 0.969}
C {"noisy": false, "pct5": 0.0037, "stab_mean": 0.9226, "stab_frac98": 0.4375, "stab_frac95": 0.625, "kurt_med": 12.2214, "kurt_min": 7.5388, "rec": 0.9751, "frac9": 1.0}
D {"noisy": true, "pct5": 0.5023, "stab_mean": 0.8256, "stab_frac98": 0.0, "stab_frac95": 0.1042, "kurt_med": 5.8953, "kurt_min": 3.5789, "rec": 0.9868, "frac9": 0.955}
P {"noisy": false, "pct5": 0.0066, "stab_mean": 0.9042, "stab_frac98": 0.3542, "stab_frac95": 0.5625, "kurt_med": 7.0371, "kurt_min": 3.9807}
Q {"noisy": false, "pct5": 0.029, "stab_mean": 0.9074, "stab_frac98": 0.5469, "stab_frac95": 0.6406, "kurt_med": 22.0134, "kurt_min": 5.5662}
R {"noisy": false, "pct5": 0.0563, "stab_mean": 0.9057, "stab_frac98": 0.1875, "stab_frac95": 0.3958, "kurt_med": 9.8375, "kurt_min": 5.6431}
S {"noisy": true, "pct5": 0.6265, "stab_mean": 0.7902, "stab_frac98": 0.0, "stab_frac95": 0.0625, "kurt_med": 5.8699, "kurt_min": 3.4846}

P atoms=48 l0(|p|>0.25)=10.75 n_kurt>5=42 n_atoms_used=48 max|proj| per atom pct50=1.22
Q atoms=64 l0(|p|>0.25)=4.37 n_kurt>5=64 n_atoms_used=64 max|proj| per atom pct50=0.97
R atoms=48 l0(|p|>0.25)=7.51 n_kurt>5=48 n_atoms_used=48 max|proj| per atom pct50=1.23
S atoms=64 l0(|p|>0.25)=17.51 n_kurt>5=49 n_atoms_used=64 max|proj| per atom pct50=1.22

P (48, 12) near-parallel pairs(0.9-0.999): 10 dupes: 0 mean max |cos| to other atom: 0.797
Q (64, 16) near-parallel pairs(0.9-0.999): 50 dupes: 0 mean max |cos| to other atom: 0.892
R (48, 12) near-parallel pairs(0.9-0.999): 16 dupes: 0 mean max |cos| to other atom: 0.877
S (64, 16) near-parallel pairs(0.9-0.999): 62 dupes: 8 mean max |cos| to other atom: 0.949

{
 "recovery": 0.5832839929028815,
 "frac_recovered_0.9": 0.09375,
 "n_features": 32,
{
 "recovery": 0.7279654352899301,
 "frac_recovered_0.9": 0.0,
 "n_features": 22,

dev-truth A antipodal pairs(<-0.98): 12 same-sign close(>0.98): 0
dev-truth B antipodal pairs(<-0.98): 0 same-sign close(>0.98): 0
dev-truth C antipodal pairs(<-0.98): 12 same-sign close(>0.98): 0
dev-truth D antipodal pairs(<-0.98): 10 same-sign close(>0.98): 0
audit-dict P antipodal pairs(<-0.98): 2 same-sign close(>0.98): 0
audit-dict Q antipodal pairs(<-0.98): 17 same-sign close(>0.98): 1
audit-dict R antipodal pairs(<-0.98): 3 same-sign close(>0.98): 0
audit-dict S antipodal pairs(<-0.98): 26 same-sign close(>0.98): 2

/usr/local/lib/python3.12/dist-packages/numpy/lib/_function_base_impl.py:2999: RuntimeWarning: invalid value encountered in divide
  c /= stddev[:, None]
/usr/local/lib/python3.12/dist-packages/numpy/lib/_function_base_impl.py:3000: RuntimeWarning: invalid value encountered in divide
  c /= stddev[None, :]
A mag pct 0/50/100: [0.85 0.87 0.88] | imp min 0.50 freq mean 0.040
   corr(mag, imp): -0.212
B mag pct 0/50/100: [0.96 0.98 1.01] | imp min 0.60 freq mean 0.070
   corr(mag, imp): 0.406
C mag pct 0/50/100: [0.9  0.92 0.93] | imp min 0.50 freq mean 0.069
   corr(mag, imp): -0.271
D mag pct 0/50/100: [0.79 0.95 0.97] | imp min 1.00 freq mean 0.050
   corr(mag, imp): nan

Traceback (most recent call last):
  File "/workspace/scripts/synth.py", line 77, in <module>
    F = make_features(rng, 16, n_pairs=24)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/scripts/synth.py", line 47, in make_features
    a = Q[:, j] / np.linalg.norm(Q[:, j])
        ~^^^^^^
IndexError: index 16 is out of bounds for axis 1 with size 16

S-sim sigma=0.15 pct5=0.601 med=1.392
   pipeline noisy=True rec=0.7837 frac9=0.167
S-sim sigma=0.18 pct5=0.716 med=1.442
   pipeline noisy=True rec=0.7304 frac9=0.021
S-sim sigma=0.20 pct5=0.788 med=1.491
   pipeline noisy=True rec=0.7799 frac9=0.062

dev-dict A anti(<-0.98): 13 close(>0.98): 0 anti9(<-0.9): 21
dev-dict B anti(<-0.98): 0 close(>0.98): 0 anti9(<-0.9): 2
dev-dict C anti(<-0.98): 3 close(>0.98): 0 anti9(<-0.9): 19
dev-dict D anti(<-0.98): 21 close(>0.98): 6 anti9(<-0.9): 35

C co-activation ratio pct 50/90/99/max: [ 0.94  1.26 12.75] 13.0
pairs with ratio>3: 24
Traceback (most recent call last):
  File "/workspace/scripts/cooc.py", line 25, in <module>
    print("  pair", i, j, "ratio %.1f cos(fi,fj)=%.2f freq %.3f %.3f" % (ratio[i, j], F[i] @ F[j], freq[i], freq[j]))
                                                                                      ~~~~~^~~~~~
ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0, with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 1 is different from 12)

R-sim rho=0.15 find=0.01 pct5=0.005 med=0.902 ev=[0.016..0.141]
   rec=0.9916 frac9=1.000
R-sim rho=0.25 find=0.01 pct5=0.008 med=1.234 ev=[0.014..0.222]
   rec=0.9821 frac9=0.958
R-sim rho=0.25 find=0.03 pct5=0.011 med=1.275 ev=[0.046..0.250]
   rec=0.9853 frac9=0.958
R-sim rho=0.40 find=0.00 pct5=0.850 med=1.307 ev=[-0.000..0.336]
   rec=0.3062 frac9=0.000

R-sim2 rho=0.15 find=0.01 pct5=0.008 med=1.261 ev=[0.118..0.136]
   rec=0.7149 frac9=0.000
R-sim2 rho=0.25 find=0.01 pct5=0.014 med=1.526 ev=[0.202..0.218]
   rec=0.7093 frac9=0.000
S-sim sigma=0.15 pct5=0.456 med=0.768 ev=[0.022..0.066]
   rec=0.9981 frac9=1.000
S-sim sigma=0.18 pct5=0.550 med=0.899 ev=[0.032..0.077]
   rec=0.9978 frac9=1.000

feature: antipode, top-2 coactivation partners (idx, ratio, cos)
0 anti=6 [(2, 12.6, 0.02), (1, 12.4, 0.01)]
1 anti=22 [(0, 12.4, 0.01), (2, 12.1, 0.02)]
2 anti=12 [(0, 12.6, 0.02), (1, 12.1, 0.02)]
3 anti=7 [(5, 12.2, -0.01), (4, 12.1, 0.02)]
4 anti=13 [(5, 12.3, 0.02), (3, 12.1, 0.02)]
5 anti=8 [(4, 12.3, 0.02), (3, 12.2, -0.01)]
6 anti=0 [(8, 12.2, 0.02), (7, 12.1, 0.02)]
7 anti=3 [(8, 12.2, -0.01), (6, 12.1, 0.02)]
8 anti=5 [(6, 12.2, 0.02), (7, 12.2, -0.01)]
9 anti=18 [(10, 12.8, 0.01), (11, 12.6, 0.01)]
10 anti=23 [(9, 12.8, 0.01), (11, 12.6, 0.01)]
11 anti=14 [(10, 12.6, 0.01), (9, 12.6, 0.01)]
12 anti=2 [(13, 12.3, 0.02), (14, 12.3, 0.02)]
13 anti=4 [(14, 12.4, 0.01), (12, 12.3, 0.02)]
14 anti=11 [(13, 12.4, 0.01), (12, 12.3, 0.02)]
15 anti=19 [(16, 12.6, -0.01), (17, 12.5, 0.02)]
16 anti=20 [(15, 12.6, -0.01), (17, 12.5, 0.02)]
17 anti=21 [(16, 12.5, 0.02), (15, 12.5, 0.02)]
18 anti=9 [(19, 12.9, 0.02), (20, 12.7, 0.02)]
19 anti=15 [(20, 13.0, -0.01), (18, 12.9, 0.02)]
20 anti=16 [(19, 13.0, -0.01), (18, 12.7, 0.02)]
21 anti=17 [(23, 12.2, 0.02), (22, 12.1, 0.01)]
22 anti=1 [(23, 12.2, 0.01), (21, 12.1, 0.01)]
23 anti=10 [(22, 12.2, 0.01), (21, 12.2, 0.02)]

Rsim-Cmech q=0.10 pct5=0.005 med=1.218 ev=[0.017..0.360]
   rec=0.8672 frac9=0.417
Rsim-Cmech q=0.20 pct5=0.015 med=1.743 ev=[0.015..0.722]
   rec=0.7912 frac9=0.125
Rsim-Cmech q=0.30 pct5=0.067 med=1.798 ev=[0.013..0.939]
   rec=0.8108 frac9=0.375

Rsim-Cmech q=0.04 pct5=0.004 med=0.004 ev=[0.018..0.163]
   rec=0.9385 frac9=0.792
Rsim-Cmech q=0.06 pct5=0.007 med=0.875 ev=[0.017..0.269]
   rec=0.8986 frac9=0.625
Rsim-Cmech q=0.10 pct5=0.011 med=0.911 ev=[0.007..0.410]
   rec=0.8825 frac9=0.542

Rsim-weak nf=28 f=0.04 c=0.15 pct5=0.137 med=0.898 ev=[0.011..0.197] ratio=17.7
   rec=0.8771 frac9=0.464
Rsim-weak nf=28 f=0.04 c=0.30 pct5=0.352 med=0.922 ev=[0.016..0.229] ratio=14.3
   rec=0.7819 frac9=0.107
Rsim-weak nf=32 f=0.04 c=0.30 pct5=0.257 med=0.903 ev=[0.022..0.224] ratio=10.4
   rec=0.6946 frac9=0.031

Rsim-PM f=0.045 c=0.20 pct5=0.007 med=0.901 ev=[0.055..0.109] ratio=2.0
   rec=0.9967 frac9=1.000
Rsim-PM f=0.045 c=0.40 pct5=0.007 med=0.916 ev=[0.042..0.150] ratio=3.6
   rec=0.9764 frac9=1.000
Rsim-PM f=0.050 c=0.60 pct5=0.009 med=1.225 ev=[0.031..0.209] ratio=6.8
   rec=0.9707 frac9=1.000

{
 "recovery": 0.9999325264809333,
 "frac_recovered_0.9": 1.0,
{
 "recovery": 0.9877329931761921,
 "frac_recovered_0.9": 1.0,

P (48, 12) float32 unit rows: True shipped score: 0.9977 accepted: True
Q (64, 16) float32 unit rows: True shipped score: 0.9981 accepted: True
R (48, 12) float32 unit rows: True shipped score: 0.9966 accepted: True
S (64, 16) float32 unit rows: True shipped score: 0.9965 accepted: True
dev A shipped score: 0.9992 l0: 0.933 accepted: True
dev B shipped score: 0.9971 l0: 9.23 accepted: True
dev C shipped score: 0.9978 l0: 2.043 accepted: True
dev D shipped score: 0.9966 l0: 11.394 accepted: True

VALID_JSON

44 /workspace/submission/summary.md
/workspace/submission:
total 24
drwxr-sr-x 3 1001 1001 4096 Nov 10 08:18 .
drwxrwsrwx 5 root 1001 4096 Nov 10 07:48 ..
-rw-r--r-- 1 1001 1001 1848 Nov 10 08:17 answer.json
drwxr-sr-x 2 1001 1001 4096 Nov 10 08:04 dictionaries
-rw-r--r-- 1 1001 1001 7500 Nov 10 08:18 summary.md

/workspace/submission/dictionaries:
total 32
drwxr-sr-x 2 1001 1001 4096 Nov 10 08:04 .
drwxr-sr-x 3 1001 1001 4096 Nov 10 08:18 ..
-rw-r--r-- 1 1001 1001 2432 Nov 10 08:04 P.npy
-rw-r--r-- 1 1001 1001 4224 Nov 10 08:04 Q.npy
-rw-r--r-- 1 1001 1001 2432 Nov 10 08:04 R.npy
-rw-r--r-- 1 1001 1001 4224 Nov 10 08:04 S.npy

{
 "dev_shipped": {
  "A": {
   "dictionary_score": 0.9992,
   "recovery": 0.9999
  },
  "B": {
   "dictionary_score": 0.9971,
   "recovery": 0.5833
  },
  "C": {
   "dictionary_score": 0.9978,
   "recovery": 0.9877
  },
  "D": {
   "dictionary_score": 0.9966,
   "recovery": 0.728
  }
 },
 "dev_accepted_but_low_recovery": [
  "B",
  "D"
 ],
 "audit": {
  "P": {
   "shipped_dictionary_score": 0.9977,
   "method": "modified",
   "method_note": "Same featlib SAE learner but retuned: n_atoms=4*hidden_dim=48 (was 2*m) and l1=0.1 (was 0.03); shipped defaults split features (dev B: 32 features > 24 atoms, recovery 0.58). P is noiseless (5th-pct centred norm 0.007), so no refit.",
   "estimated_recovery": 0.96,
   "audit_ready": true