Contributing

Dev setup, commands, conventions, and the change-control processes that keep the statistics trustworthy. Companion to architecture.md (as-built) and docs/specs/ (design contracts).

Session skills (.claude/skills/)

Three repo-local skills encode the rituals this track re-derives every session. Invoke them by name (/abk-wp, …) or let them trigger on the work:

Skill	When
abk-wp	one work package end to end — branch → implement → adversarial review → sync the three doc bodies → the full gate → PR → 10 CI jobs → squash → memory + handoff
abk-release	a version cut: bump, cut [Unreleased], sweep the previous release’s status lines, tag vX.Y.Z (that is what publishes), verify from PyPI in a clean venv
abk-web-bundle	any web/src/** edit — rebuild, commit the artifact in the SAME PR, and the build/CI gates (globals, §4 markers, the </script hazard, brand-token lockstep, jsdom)

They are contributor context and are not shipped in the wheel — the abk init-claude payload for a user’s project is abkit/cli/assets/claude/.

Setup

pip install -e ".[dev]"          # numpy/scipy/statsmodels/pydantic/click/jinja2 + pytest/black/ruff/mypy
pip install -e ".[dev,all-db]"   # + clickhouse-driver / psycopg2 / pymysql (DB work)
pre-commit install

Python ≥ 3.10. pip package ab-analysis-kit, import package abkit, terminal command abk.

Commands

What	Command
All tests	python3 -m pytest tests/
Stats unit tests	python3 -m pytest tests/stats/
Golden (legacy-parity) tests	python3 -m pytest tests/golden/
Lint/format/types (ruff, black, mypy over abkit/)	pre-commit run --all-files
Version	single source: __version__ in abkit/__init__.py

CI runs the full matrix on every push; keep it green.

Conventions

• numpy-first, no pandas in core logic. Vectorise; avoid Python loops over units.
• Type hints everywhere; mypy runs over abkit/ in pre-commit.
• Docstrings cite the governing spec section (e.g. docs/specs/declarative-config.md §7) when implementing a contract — the spec is the requirement, the docstring is the pointer.
• Commit style: conventional commits scoped by package — feat(stats): …, fix(stats): …, docs(specs): …, chore: …, ci: ….
• CHANGELOG.md (Keep a Changelog) is authoritative for behavior changes; update it in the same PR.
• Repo docs and code comments are English; keep comments to constraints the code can’t show.

Adding a statistical method (the plugin checklist)

1. One BaseMethod subclass in abkit/stats/parametric/ or bootstrap/; decorate with @register (canonical name, optional aliases).
2. Declare params as ParamSpecs — typed, defaulted, identity-flagged (seed must be identity-excluded for bootstrap methods).
3. Implement both entries where the math allows: from_samples and from_suffstats (dual-entry equivalence is tested). A bootstrap-family method implements _resample instead of from_samples — the base class composes the two halves (see step 4b).
4. (Optional, M7) If the method can score suffstats arrays, opt in to the vectorized validate path: set supports_vectorized = True + implement from_suffstats_array → BatchEffectResult, route every power term through effects._libm_pow (numpy ** is 1 ULP off libm — the parity gates demand bit-exact scalar↔batch agreement), and extend the capability-roster test in tests/stats/test_vectorized_parity.py. Without the flag the method just takes the scalar fallback — never required. 4b. (Optional, M10 WP5) If the method’s cost is an alpha-INDEPENDENT step followed by a cheap alpha-dependent finish, split it: implement _resample(s1, s2) -> ResampleOutcome (immutable — warnings is a tuple, because _finalize appends to the list it is given) plus a _finalize that reads the outcome without mutating it, inherit from_samples from the base class, and set supports_resample_memo = True. Explore then redraws nothing across an alpha drag. The roster gate in tests/stats/test_bootstrap_methods.py requires flag, override and inherited template together; without the flag the method is simply recomputed per alpha — never special-cased. 4c. (M13 STAT-3a, amended by STAT-4) If the method’s CI is not effect ± z·SE — a score, Fieller or percentile interval — say so: set asymmetric_ci = True. When a PARAM selects the interval shape, declare it on the ParamSpec (asymmetric_values=("fieller",)) rather than resolving it in __init__: BaseMethod folds every spec’s declaration into the bound instance, so a method that merely adopts a shared spec cannot forget the capability. (STAT-3 resolved it in ZTest.__init__; STAT-4 added a second param-switched interval across five methods, which is where one hand-written resolution per class turns into five copies of a knob-dependent fact.) It stays a plain attribute rather than a ClassVar precisely so the instance can differ from the class. Every SE-by-CI-inversion entry then refuses loudly instead of widening a number that is not a standard error. Do not reach for supports_sequential = False to express the consequence: that flag is a ClassVar read at CLASS level by five eligibility gates, so a param-switched narrowing is invisible to all of them (m13 STAT-3). The contradiction is caught where it is STATIC — validate_experiment_level2 errors on sequential.enabled beside an asymmetric interval, naming both knobs — and the AsymmetricCIError at the inversion is the backstop under it. The roster gate in tests/stats/sequential/test_asymmetric_ci_guard.py asserts no method declares the flag at class level and enumerates every declared asymmetric (method, param, value) triple, so flipping the CLASS default — or adding a configuration nobody recorded — is a conscious act. Shipped examples: z-test’s interval: score and the five mean methods’ interval: fieller. A param that is consulted only under one test_type also declares relative_only=True, which makes the inert combination a refusal instead of a no-op that still forks method_config_id.
5. Tests: known-answer test; dual-entry equivalence; params/identity hash addition to tests/stats/test_identity.py; golden test if reproducing a legacy method.
6. Never touch the pipeline/DB/CLI to make a method work — if you need to, the design is wrong (methods are plugins).

Changing a statistical number (change control — hard rule)

Any deviation from the captured baseline (docs/specs/statistics-baseline.md):

1. Bump the method’s ALGORITHM_VERSION.
2. Record the deviation in docs/specs/statistics-changes.md (what, why, expected numeric impact).
3. Entry in CHANGELOG.md.
4. A/A validation through abk validate (shipped in M4): run the matrix on the affected method + metric and confirm FPR ≈ α / power holds. The golden tests still pin the baseline in parallel.

Golden tests pin the baseline, not your improvement: a deliberate deviation gets a new test; the baseline reproduction stays intact behind its original entry (legacy-parity mode). Tolerance is relative 1e-9 — never loosen it to make a test pass.

Porting from detectkit (M2+)

The donor is /home/aleksei/wsl_analytics/detektkit. Components marked ⟲ in architecture.md §4 port near-verbatim: rename dtk→abk, detectkit→abkit, _dtk_*→_ab_*; keep the donor’s structure/tests where they hold. Anything metric-primary-shaped (detectkit’s primary entity) must be consciously reshaped to experiment-primary — flag it in the PR rather than silently diverging.

Release checklist (from M6 onward — first tagged release is 0.1.0)

• __version__ (in abkit/__init__.py) bumped; CHANGELOG.md [Unreleased] cut into a dated section; the Dev Status classifier in pyproject.toml current.
• The three single-source bodies tell one story: (a) the user docs docs/ (rendered to the site), (b) the contributor rules .claude/rules/, (c) the packaged init-claude operator assets abkit/cli/assets/claude/ (the managed block + 9 rules + 7 skills). The drift gate tests/docs/test_docs_single_source.py asserts every packaged operator rule has a published docs/ home.
• Packaging DoD: the built wheel ships report.js + explore.js + dashboard.js (M11) + every abkit/cli/assets/claude/** asset, and the pip install-smoke job proves abk --version + abk init-claude resolve from a clean-venv install across Py 3.10/3.11/3.12. web/ never ships in the wheel. A new bundle must be added to two hand-maintained namelists — .github/workflows/ci.yml (the wheel gate) and tests/e2e/test_release_readiness.py (the self-contained-bundles check) — neither of which is derived from the build config.
• Layout to keep in mind when touching a release: cli/assets/claude/ (init-claude payload), abkit/notify/ (abk test-report channels), website/ (the Astro site, auto-redeploys on website/** push to main).
• Website sync (abkit.pipelab.dev) is automatic on website/** merge to main.
• PyPI publish is tag-triggered by CI (publish.yml, OIDC trusted-publisher); the maintainer pushes v<version> after the exit gate is green (never before the version bump — a duplicate version upload is rejected).