Advanced analysis boundaries

Start with the implemented methodology. This page distinguishes supported options from analyses requiring a separate statistical specification.

BCa and stratified DiD inference remain experimental after the 0.5 release screen. Numerical agreement with an independent calculation does not guarantee interval coverage.

Few clusters

The number of independent geos limits information. More dates do not create more randomised units. Parallel inference uses geo summaries and Welch–Satterthwaite intervals. CRT uses clustered covariance and a t(G-1) reference. Neither procedure provides a universal small-sample guarantee.

For parallel designs, the optional conditional randomisation test gives a sharp-null test under the stated allocation mechanism. With very few admissible allocations, the attainable p-values are coarse. Report that limitation rather than increasing the number of Monte Carlo draws and implying more information.

Exposure and non-adherence

The primary experimental contrast concerns assignment. An effect of exposure requires additional identification assumptions. Random assignment can sometimes serve as an instrument, but exclusion, relevance and the interpretation of the resulting local effect need justification. Spillovers and changes to other marketing activity can violate those assumptions.

The package does not implement an instrumental-variable estimator or a weak-instrument diagnostic. The previous illustrative two-regression snippet has been removed. Do not use an R-squared threshold as an instrument-validity test.

Delayed effects and interference

The CRT treatment indicator represents an immediate common additive effect. A rollout with gradual adoption, carryover or exposure-time effects requires a prespecified extension. Calendar-period adjustment alone does not model these mechanisms. Inspect whether the schedule can distinguish the proposed effects.

Geographic buffer zones and operational delivery checks may reduce contamination. They do not establish that it is absent. State which forms of interference the causal interpretation excludes and what evidence supports those exclusions.

Missing outcomes and unusual values

The supported parallel workflow rejects incomplete supplied panels. It does not implement imputation, inverse-probability weighting or bounds for attrition. Choose and justify such methods before analysing incomplete trial outcomes.

Do not delete large post-treatment outcomes because they weaken significance or look unusual. Treatment can change the outcome distribution. Prespecify any outcome transformation or eligibility rule using pre-treatment information.

Multiplicity and monitoring

geostep.diagnostics.multiple_testing_correction provides Bonferroni, Holm and Benjamini–Hochberg adjustments for a declared family of valid p-values. Choose the family and error criterion in advance. Benjamini–Hochberg control needs its usual dependence assumptions. These adjustments do not make repeated peeking at a growing experiment valid.

Unsupported extensions

GeoStep does not currently provide Bayesian experimental models, treatment-effect heterogeneity estimators, IV analysis, spatial interference models, staircase power optimisation or sequential testing. Previous illustrative snippets were not validated implementations and have been removed.

For any proposed extension, define the estimand, assignment mechanism, observation process, estimator, uncertainty calculation and failure criteria together. Check against an independent reference and run design-specific simulation before making decision claims. See the repair contract and parallel validation.