Proof
Proof in progress.
There are no public results on this site yet. This page says so plainly, and it shows how a result will be scored before one is shown.
The method asks you to write the rule, state the odds, and prove the cause. The same three tests apply to the method itself.
- Status
- In progress
- Calls recorded
- 0
- Calls scored
- 0
What is here today
The method is written down. The scoring rules on this page are written down. The scorecard is public and it is empty.
No client results, no testimonials, and no figures are shown, because none have been published. An empty record is the honest record today.
How a call gets scored in public
Three tests, in this order. A result that skips one is not shown as proof.
Stated odds
Each call is written down with a date and the odds, before the outcome is known. The entry does not change after that.
Brier score against a baseline
When the outcome is known, the stated odds are scored. A Brier score is the average squared gap between the odds you stated and what happened. Lower is better. The score sits next to a baseline, such as always stating the base rate, so you can see whether the calls beat a simple guess.
Effect measured on held out data
Held out data is set aside before the work starts and is not used to build the rule or the model. A claim that something caused a lift has to hold there. A lift that only shows up on the data it was built from is not proof.
The calibration chart
Stated odds run one way. How often those calls happened runs the other. Honest odds sit on the diagonal. The chart below is empty because no calls have been scored in public yet.
How often it happened
No calls scored yet
Dashed line: the line you are aiming at
Stated odds, low to high
What this page will not show
- A figure with no source.
- A score with no baseline next to it.
- A claim of cause that was only tested on the data it came from.
- A client name, a logo, or a testimonial used in place of a scored result.
Where the record lives
The scorecard is the dated record. Each call goes in with its stated odds and gets scored later. It is empty today.
Open the scorecardHow a forecast is scoredAsk what caused it