Experiment Canvas

The left side of the experiment canvas is where we capture the details of the experiment we want to run. If we’re using an experimentation approach for our day-to-day work, then it gives us the chance to think about it differently, we’re now adding success criteria in the form of a measurable metric as well as clearly defining the method we will use.
Use the “Riskiest assumption or risk” box to capture the most impactful risk or assumption from the analysis earlier.
In the “Testable hypothesis” box formulate the hypothesis described earlier, confirming that the metric, (“We will know this is true when,”) is a real success measure and not a vanity metric or merely the easiest one to baseline and track.
Plan out the steps you will take in executing the experiment in the “Method” box being careful to define a method that supports an effective experiment.
• Include any setup or clean up steps.
Do you need to baseline the metric first, or do you already have it?
How will you collect the metric once the, “Within,” period has elapsed?
Are there any elements of your experiment that need to be cleaned up afterwards?
• Is it possible to define a control group?
If possible it will help to understand if your experiment is proven, disproven or inconclusive is you leave a subset of the total unchanged so you can demonstrate that the impact of the changes
• How much can you reduce the “observer effect”
When making the changes as an experiment, you become the observer. The mere fact that the changes are being observed will impact the result, try to eliminate yourself and your observations as much as possible from the actions or changes taking place. If you can step back entirely and simply measure the results after the, “Within,” period, you will go a long way towards achieving this.
Execute the experiment as per the method.
Gather the results, try to be as objective as possible when gathering the results and be sure to record as much information as possible about the outcome of the experiment.
• What happened?
• What did you observe?
• What aspects of the implementation as well as the outcome were unexpected?
• Were there any other things going on that might have impacted the metric so as to reduce your clarity that it was the experiment that made the impact on the other things.
Decide about the overall outcome.