|
Testify is, I think, the most "me" thing I've built yet. You can get it from the App Store now. The idea is stupidly simple but the implementation might be a bit too complicated: it lets you run actual A/B tests on yourself. Not on users. Not on a website. On you. Does drinking coffee actually help you get through your task list, or have you just been telling yourself that for ten years? Does the expensive running shoe make you faster, or is that placebo and marketing? You can just... check. With real statistics. On your phone. Let me walk through how it works, and then I'll get into some experiments I've actually been running with it, because that's the fun part. The basic idea: the Scientific method The basic idea: hypothesis, variable, result. Every experiment in Testify starts with three things:
Pick the result type and Testify quietly picks the right statistical test for you behind the scenes, a chi-squared test for yes/no outcomes, a t-test for numbers. You do not need to know what either of those mean. That is rather the point of the app. You can also turn on a daily notification, which is honestly the feature that makes this whole thing work, because the number one failure mode of any self-experiment is forgetting to log the data. More on that in a second. Logging a trial Each day (or each instance of the thing happening, if it's not daily) you add a trial. A trial is just three pieces of information: did you apply the variable today (toggle on or off), what was the result, and when. That is it. Takes about ten seconds. Here's the screen where you'd review an experiment in progress:  This one is tracking "Does drinking coffee make me finish my daily tasks?" After 24 trials, you get a results chart right there, showing how often "Success" happened in the control days versus the coffee days. Green bar bigger on one side, you've got a hint of an answer already, before you even read a single statistic. Reading the actual statistics, in actual English I did not want to build an app that spits out "p = 0.007" and leaves you to go Google what that means - so Testify explains the result in plain English first, then gives you the numbers underneath if you need them.  For my coffee experiment: "When you apply 'Drank coffee', Success happens noticeably more often than without it." Then below that: control group hit 0% success, treatment group hit 58%, p-value of 0.007. There's also a statistical power reading (98% in this case, which is excellent), and a note about whether you have enough data yet to actually conclude anything. Red means keep collecting. Green means you're good to call it. There's one detail in there worth calling out: a "sequential threshold" check. If you're the type of person who peeks at results constantly (you are, I am too), checking your p-value every single day after every single trial technically inflates your false-positive rate. Testify accounts for that, so the "you can conclude this" recommendation already knows you've been peeking and adjusts accordingly. I added this after embarrassing myself with my own experiments more than once. Bayesian or Frequentist, your call Some people think in p-values. Some people think in probabilities. Testify doesn't make you pick a side, there's a toggle.  Same coffee data, but now phrased as "Probability 'Drank coffee' is better: 100%," with an expected lift of +49.9 percentage points. I will be honest, I find the Bayesian framing more intuitive for day to day decisions ("there's a 95% chance this helps" just clicks faster than a p-value does), but I kept both because some people were trained on frequentist stats in school and that's just how their brain parses certainty. No reason to fight that. Staying on it The other thing that kills self-experiments is simply forgetting they exist. You start tracking something on a Monday with great enthusiasm and by the following Wednesday the app is buried four folders deep and the experiment quietly dies of neglect.  So there's a home screen widget that shows your most relevant running experiment right there, current trial count, current split, an Add Trial button sitting right on it. Combined with the daily notification (which links straight into the experiment when you tap it), the friction to logging a day's data is about as close to zero as I could get it.
Experiments I've actually been running OK, here's the part I actually wanted to write about. A few of the things I've tested on myself so far: Coffee versus tasks. The one in the screenshots is real, that's actually my data. Variable was "drank coffee before noon," result was a yes/no on "did I finish my top 3 tasks that day." Control group: 0 out of 12 successes. Coffee group: 7 out of 12. p = 0.007. I was fairly smug about this result, not gonna lie, though I'll admit the experiment design has an obvious confound: the days I remember to make coffee are probably also the days I'm already more on top of things in general. Correlation, causation, all that. Still nice to see it laid out. Creatine and running distance. Numeric result this time, distance in km on my usual loop, variable being whether I'd taken creatine that morning. This one came back inconclusive after about two weeks, statistical power sitting around 40%, which the app flagged orange and told me, correctly, that I needed more trials before drawing any conclusion. Which is its own useful answer. "Not yet proven" is a perfectly fine result, and a lot more honest than what I would have told you if you'd just asked me how I felt about it. Reading before bed versus scrolling before bed. Variable: read a physical book for 20 minutes instead of phone. Result: hours of sleep, logged from my watch the next morning. Still running this one. Early read is leaning the way you'd expect, but power's not there yet, so I'm not calling it. The shoe thing. I have, in fact, started an experiment comparing two pairs of running shoes on 5k time. This is the platonic ideal of a Testify experiment and I'm slightly delayed on it because I keep forgetting which shoes I wore, which, ironically, is exactly the kind of friction the notification feature exists to prevent. Working on it. The pattern across all of these: the result usually does not confirm what I expected going in, or it confirms it but weaker than I expected, or it tells me I simply don't have enough data and need to stop having opinions for another week. All three outcomes are more useful than however confident I felt about the question before I started measuring anything. A couple of practical notes A few things worth knowing if you're going to try this yourself:
Wrapping up I built this because I got tired of having strong opinions about my own life backed by absolutely no data, just vibes and whatever I happened to notice on the one day it confirmed what I already believed. Testify is my attempt to fix that, for myself first, and now for anyone else who's curious enough to actually check. Go pick something you've always assumed about yourself and test it. You'll probably be at least a little wrong, and that's the good part. Comments are closed.
|
(c) 2026 ganglion software. See license page for more info.