|
Testify is, I think, the most "me" thing I've built yet. You can get it from the App Store now. The idea is stupidly simple but the implementation might be a bit too complicated: it lets you run actual A/B tests on yourself. Not on users. Not on a website. On you. Does drinking coffee actually help you get through your task list, or have you just been telling yourself that for ten years? Does the expensive running shoe make you faster, or is that placebo and marketing? You can just... check. With real statistics. On your phone. Let me walk through how it works, and then I'll get into some experiments I've actually been running with it, because that's the fun part. The basic idea: the Scientific method The basic idea: hypothesis, variable, result. Every experiment in Testify starts with three things:
Pick the result type and Testify quietly picks the right statistical test for you behind the scenes, a chi-squared test for yes/no outcomes, a t-test for numbers. You do not need to know what either of those mean. That is rather the point of the app. You can also turn on a daily notification, which is honestly the feature that makes this whole thing work, because the number one failure mode of any self-experiment is forgetting to log the data. More on that in a second. Logging a trial Each day (or each instance of the thing happening, if it's not daily) you add a trial. A trial is just three pieces of information: did you apply the variable today (toggle on or off), what was the result, and when. That is it. Takes about ten seconds. Here's the screen where you'd review an experiment in progress:  This one is tracking "Does drinking coffee make me finish my daily tasks?" After 24 trials, you get a results chart right there, showing how often "Success" happened in the control days versus the coffee days. Green bar bigger on one side, you've got a hint of an answer already, before you even read a single statistic. Reading the actual statistics, in actual English I did not want to build an app that spits out "p = 0.007" and leaves you to go Google what that means - so Testify explains the result in plain English first, then gives you the numbers underneath if you need them.  For my coffee experiment: "When you apply 'Drank coffee', Success happens noticeably more often than without it." Then below that: control group hit 0% success, treatment group hit 58%, p-value of 0.007. There's also a statistical power reading (98% in this case, which is excellent), and a note about whether you have enough data yet to actually conclude anything. Red means keep collecting. Green means you're good to call it. There's one detail in there worth calling out: a "sequential threshold" check. If you're the type of person who peeks at results constantly (you are, I am too), checking your p-value every single day after every single trial technically inflates your false-positive rate. Testify accounts for that, so the "you can conclude this" recommendation already knows you've been peeking and adjusts accordingly. I added this after embarrassing myself with my own experiments more than once. Bayesian or Frequentist, your call Some people think in p-values. Some people think in probabilities. Testify doesn't make you pick a side, there's a toggle.  Same coffee data, but now phrased as "Probability 'Drank coffee' is better: 100%," with an expected lift of +49.9 percentage points. I will be honest, I find the Bayesian framing more intuitive for day to day decisions ("there's a 95% chance this helps" just clicks faster than a p-value does), but I kept both because some people were trained on frequentist stats in school and that's just how their brain parses certainty. No reason to fight that. Staying on it The other thing that kills self-experiments is simply forgetting they exist. You start tracking something on a Monday with great enthusiasm and by the following Wednesday the app is buried four folders deep and the experiment quietly dies of neglect.  So there's a home screen widget that shows your most relevant running experiment right there, current trial count, current split, an Add Trial button sitting right on it. Combined with the daily notification (which links straight into the experiment when you tap it), the friction to logging a day's data is about as close to zero as I could get it.
Experiments I've actually been running OK, here's the part I actually wanted to write about. A few of the things I've tested on myself so far: Coffee versus tasks. The one in the screenshots is real, that's actually my data. Variable was "drank coffee before noon," result was a yes/no on "did I finish my top 3 tasks that day." Control group: 0 out of 12 successes. Coffee group: 7 out of 12. p = 0.007. I was fairly smug about this result, not gonna lie, though I'll admit the experiment design has an obvious confound: the days I remember to make coffee are probably also the days I'm already more on top of things in general. Correlation, causation, all that. Still nice to see it laid out. Creatine and running distance. Numeric result this time, distance in km on my usual loop, variable being whether I'd taken creatine that morning. This one came back inconclusive after about two weeks, statistical power sitting around 40%, which the app flagged orange and told me, correctly, that I needed more trials before drawing any conclusion. Which is its own useful answer. "Not yet proven" is a perfectly fine result, and a lot more honest than what I would have told you if you'd just asked me how I felt about it. Reading before bed versus scrolling before bed. Variable: read a physical book for 20 minutes instead of phone. Result: hours of sleep, logged from my watch the next morning. Still running this one. Early read is leaning the way you'd expect, but power's not there yet, so I'm not calling it. The shoe thing. I have, in fact, started an experiment comparing two pairs of running shoes on 5k time. This is the platonic ideal of a Testify experiment and I'm slightly delayed on it because I keep forgetting which shoes I wore, which, ironically, is exactly the kind of friction the notification feature exists to prevent. Working on it. The pattern across all of these: the result usually does not confirm what I expected going in, or it confirms it but weaker than I expected, or it tells me I simply don't have enough data and need to stop having opinions for another week. All three outcomes are more useful than however confident I felt about the question before I started measuring anything. A couple of practical notes A few things worth knowing if you're going to try this yourself:
Wrapping up I built this because I got tired of having strong opinions about my own life backed by absolutely no data, just vibes and whatever I happened to notice on the one day it confirmed what I already believed. Testify is my attempt to fix that, for myself first, and now for anyone else who's curious enough to actually check. Go pick something you've always assumed about yourself and test it. You'll probably be at least a little wrong, and that's the good part. It has been quite a while since I last posted here. I've been busy building stuff for the Mac - and one of those things, Quickscriber, finally got an MCP server, which means it's time to write about it 😄 So anyway, a quick bit of background for anyone who hasn't used Notational Velocity. NV was this beautifully minimal Mac app from way back - no folders, no save button, just a hotkey, a search box, and a blank space to type into. You hit the shortcut, the window pops up, you type, you hit the shortcut again, it's gone. The genius of it wasn't really the features (there weren't many), it was that it got out of your way. Capturing a thought took exactly as long as typing the thought.  Quickscriber is my attempt to bring that same idea forward to a modern Mac, with Markdown, Mermaid diagrams, KaTeX, the works. But the part I actually want to talk about today is the thing NV never had and honestly couldn't have had in 2010: an MCP server, so an AI agent - Claude Code, in this case - can read and write your notes directly. That sounds like a small feature. It is not a small feature. Let me explain why. Engineering managers and idea dumping Being an engineering manager (or really, doing any job with lots of small unstructured inputs) means your day looks something like this: someone mentions a flaky test in passing, you have a half-formed thought about reorg during your commute, a Slack thread surfaces a risk nobody's tracking, you remember mid-meeting that you still haven't followed up with Jose about the tech lead thing. None of that is a "document." None of it deserves a ticket yet. But all of it needs to land somewhere, immediately, or it's gone. That's the capture problem, and Quickscriber solves it the same way NV always did - friction approaching zero. But capture was never really the hard part. The hard part is what happens to all those notes after you've dumped them. Normally: nothing. They just sit there until you manually go digging, usually right when you need them and can't remember what you titled the thing. MCP Servers and how to set it up This is where Claude Code comes in. It can do the digging for you. AI agents are getting better and better, and the biggest innovation the past year is their ability to use tools. The key thing is how to tell your agent that these tools exist. That is where the MCP server comes in - it lets your AI agent know that you applications have tools available for it to use. So let's set one up. For this example I'm going to show you how it is done with Claude, but in theory you can use it with any agent (even local, with LM Studio and an open-weight LLM like Qwen for example. I'll write something up for that later). OK, let's get into it. Open Quickscriber, go to Settings → Integrations, and flip on Enable local MCP server. You'll see something like this:
Copy that URL, then register it with Claude Code from your terminal: claude mcp add --transport sse quickscriber "http://127.0.0.1:8765/sse?token=YOUR_TOKEN"That's it. Restart your Claude Code session and you'll have access to five tools:
A couple of things worth flagging before you go wild with this, because they actually matter:
 Making sense of the data
OK but what do I actually DO with this? Right, the fun part. Here's the workflow I've landed on, and the thing to understand is it has two completely separate phases that you should NOT try to do at the same time: Phase 1: Capture everything, don't think Hotkey, type, hotkey, gone. Don't tag it, don't decide where it belongs, don't even finish the sentence if you don't want to. A day's worth of this looks like:
Phase 2: Let Claude Code impose structure on the pile, after the fact. This is the part NV never had an answer for, because NV had no idea what was in your notes. It could find them, but it couldn't think about them. Claude Code can. Turning yesterday's noise into today's TODO list. Some examples - instead of starting your morning trying to remember what was urgent, just ask: "Pull my Quickscriber notes from the last 2 days and draft today's TODO, prioritized by anything time-sensitive or blocking someone else."
"Search my notes from the last two weeks tagged #idea or #eng-priority, cluster them into themes, and draft a one-paragraph proposal for each I could bring to planning."
Maybe you just want to tidy everything up: "Find notes older than 30 days that look resolved based on later notes, and tell me which ones you'd delete - don't delete them yet." More on that "don't delete them yet" bit in a second. Now here's where it gets actually interesting: combine it with other MCP servers! This is the part that took the idea from "nice feature" to "oh, this changes how I work" for me. Claude Code doesn't just talk to Quickscriber - it can talk to Quickscriber AND your JIRA board AND your Google Calendar in the same breath, which means your scribbled notes can become real, scheduled, tracked work without you ever opening JIRA or Calendar yourself. With Jira MCP configured: notes become sprint tickets. Say you've got a pile of half-formed engineering complaints sitting in Quickscriber from the week. Instead of manually transcribing each one into a ticket (the part of ticket-filing everybody hates), you can say: "Look at my Quickscriber notes tagged #eng-debt from this week. For each one that's concrete enough to be actionable, create a Jira ticket in the MOB project with a title, description, and the tech-debt label. Then update the Quickscriber note with the ticket link." Claude Code reads the note with get_note, creates the ticket through the Jira MCP server, then calls update_note to write the ticket URL back into the original note. Now your scratch note and your tracked work are linked, and you never typed the same sentence twice. You can go the other direction too: "Pull my open JIRA tickets assigned to me with status 'In Progress', and create a Quickscriber note for each one I haven't touched in 5+ days, so I remember to follow up." That's Jira's structured-but-buried data becoming a fast, searchable scratch note again. Round trip complete. Quickscriber + Calendar - your personal assistant With Google Calendar MCP configured, notes become actual time on your calendar. This is the one I didn't expect to like as much as I do. A lot of EM notes are really just "I need to have a conversation with someone," and those conversations have a way of never happening unless they're on a calendar. "Find Quickscriber notes mentioning a 1:1 topic with Jose, summarize them into talking points, and create a calendar event for our next 1:1 with those points in the description." Or, for the planning-cycle stuff: "I have a note about pushing back on the analytics rewrite in Q3 planning. Create a 30-minute prep block on my calendar sometime before Thursday, and put a summary of my argument in the event description." Claude Code does the thinking (turning a half-sentence scribble into actual talking points), the Calendar MCP server does the scheduling, and Quickscriber stays the single place you dumped the original thought. None of the three tools needed to know about the other two ahead of time - that's the whole point of MCP, really. Other combos worth trying A few more I've poked at, in roughly ascending order of "how much do I trust the agent to just go do this":
Notice the pattern in basically all of these: Claude Code drafts, you approve. That's not me being timid for no reason. Here's the thing about create_note, update_note, and especially delete_note: you're giving an agent write access to a data store. Same logic applies once you've got Jira, Calendar, GitHub, Slack, and email all wired up - an agent that can create a ticket can also create fifty of them by misreading your intent, and one that can draft a calendar invite can, with a slightly different prompt, just send it. So a couple of habits I'd actually recommend, not just as a disclaimer to cover myself:
None of this is a reason not to do it - it's a reason to keep a human in the loop on the irreversible steps, same as you would for literally any tool that touches production. The thing I actually like about this setup is that it doesn't ask Quickscriber to be anything other than what NV always was - a frictionless place to dump a thought before it disappears. It just adds a second half to the equation that never used to exist: something that can come along after the dumping is done, read the whole pile, and turn it into a TODO list, a sprint ticket, a calendar invite, whatever the thought actually needed to become. Capture stays dumb and fast, on purpose. Structure happens later, on demand, and you don't have to be the one doing it. That's it for this one - go set up the MCP server, dump some thoughts in there, and see what Claude Code does with them. |
(c) 2026 ganglion software. See license page for more info.