A/B testing for creators: a beginner's guide

A/B testing has a branding problem. The phrase sounds like something done in a windowless room by someone with three monitors and a Confluence page. In practice, an A/B test is just this: 'I think Version B might be better than Version A. Let's show both to people, see which one wins, and stop arguing about it.'

If you've ever picked between two thumbnails, two captions, or two pinned posts based on a hunch — you've already done the loose, unstructured version. A real A/B test just adds a clean experimental setup, a clear metric, and the discipline to wait long enough to trust the result.

For creators, A/B testing your link-in-bio page can mean the difference between 100 clicks a week and 300, with zero new followers. Same audience. Better page. That's the trade.

What can you actually test?

Almost anything visible to a visitor. The trick is testing one thing at a time, otherwise you can't tell which change moved the needle.

• Page headline or bio text — the most impactful test for most pages
• Link titles — second-most impactful, easy to iterate on
• Order of links — does the merch link work harder above or below the YouTube link?
• Button text ("Stream now" vs "Listen here" vs "Tonight's episode")
• Profile photo, cover photo, or theme — visual changes can shift trust
• Whether to show a countdown timer on a launch link
• Whether to display social proof numbers ("14,000 downloads")

How a basic A/B test actually works

Behind the scenes, an A/B test is just a coin flip. When a visitor lands on your page for the first time, the system flips that coin: heads, they see Variant A; tails, they see Variant B. Their assignment is then locked in (via cookie or hashed identifier) so they don't see different versions on different visits — that would muddy the data.

After enough traffic — usually 200–500 visitors per variant — you check which version got more clicks on your primary CTA. If one is meaningfully ahead, you have a winner. You promote it to 100% of traffic and start your next test.

The single most important rule: change one thing

If you change your headline AND your button colour AND your link order all at once, you've turned your test into a guessing game. Variant B might win — but you have no idea which of the three changes did the work, or whether two of them helped and one hurt.

Change one variable. Measure it. Document the result. Then change the next variable. Boring? Yes. Reliable? Also yes.

How long should you actually run a test?

At least one full week. Always. Even if you're 95% confident on day three.

Why? Because traffic patterns vary by day of week. People who visit your bio on a Saturday afternoon are not the same crowd as the ones who scroll past during a Wednesday lunch. A test that ran from Sunday to Wednesday will favour weekend behaviour. A test that ran from Wednesday to Friday will miss your weekend visitors entirely. One full week (Monday-to-Monday or any other 7-day stretch) gives you a representative sample of when your audience actually shows up.

For most creators, two weeks gives reliable, defensible results. Don't stop early just because one variant is leading on day two. The phenomenon of early leads disappearing into ties is so common it has a name: regression to the mean.

How to read the results without fooling yourself

The single most common rookie mistake is comparing raw clicks instead of click-through rate (CTR).

Variant A got 150 clicks. Variant B got 120 clicks. So A wins, right? Maybe. Or maybe Variant A got served to 400 visitors and Variant B only got served to 280. Now A's CTR is 37.5% and B's is 42.8% — and B is actually winning by a meaningful margin despite having fewer raw clicks.

Always look at the rate, not the count. CTR = clicks ÷ impressions. That's the number that comparing variants on equal footing.

A simple sanity table

• Under 100 visitors per variant → too early to call anything
• 100–300 per variant → directional, not definitive
• 300+ per variant → trust the trend, look at confidence
• 95%+ confidence → declare a winner and move on

What to do after a winner emerges

Promote the winner to 100% of traffic. Take 30 seconds to write down what you tested, what won, and what you think the lesson is. (Future you will thank present you — patterns emerge over months that you can't see in a single test.) Then queue your next test.

Great creators run a continuous testing loop. There's always something to optimise: a button label, a link order, a section heading, a hero image. Even small lifts compound. A 5% improvement every two weeks is a 70% improvement over six months.

What if there's no clear winner?

After two weeks and 500+ visitors per variant, the two variants are within 2% of each other. What does that mean?

It means those two variables perform equally for your audience. That's a result, not a failure. The implication is that you should pick whichever variant you prefer (the one that fits your brand, the one that's easier to maintain) and move on to test something more impactful.

If you keep getting ties on subtle copy changes, it's a strong signal that copy is not your bottleneck. Move up the stack: test page structure, hero choice, profile photo. Bigger swings, bigger results.

Mistakes I see weekly

• Stopping a test early because Variant B looks great on day two. Don't. Wait the full week.
• Running tests with not enough traffic. Under 100 visitors per variant is just superstition with extra steps.
• Letting tests run forever. After 30 days with no winner, the variables are functionally identical.
• Forgetting to declare a winner. Tests that just keep running cost you the conversion uplift you already won.
• Changing the destination URL between variants. You're now testing two completely different things.

A worked example

Last quarter we worked with an indie songwriter testing their hero link title. Variant A: 'Stream my new EP'. Variant B: 'Hear what I made when I almost quit music'.

A felt safer. B felt vulnerable, almost too personal. After 11 days at 50/50 split, B was at 41% CTR vs A at 24%. B got promoted to 100%. Streams that month nearly doubled — same audience, same EP, just a title that gave the visitor a reason to care.

The lesson isn't 'be vulnerable on every link.' The lesson is: don't trust your gut about copy. Test it. Some of the most uncomfortable phrasing produces the best numbers, and you'll never find that out by guessing.

Your first test, this week

1. 1Pick the link on your page that gets the most clicks today.
2. 2Write a second title for it that says the same thing in a meaningfully different way.
3. 3Set up a 50/50 split and start the test.
4. 4Don't look at the results before day 5.
5. 5On day 7, check CTR. Declare a winner if confidence is over 95%.
6. 6Document what won. Plan your next test.