Initial feature gating and A/B testing integration

[gaearon]

This introduces a proof of concept setup we plan to use for gated feature rollout and AB testing with Statsig.

Scope

This proof of concept is very limited:

• Currently it's web-only. I'll hook up native later on after we get the initial sense of event volume.
• We're not yet sending any product analytic events except the built-in ones statsig does (like crash reporting and some page load metrics). We'll be adding those later.
• We're not yet applying any sampling. We'll do that later after we get a sense of the volume.

The goal here is just to get something running and make sure the setup works end-to-end.

Details

I've chosen a slightly quirky approach. I want to preserve two constraints:

1. We should never block startup on fetching the A/B gate config.
2. We also shouldn't change A/B values midsession for the user. (At least on the web.)

So the approach I'm going with is that we always use stale values from localStorage from the last run. This means that feature rollout settings (e.g. bumping the %) kicks in on the next page load rather than on the page load after it was done. So rollouts lag behind a bit, but the upside is the UI is always consistent and startup is fast.

Statsig SDK already deals with localStorage for us, so no need to do something special.

See code comments for more details.

statsig_v0.mov

Let's experiment with this approach and see if there are any downsides.

How to use this

You call useGate('test_gate') and it gives you false (default) or true.

Gates can be created in the Statsig console.

Here's the toy code I used in the video above.

import {useGate} from '#/lib/statsig/statsig'

function TestGate() {
  const value = useGate('test_gate')
  let color = value ? 'green' : 'red'
  let text = value ? 'PASS' : 'FAIL'
  return (
    <div
      style={{
        position: 'fixed',
        left: '50%',
        transform: 'translateX(-50%)',
        backgroundColor: color,
        width: 200,
        height: 200,
        color: 'white',
        fontSize: 30,
        zIndex: 100,
        fontFamily: 'monospace',
      }}>
      {text}
    </div>
  )
}

[gaearon]

This one technically isn't used yet.