A/B test your ReactJS app with PlanOut

What is A/B testing?

A/B testing is an important part of product development. It allows product owners to make informed decisions about the usage and uptake of different features of the product.

A/B testing can be used to run varied experiments on the products from small things like the text of a widget or the colour of a button to whether the product should have an onboarding flow or not.

Let’s take an example of A/B testing a feature so that it is more easy to understand how the entire thing works. Suppose we have a video conferencing app and say we want to decide whether the onboarding flow we show the user has any impact on the user performing a desired action, like turning on their video.

For this case, we would run an experiment where we would show the onboarding flow to 50% of our users and not show it to the rest. Finally at the end of the experiment we can look at the results to find out if the onboarding flow did indeed result in more number of users turning on their video.

Run fun experiments with PlanOut + ReactJS

We will use PlanOut to create and run these experiments.

PlanOut.js

PlanOut is an A/B testing library developed by Facebook to help engineers create the framework into their products easily. It does the heavy lifting for us by running the experiment and giving us a result.

It’s brilliance lies in the fact that it will make sure that an experiment run for a particular user will always return the same result.

For example, if we decide to show the onboarding flow to User A, PlanOut will make sure that User A will always see the onboarding flow — no matter how many times they launch our app.

You can read more about the PlanOut here.

We will use the JS port of the original framework. It can be found here.

The PlanOut.js github repository has an example implementation which is class based. The newer versions of ReactJS are hook based and we don't really have class based components anymore.

Therefore, in this article, we will see how we can use the factory pattern to create an experiment factory which will spit out experiments for us without creating different classes.

So let’s dive right into creating this factory.

Experiment factory

Create a new file called experiment.factory.ts

This file will house our experiment factory. Its job is to create experiments and have them ready for us to use.

First, let’s import the necessary bits of the PlanOut library.

import { Assignment, Experiment, Inputs, Params } from 'planout';

Once we have what we need from PlanOut, let’s first create a base experiment class. This class will contain the methods common to all experiments we create going forward.

class BaseExperiment extends Experiment<Inputs, Params> {
  configureLogger() {
		// Configure your logger here
    return;
  }
  log(eventObj: any) {
    logger.debug('Experiment', eventObj);
  }
  previouslyLogged() {
    return this._exposureLogged;
  }
}

The above class implements the logging methods which we will use to log our experiment runs. This will be common across all experiments.

The previouslyLogged method returns true if we have already logged this experiment earlier. This will make sure that we log an experiment only once even if our React component is re rendered.

Now that we have the BaseExperiment ready, let's go on to create the factory method.

Our experiment factory should return two things -

1. An object that holds all our experiments.
2. A method which we can use to create a new experiment.

Let’s declare the method

function experimentFactory(): [
  {
    [key: string]: (args: Inputs) => Experiment<Inputs, Params>
  },
  (
    name: string,
    assignments: {
      [key: string]: (arg: Inputs) => any
    }
  ) => void
] ()

As you can see, experimentFactory method returns an array of two items -

1. Object where key is the name of the experiment and the value is a method that returns the experiment depending on the arguments provided.
2. A method which accepts a name for the experiment and the assignments and creates an experiment.

Let’s define the return items for the experimentFactory functions

const experiments: { [key: string]: (args: Inputs) => BaseExperiment } = {};
  return [
    experiments,
    function createExperiment(name: string, assignments: { [key: string]: (arg: Inputs) => any}) {
      if (!name.startsWith('exp')) {
        throw new Error(`Experiments must start with exp but got ${name}`);
      }
      class FactoryExperiment extends BaseExperiment {
        setup() {
          this.setName(name);
        }
        getParamNames() {
          return Object.keys(assignments);
        }
        assign(params: Assignment<Inputs>, args: Inputs) {
          for (let [key, value] of Object.entries(assignments)) {
            params.set(key, value(args));
          }
        }
      }
      experiments[name] = (args: Inputs) => new FactoryExperiment(args);
    }
  ]

In the code above, we return an array where the first item are the experiments we have created.

The second item is a function that will create an experiment for us. We need to pass in the name of the experiment and the assignments.

The function will create the experiment and add it to the experiments object.

Ok now that we have the factory ready, let’s move on to creating the actual experiment. This last step is pretty straight forward thanks to all the hard work we’ve done so far.

Create a file called ExperimentList.ts. In this file we will list out all the experiments that we are running.

import { Inputs, Ops } from 'planout';
import experimentFactory from './experiment.factory';
const [experiments, createExperiment] = experimentFactory();
const {
  Random: { UniformChoice },
} = Ops;
// List of all experiments that are available
createExperiment('exp_success_hub_widget_text', {
  version: (arg: Inputs) => new UniformChoice({ choices: ['Success Hub', 'Lifeguard'], unit: arg.userId || arg.id }),
});
export default experiments;

We create an experiment which modifies the text of a widget and there’s a 50% chance of any of the two choices being shown to the user. We also pass the userId to the experiment to make sure that each user is consistently shown the same result.

And that’s it! We can now create experiments with just 3 lines of code.

Experiment component

Let’s see how we can use these experiments. We will create a new component which can be used as a wrapper around any component that we want to run the experiment on.

Create a file called Experiment.tsx. This will be our wrapper component. Let's look at its code.

import * as React from 'react';
import { Inputs } from 'planout';
import experiments from './ExperimentList';
interface IExperiment {
  name: string;
  args?: Inputs;
  children(result?: string): React.ReactElement | null;
}
const Experiment = ({ name, args, children }: IExperiment) => {
  const [result, setResult] = React.useState(null);
  React.useEffect(() => {
    if (!experiments[name]) {
      throw new Error(`Could not find any experiments with name ${name}`);
    }
    const exp = experiments[name](args).get('version', null);
    
    setResult(exp);
  }, [name]);
  return children(result);
};
export default Experiment;

The meat of the above code is this -

const exp = experiments[name](args).get('version', null);

Here we run the experiment and get the result. We then pass on this result to the child component for it to use as needed.

An example of this could look like this

<Experiment name='exp_success_hub_widget_text' args={{userId: user.id}}>
      {(result) => 
        <WidgetContextProvider>
          <div className={'success-hub-widget-ctn'}>
            <Widget title={result} />
          </div>
        </WidgetContextProvider>
      }
</Experiment>

And that’s it!! We have an experiment running for our widget component.

Tracking

Finally, we can send these results to a tracking service like Amplitude to analyse the results. This will help the product owners make informed decisions about product features.

It will help remove a lot of guesswork from product development.

Credit to John for the code snippets.

Follow me on Twitter for more coding, physics and random thoughts.