Quick Start

This section guides you through the process of getting Proctor and implementing a simple A/B test.

Get Proctor

Pulling from maven repository

Add these dependencies to your pom.xml:

    <dependencies>
        <!-- use this dependency only if your environment is providing tomcat libraries
        <dependency>
            <groupId>com.indeed</groupId>
            <artifactId>proctor-tomcat-deps-provided</artifactId>
            <version>1.0</version>
            <type>pom</type>
        </dependency>
        -->
        <dependency>
            <groupId>com.indeed</groupId>
            <artifactId>proctor-common</artifactId>
            <version>1.0</version>
        </dependency>
        <dependency>
            <groupId>com.indeed</groupId>
            <artifactId>proctor-consumer</artifactId>
            <version>1.0</version>
        </dependency>
        <dependency>
            <groupId>com.indeed</groupId>
            <artifactId>proctor-codegen</artifactId>
            <version>1.0</version>
        </dependency>
        <dependency>
            <groupId>com.indeed</groupId>
            <artifactId>proctor-maven-plugin</artifactId>
            <version>1.0</version>
        </dependency>
        <dependency>
            <groupId>com.indeed</groupId>
            <artifactId>util-core</artifactId>
            <version>1.0</version>
        </dependency>
    </dependencies>

Building from source (using maven)

Use git to clone https://github.com/indeedeng/proctor, and run mvn install to build.

Implement an A/B test with Proctor

For this process, we’ll use an example similar to the https://github.com/indeedeng/proctor-demo reference implementation project. You’ll run an A/B test of a new background color for your web application. The example uses maven to build and the Proctor maven plugin to generate Proctor convenience classes.

Specifying the test

  1. Write the JSON specification for your test and put it into a Java-like package structure under src/main/proctor/. For this example, name the package org.example.proctor and use ExampleGroups in generated Java class names.
  2. Create the file src/main/proctor/org/example/proctor/ExampleGroups.json with this content:
{
"tests" : {
"bgcolortst": {
"buckets":{
"inactive":-1,
"altcolor1":0,
"altcolor2":1,
"altcolor3":2,
"altcolor4":3
},
"fallbackValue": -1
}
}
}

The result is a single test bgcolortst with five buckets: inactive, altcolor1, altcolor2, altcolor3, altcolor4.

Setting up the maven plugin and generate code

  1. Edit your pom.xml to enable the Proctor maven plugin:

     <build>
         <plugins>
           <!-- add this plugin definition -->
           <plugin>
             <groupId>com.indeed</groupId>
             <artifactId>proctor-maven-plugin</artifactId>
             <version>1.0</version>
             <executions>
               <execution>
                 <id>proctor-generate</id>
                 <goals>
                   <goal>generate</goal>
                 </goals>
               </execution>
             </executions>
           </plugin>
         </plugins>
     </build>
    
  2. To generate the Proctor convenience classes, run mvn proctor:generate, which creates ExampleGroups.java and ExampleGroupsManager.java in target/generated-sources/proctor/org/example/proctor/.

Creating your initial test definition

In the test specification you created, you set up 4 test buckets and one inactive bucket. For this example, you’ll put 10% of your users in each test bucket. Leaving space between each bucket allows you to increase them later without moving users between buckets. Your definition would be:

altcolor1 inactive altcolor2 inactive altcolor3 inactive altcolor4
10% 20% 10% 20% 10% 20% 10%
  1. Create a file called proctor-definition.json with this content:

    {
    "tests" : {
    "bgcolortst" : {
    "version" : 1,
    "description" : "background color test",
    "salt" : "bgcolortst",
    "buckets" : [ {
    "name" : "inactive",
    "value" : -1,
    "description" : "",
    }, {
    "name" : "altColor1",
    "value" : 0,
    "description" : "test color 1",
    }, {
    "name" : "altColor2",
    "value" : 1,
    "description" : "test color 2",
    }, {
    "name" : "altColor3",
    "value" : 2,
    "description" : "test color 3",
    }, {
    "name" : "altColor4",
    "value" : 3,
    "description" : "test color 4",
    }],
    "allocations" : [ {
    "rule" : null,
    "ranges" : [ {
    "length" : 0.1,
    "bucketValue" : 0
    }, {
    "length" : 0.2,
    "bucketValue" : -1
    }, {
    "length" : 0.1,
    "bucketValue" : 1
    }, {
    "length" : 0.2,
    "bucketValue" : -1
    }, {
    "length" : 0.1,
    "bucketValue" : 2
    }, {
    "length" : 0.2,
    "bucketValue" : -1
    }, {
    "length" : 0.1,
    "bucketValue" : 3
    } ]
    } ],
    "testType" : "USER"
    }
    },
    "audit" : {
    "version" : 1,
    "updatedBy" : "jack",
    "updated" : 1379048400000
    }
    }
  2. Load this definition from the file system as your test matrix.

Writing some code

  1. Load your specification (from the classpath) and use it to load your test matrix.

    import com.indeed.proctor.common.*;
    import com.indeed.proctor.common.model.*;
    import javax.el.FunctionMapper;
    ...
    public Proctor loadProctor(final String definitionFile) throws MissingTestMatrixException, IOException {
    final String specPath = "/org/example/proctor/ExampleGroups.json";
    final ProctorSpecification spec = ProctorUtils.readSpecification(getClass().getResourceAsStream(specPath));
    final FunctionMapper functionMapper = RuleEvaluator.defaultFunctionMapperBuilder().build();
    final FileProctorLoader loader = new FileProctorLoader(spec, definitionFile, functionMapper);
    final Proctor proctor = loader.doLoad();
    return proctor;
    }
    ...
  2. Use the Proctor object to get the generated convenience class objects. You’ll need to provide a user unique id, typically stored in a cookie for web applications, since this is a USER test.

    import com.google.common.base.Suppliers;
    import org.example.proctor.ExampleGroups;
    import org.example.proctor.ExampleGroupsManager;
    import com.indeed.proctor.common.*;
    import com.indeed.proctor.common.model.*;
    ...
    public ExampleGroups getTestGroups(final Proctor proctor, final String userId,
    final HttpServletRequest request, final HttpServletResponse response) {
    final ExampleGroupsManager groupsManager = new ExampleGroupsManager(Suppliers.ofInstance(proctor));
    final Identifiers identifiers = new Identifiers(TestType.USER, userId);
    final boolean allowForceGroups = true; // normally you'd only want this to be true in "privileged" conditions
    final ProctorResult result = groupsManager.determineBuckets(
    request, response, identifiers, allowForceGroups);
    final ExampleGroups groups = new ExampleGroups(result);
    return groups;
    }
    ...

You can now use the ExampleGroups object in your Java code or your view templates.

Java example

switch (groups.getBgcolortst()) {
case ALTCOLOR1:
model.setColor(color1);
break;
case ALTCOLOR2:
model.setColor(color2);
break;
case ALTCOLOR3:
model.setColor(color3);
break;
case ALTCOLOR4:
model.setColor(color4);
break;
}

JSP example

<c:if test="${groups.bgcolortstAltcolor1}">
html,body { background-color: #ff6600; color: #ffffff; }
</c:if>
<c:if test="${groups.bgcolortstAltcolor2}">
html,body { background-color: #cd29c0; color: #ffffff; }
</c:if>
<c:if test="${groups.bgcolortstAltcolor3}">
html,body { background-color: #2164f3; color: #ffffff; }
</c:if>
<c:if test="${groups.bgcolortstAltcolor4}">
html,body { background-color: #008040; color: #ffffff; }
</c:if>

Testing your groups

Test the different buckets by appending the query param prforceGroups. For example, ?prforceGroups=bgcolortst1 would temporarily put you into bucket 1 of the bgcolortst test, and would set a prforceGroups cookie to keep you in that group for the length of your browser session.