Quick Start
This section guides you through the process of getting Proctor and implementing a simple A/B test.
Get Proctor
Pulling from maven repository
Add these dependencies to your pom.xml:
<dependencies>
<!-- use this dependency only if your environment is providing tomcat libraries
<dependency>
<groupId>com.indeed</groupId>
<artifactId>proctor-tomcat-deps-provided</artifactId>
<version>1.0</version>
<type>pom</type>
</dependency>
-->
<dependency>
<groupId>com.indeed</groupId>
<artifactId>proctor-common</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.indeed</groupId>
<artifactId>proctor-consumer</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.indeed</groupId>
<artifactId>proctor-codegen</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.indeed</groupId>
<artifactId>proctor-maven-plugin</artifactId>
<version>1.0</version>
</dependency>
<dependency>
<groupId>com.indeed</groupId>
<artifactId>util-core</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
Building from source (using maven)
Use git to clone https://github.com/indeedeng/proctor, and run mvn install
to build.
Implement an A/B test with Proctor
For this process, we’ll use an example similar to the https://github.com/indeedeng/proctor-demo reference implementation project. You’ll run an A/B test of a new background color for your web application. The example uses maven to build and the Proctor maven plugin to generate Proctor convenience classes.
Specifying the test
- Write the JSON specification for your test and put it into a Java-like package structure under
src/main/proctor/
. For this example, name the packageorg.example.proctor
and useExampleGroups
in generated Java class names. - Create the file
src/main/proctor/org/example/proctor/ExampleGroups.json
with this content:
{ | |
"tests" : { | |
"bgcolortst": { | |
"buckets":{ | |
"inactive":-1, | |
"altcolor1":0, | |
"altcolor2":1, | |
"altcolor3":2, | |
"altcolor4":3 | |
}, | |
"fallbackValue": -1 | |
} | |
} | |
} |
The result is a single test bgcolortst
with five buckets: inactive, altcolor1, altcolor2, altcolor3, altcolor4
.
Setting up the maven plugin and generate code
-
Edit your
pom.xml
to enable the Proctor maven plugin:<build> <plugins> <!-- add this plugin definition --> <plugin> <groupId>com.indeed</groupId> <artifactId>proctor-maven-plugin</artifactId> <version>1.0</version> <executions> <execution> <id>proctor-generate</id> <goals> <goal>generate</goal> </goals> </execution> </executions> </plugin> </plugins> </build>
-
To generate the Proctor convenience classes, run
mvn proctor:generate
, which createsExampleGroups.java
andExampleGroupsManager.java
intarget/generated-sources/proctor/org/example/proctor/
.
Creating your initial test definition
In the test specification you created, you set up 4 test buckets and one inactive bucket. For this example, you’ll put 10% of your users in each test bucket. Leaving space between each bucket allows you to increase them later without moving users between buckets. Your definition would be:
altcolor1 | inactive | altcolor2 | inactive | altcolor3 | inactive | altcolor4 |
---|---|---|---|---|---|---|
10% | 20% | 10% | 20% | 10% | 20% | 10% |
-
Create a file called
proctor-definition.json
with this content:This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters{ "tests" : { "bgcolortst" : { "version" : 1, "description" : "background color test", "salt" : "bgcolortst", "buckets" : [ { "name" : "inactive", "value" : -1, "description" : "", }, { "name" : "altColor1", "value" : 0, "description" : "test color 1", }, { "name" : "altColor2", "value" : 1, "description" : "test color 2", }, { "name" : "altColor3", "value" : 2, "description" : "test color 3", }, { "name" : "altColor4", "value" : 3, "description" : "test color 4", }], "allocations" : [ { "rule" : null, "ranges" : [ { "length" : 0.1, "bucketValue" : 0 }, { "length" : 0.2, "bucketValue" : -1 }, { "length" : 0.1, "bucketValue" : 1 }, { "length" : 0.2, "bucketValue" : -1 }, { "length" : 0.1, "bucketValue" : 2 }, { "length" : 0.2, "bucketValue" : -1 }, { "length" : 0.1, "bucketValue" : 3 } ] } ], "testType" : "USER" } }, "audit" : { "version" : 1, "updatedBy" : "jack", "updated" : 1379048400000 } } -
Load this definition from the file system as your test matrix.
Writing some code
-
Load your specification (from the classpath) and use it to load your test matrix.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersimport com.indeed.proctor.common.*; import com.indeed.proctor.common.model.*; import javax.el.FunctionMapper; ... public Proctor loadProctor(final String definitionFile) throws MissingTestMatrixException, IOException { final String specPath = "/org/example/proctor/ExampleGroups.json"; final ProctorSpecification spec = ProctorUtils.readSpecification(getClass().getResourceAsStream(specPath)); final FunctionMapper functionMapper = RuleEvaluator.defaultFunctionMapperBuilder().build(); final FileProctorLoader loader = new FileProctorLoader(spec, definitionFile, functionMapper); final Proctor proctor = loader.doLoad(); return proctor; } ... -
Use the Proctor object to get the generated convenience class objects. You’ll need to provide a user unique id, typically stored in a cookie for web applications, since this is a
USER
test.This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode charactersimport com.google.common.base.Suppliers; import org.example.proctor.ExampleGroups; import org.example.proctor.ExampleGroupsManager; import com.indeed.proctor.common.*; import com.indeed.proctor.common.model.*; ... public ExampleGroups getTestGroups(final Proctor proctor, final String userId, final HttpServletRequest request, final HttpServletResponse response) { final ExampleGroupsManager groupsManager = new ExampleGroupsManager(Suppliers.ofInstance(proctor)); final Identifiers identifiers = new Identifiers(TestType.USER, userId); final boolean allowForceGroups = true; // normally you'd only want this to be true in "privileged" conditions final ProctorResult result = groupsManager.determineBuckets( request, response, identifiers, allowForceGroups); final ExampleGroups groups = new ExampleGroups(result); return groups; } ...
You can now use the ExampleGroups object in your Java code or your view templates.
Java example
switch (groups.getBgcolortst()) { | |
case ALTCOLOR1: | |
model.setColor(color1); | |
break; | |
case ALTCOLOR2: | |
model.setColor(color2); | |
break; | |
case ALTCOLOR3: | |
model.setColor(color3); | |
break; | |
case ALTCOLOR4: | |
model.setColor(color4); | |
break; | |
} |
JSP example
<c:if test="${groups.bgcolortstAltcolor1}"> | |
html,body { background-color: #ff6600; color: #ffffff; } | |
</c:if> | |
<c:if test="${groups.bgcolortstAltcolor2}"> | |
html,body { background-color: #cd29c0; color: #ffffff; } | |
</c:if> | |
<c:if test="${groups.bgcolortstAltcolor3}"> | |
html,body { background-color: #2164f3; color: #ffffff; } | |
</c:if> | |
<c:if test="${groups.bgcolortstAltcolor4}"> | |
html,body { background-color: #008040; color: #ffffff; } | |
</c:if> |
Testing your groups
Test the different buckets by appending the query param prforceGroups
. For example, ?prforceGroups=bgcolortst1
would temporarily put you into bucket 1 of the bgcolortst
test, and would set a prforceGroups
cookie to keep you in that group for the length of your browser session.