Online Controlled Experiments
Platform-as-a-Service

Variant is a ground-breaking, performant server for instrumentation and management of A/B tests and feature flags with unmatched capabilities and performance. Architected as client/server, Variant’s runtime executes on its own system, exerting no overhead on the host application, save for very small local network latency.

Variant enables complete decoupling of experience implementation from its instrumentation as an experiment or feature flag. The implementation of new application code paths remains within the host application’s domain, while their instrumentation for co-existence with the existing code-path in the form of an A/B test or a feature flag. There’s no instrumentation code smell as it takes as many lines of code (about a dozen) to instrument your 100th concurrent experiment as it took you to instrument your first.

Read the White Paper

Quick Example

To instrument a simple A/B test, you may simply take advantage of the example schema, shipped with the server in the file schemata/example.yaml, and reproduced below.

name: example
description: 'Basic sample variation schema. See User Guide for details'
states:
  - name: MyState
variations:
  - name: MyExperiment
    experiences:
      - name: existing
        isControl: true
      - name: treatment
    onStates:
      - state: MyState

Start Variant server:

$ /path/to/variant/server/bin/variant start
2019-12-26 15:23:26,412 INFO - c.v.s.boot.ConfigLoader$ - Found  config resource [/variant.conf] as [/Users/Igor/soft/variant-server-0.10.3/conf/variant.conf]
2019-12-26 15:23:27,425 INFO - c.v.s.schema.SchemaDeployerFileSystem - Mounted schemata directory [/Users/Igor/soft/variant-server-0.10.3/schemata]
2019-12-26 15:23:27,428 INFO - c.v.s.schema.SchemaDeployerFileSystem - [421] Deploying schema from file [/Users/Igor/soft/variant-server-0.10.3/schemata/example.yaml]
2019-12-26 15:23:27,689 INFO - c.v.s.schema.Schemata - [422] Deployed schema [example] from file [example.yaml]
2019-12-26 15:23:27,695 INFO - c.v.s.boot.VariantServerImpl - [433] Variant CVM Server release 0.10.3 started on port [5377] in 2.63s

In your JVM application, create a Variant client. You only need one per your application. Then connect to the above variation schema on the server. When user navigates to the state of interest MyState (may be an HTML page, or a React view, or an IVR menu, etc), target current session for that state and obtain the state request object, containing the experience targeting information you need to decide on the code path.

// Create Variant client.
VariantClient client = VariantClient.build(
   builder -> builder.withSessionIdTracker(MySessionIdTracker.class) 
)
// Connect to the Example schema on the server
Connection connection = client.connectTo("http://localhost:5377/example");
// Create Variant session
Session session = connection.getOrCreateSession(myParameters);
// The schema we're connected to
Schema schema = session.getSchema();
// The variable state, e.g. an HTML page, we want to target the session for.
State state = schema.getState("MySate").get();
// Target the session for the state.
StateRequest request = session.targetForState(state);
// Find out what experience in the A/B test we got.
Variation variation = schema.getVariation("MyExperiment").get();
Experience experience = request.getLiveExperience(variation).get();
// Bifurcate
if (experience.getName().equals("control")) {
  // the control code path.
{
else {
  // the treatment code path
}
request.commit();

Note, that for simplicity we’ve skipped all the error handling, e.g. what happens if the Variant server is unavailable on line 6, or the experiment MyExperiment is not online and the getVariation() call returns an empty Optional on line 16. Typically, the right thing to do in such cases is to show the control experience, which should always be safe.

Note as well, that each state request (line 24) be either committed or failed. This allows the downstream to easily throw away user sessions that had a server error during an experiment.