Reporting API

Conductrics provides reports on agent learning, hypothesis testing, and the like. Most people will access those reports via the web by logging in to our administrative/reporting pages with their username and password.

We also provide a Reporting API, documented here, which can be used to access the report data programatically, so that it can be displayed in a different form elsewhere, or used for some other purpose.

This is the same API that our web-based reporting pages use internally.

Providing Your API Key: You must provide a valid API key to any of the methods in our Reporting API. You can provide it as a URL param called 'apikey' as shown here, or you can pass it in a HTTP Header called 'x-mpath-apikey' if you want to keep it out of the query string.

Contents:

Agent Status Report Data
Option Confidence / Lift Report Data
Targeting Features Report Data
Decision Scorecard Report Data
Raw Learning Report Data
System Usage Report Data

High-level information about agents, suitable for at-a-glance widgets, etc.

To get the data shown in the Agent List portion of the Conductrics administrative pages, issue a GET to a URL like the following:

GET http://api.conductrics.com/undefined/report/status?codes=agent-1,agent-2&apikey=12345

The data returned is for the past 14 days by default. For a different timeframe, pass a number of days:

GET http://api.conductrics.com/undefined/report/status?codes=agent-1,agent-2&days=30&apikey=12345

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  data: {
    agent-1: {
      dates: [

        // one object like this for each date in the range
        {
          date: 1351555200, // seconds since 1970
          sessionCount: 12345, // number of sessions started
          goalCount: 1234, // number of goal events received
          liftOverDefault: 33.3, // expressed as a percentage
          liftOverRandom: 22.2 // expressed as a percentage
        }
      ],

      // same data as above for current date, as a convenience
      today: {
        date: 1351555200,
        sessionCount: 12345,
        goalCount: 1234,
        liftOverDefault: 33.3,
        liftOverRandom: 22.2
      }

      // same data as above, transposed for plotting charts, etc.
      trends: {
        dates: [1351555200, 1351641600, ...],
        sessionCount: [12345, 12345, ...],
        goalCount: [1234, 1234, ...],
        liftOverDefault: [33.3, 33.3, ...],
        liftOverRandom: [22.2, 22.2, ...]
      },

      totals: {
        goals: {
          count: 100
          dateFrom: 1351555200
          dateThru: 1351641600
        },
        sessions: {
          count: 1000
          dateFrom: 1351555200
          dateThru: 1351641600
        }
      }
    }
  }
}

Data about the estimated mean value of your agent's options, with confidence scores

To get the data shown in the 'Option Confidence' portion of the Conductrics reporting pages, issue a GET to a URL like the following:

GET http://api.conductrics.com/undefined/agent-1/report/confidence/2014-10-01?apikey=12345

The above URL requests data for October 1, 2014. You can also request data for a date range like October 1 through October 7, like so:

GET http://api.conductrics.com/undefined/agent-1/report/confidence/2014-10-01/2014-10-07?apikey=12345

About Dates:

Dates must be specified in YYYY-MM-DD form.

Providing one date is just shorthand for providing the same date for the 'start' and 'end' dates.

Providing the special string 'today' is shorthand for the current date.

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  agent: "agent-1",
  dateFrom: "2014-10-01",
  dateThru: "2014-10-07",

  data: {
    bLo: 0.01615007001786055,
    bHi: 0.21812769810311980,

    items: [

      // one item like this for each combination of date/choice/segment
      {
        owner: "undefined",
        agent: "agent-1",
        point: "point-1",

        date: "2014-10-01",
        choice: "decision-1:a",
        seg: "(none)",

        count: 29025,
        val: 434,
        vMean: 0.01713888406049018,
        bLo: 0.01615007001786055,
        bHi: 0.01812769810311981,
        confidence: 0.0189864628893,
        confidenceLevel: 1,
        vVari: 0.01712385378105195,
        stdDev: 0.13085814373225668,
        tScore: -2.0751947553348264,
        dgFree: 57446,
        pValue: 0.9810135371107048,
        signif: true
        ts: 1341522409717
      },

      // ... more items like the item above...
    ]
  }
}

Optional parameters

You can provide these optional parameters to this report. Each of these parameters can be provided as a query string parameter, or as a header with the prefix 'x-mpath-', such as 'x-mpath-confidence-measure'.

confidence-measure: An optional confidence measure to be used in the report calculations, from 0 to 1. The default value if not provided is 0.8. This main purpose of this measure is to change when the 'signif' flag on each item changes to true; if the pValue for a given item is greater than the confidence measure then signif will be true.
point: An optional 'decision point' code that you want report data for. Defaults to the first decision point in your agent. Many agents only have one point, so there is no need to pass this parameter unless you are working with a mulit-point agent.
comparison-decision: An optional 'comparison' code. When not provided, this parameter defaults to the 'default' option(s) at the decision point in question, which means that the report is conceptually reporting on each item's performance against the default option. Providing this parameter allows you to specify a different basis item that you want each item's performance compared to and reported on.
features: An optional 'feature' code (or codes, separated by commas). If your agent uses targeting features or auto-targeting, you can provide feature code(s) here to run the report for those features. If '(none)' or not provided, it is assumed that you want the report to be run for the population as a whole. If you want to run the report for all features, provide an empty string for this parameter.
segment: An optional 'segment' code. If your agent uses targeting segments, you can provide a segment code here to run the report for a particular segment. Otherwise, it is assumed that you want the report to be run for the default '(none)' segment.
use-bonferroni: Set this to 'false' to indicate that you do not want the Bonferroni Correction applied to the calculations. The default is 'true'.
ignore-goal-rewards: Set this to 'true' to indicate that you would like the report to ignore any explicit numeric values provided when goals occur (so, the system will consider the number of goals, but not the explicit values provided). The default is 'false', which means that any numeric rewards provided when goals occur are considered by the report. If your site or app doesn't provide explicit goal rewards, then this parameter has no effect.

Data item fields

Here is a brief description of the data fields you'll see for each item.

count: The number of times this combination was served (total, during the day in question).
val: The total reward value from goal events (total, during the day in question). Used internally to compute the other statistics. Typically not terribly important to end users in bare forms.
bLo: The lower bound for the estimate of this choice's value. Typically the lower and upper bounds are used to draw a horizontal bar that makes it easier to interpret the report data visually. The bar for each item is drawn from bLo to bHi, on a scale based on the overall bLo and bHi values (at the 'data' level).
bHi: The upper bound for the estimate of this choice's value (see bLo).
confidence: A confidence score that indicates the amount of evidence there is to support the value (val) prediction. The value will range from 0 (no evidence) to 1 (certainty, though in practice 1 is seldom achieved). The confidence score is currently computed as the inverse of the p-value, assuming that the options being compared have each been selected a reasonable number of times (if not, the confidence score remains at zero until they have). See also the confidenceMeasure value (below), which is a kind of best practice for how to interpret the confidence score.
confidenceLevel: A simple ranking from 0-5 based on the confidence score (above). Currently, level 5 means a confidence of .99 or better, level 4 means .95 or better, level 3 means .90 or better, level 2 means .80 or better, level 1 means .70 or better, and anything lower than .70 is level 0.
vMean: The estimated value of this choice, based on goal events attributed to the choice in question. Typically this value is displayed in the center of the bar for the lower/upper bounds. This value is the conceptual focus of the report for most users.
vVari: The statistical variance associated with the value of the goal events attributed to the choice in question.
stdDev: Standard deviation calculation, used internally to compute some of the other calculations (typically not important to end users).
tScore: Student's T calculation, used internally to compute some of the other calculations (typically not important to end users).
dgFree: Degrees of freedom, used internally to compute some of the other calculations (typically not important to end users).
pValue: Statistical P-Value, used internally to compute some of the other calculations. Conceptually important to end users, but more typically displayed by simply indicating whether the P-Value exceeds the basis confidence measure (see the 'signif' flag, below). Generally speaking the 'confidence' and 'confidenceLevel' values are preferred over the raw p-value.
signif: A Boolean flag that indicates whether there is evidence to support that the mean value has significance, based on the confidence measure in use. Will be true if the p-value is less than (1-confidence), where confidence is 0.8 (by default, or whatever confidence has been provided to the optional 'confidence-measure' parameter). See also the 'confidenceLevel' value, which may be easier to understand for many users.
ts: Timestamp of the last time this record was updated (typically not important to end users).

Statistics and insights about targeting features (or combinations of them)

To get the data shown in the 'Targeting Features' portion of the Conductrics reporting pages, issue a GET to a URL like the following:

GET http://api.conductrics.com/undefined/agent-1/report/targeting-features/today?apikey=12345

The above URL requests the report data for the current date only. It's generally more useful to request data for a date range like October 1 through October 7, like so:

GET http://api.conductrics.com/undefined/agent-1/report/targeting-features/2014-10-01/2014-10-07?apikey=12345

About Dates:

Dates must be specified in YYYY-MM-DD form.

Providing one date is just shorthand for providing the same date for the 'start' and 'end' dates.

Providing the special string 'today' is shorthand for the current date.

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  agent: "agent-1",
  dateFrom: "2014-10-01",
  dateThru: "2014-10-07",

  data: {
    items: [
   	{
	  "feature": "my-feature-code",
	  "label": "My Feature",
	  "isCombination": false,
	  "decisions": 4,
	  "percentTraffic": 0.5,
	  "averageResponseValue": 0.19503401705181259,
	  "decisionValue": 0.10468782272804486,
	  "stability": 7.612046748871322,
	  "stabilityLevel": 0
	}

	// ... and so on for each feature ...

	]
  }
}

Optional parameters

You can provide these optional parameters to this report. Each of these parameters can be provided as a query string parameter, or as a header with the prefix 'x-mpath-', such as 'x-mpath-point'.

point
feature-combinations
feature-combinations-top-n
feature-combinations-top-by
segment
stability-rate

Data item fields

Here is a brief description of the data fields you'll see for each item.

feature
label
isCombination
decisions
percentTraffic
averageResponseValue
decisionValue
stability
stabilityLevel

Distilled information about which options work best for whom

To get the data shown in the 'Decision Scorecard' portion of the Conductrics reporting pages, issue a GET to a URL like the following:

GET http://api.conductrics.com/undefined/agent-1/report/decision-scorecard/today/today?apikey=12345

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  "agent": "vh-36",
  "dateFrom": "2015-05-28",
  "dateThru": "2015-05-28",
  "data": {
    "choices": [{

      // One item like this for each choice
      "choice": "decision-1:experience-a", // decision/choice code
      "labelText": "Experience A", // friendly name, as provided in UI, etc

      // The features array highlights the visitor features  
      // (or combinations of features) that this choice is best for
      "features": [{
        "feature": "audience:metro+visit:first", // feature combination prefers this choice
        "labelText": "Metro", // friendly name
        "decisions": 15, // number of decisions (this will be a high number IRL)
        "value": 0, // predicted conversion value from this type of visitor, given this choice
        "percent": 0.6818181818181818, // percentage of visitors
        "stability": 100 // how stable is the data? (a confidence-type measure from 0 to 100)
      }, {
        "feature": "audience:rural", // this feature alone prefers this choice
        "labelText": "Rural",
        "decisions": 7,
        "value": 0,
        "percent": 0.3181818181818182,
        "stability": 100
      }],

      // always one item like this for the choice overall (for all visitors)
      "overall": {
        "decisions": 22, // total decisions during the date range
        "percent": 1, // what percentage of visitors prefer this choice?
        "avgValue": 0, // average conversion value per visitor
        "stability": 99.99999999999999
      }

    // ...remiaining choices follow same pattern
    // this choice happens to not be favored by any audiences
    }, {
      "choice": "decision-1:experience-b",
      "label": "Experience B",
      "labelText": "Experience B",
      "features": [], // just an example - IRL expect some feature combinations here
      "overall": {
        "decisions": 0,
        "percent": 0,
        "avgValue": 0,
        "stability": 0
      }
    }]
  }
}

Optional parameters

You can provide these optional parameters to this report. Each of these parameters can be provided as a query string parameter, or as a header with the prefix 'x-mpath-', such as 'x-mpath-point'.

point
segment

Raw data about the accumulated value of options

You can get raw information about the current internal score for each option (per targeting feature and segment, if any) via a URL like the following:

GET http://api.conductrics.com/undefined/agent-1/report/learning/2014-10-01?apikey=12345

The above URL requests data for October 1, 2014. You can also request data for a date range like October 1 through October 7, like so:

GET http://api.conductrics.com/undefined/agent-1/report/learning/2014-10-01/2014-10-07?apikey=12345

About This Data: This raw data is not displayed in direct form in of our web-based report pages, because it is not necessarily illuminating for most people without further analysis and normalization to account for variance in scales and so on. While this API call is provided and supported for certain specialized integration use cases, do not recommend using it unless you are well-versed in statistics, or have gotten further instruction from us as to how to interpret the data and/or present it usefully to your own users.

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  agent: "agent-1",
  dateFrom: "2014-10-01",
  dateThru: "2014-07-04",
  data: [

    // One record like this for each choice at each point, for each
    // targeting segment and feature (default "(none)" segment/feature shown here)
    {
      date: "2014-07-01",
      agent: "agent-103",
      owner: "acme",
      // targeting segment and feature
      seg: "(none)",
      feature: "(none)",
      // option selection
      point: "point-1",
      choice: "decision-1:a",
      // actual data
      count: 0, // number of times option was selected (rolling, accumulated total)
      val: 0 // internal score for the option, based on goals achieved after selection
             // NOT NECESSARILITY USEFUL without further analysis/normalization
    }

    ...
  ]
}

Optional parameters

You can provide this optional parameter to this report. A parameter can be provided as a query string parameter, or as a header with the prefix 'x-mpath-', such as 'x-mpath-point'.

point: An optional 'decision point' code that you want report data for. Defaults to the first decision point in your agent. Many agents only have one point, so there is no need to pass this parameter unless you are working with a mulit-point agent.

Counts and other basic stats about your account's usage of the system

You can get system usage information about your account via a URL like the following:

GET http://api.conductrics.com/undefined/-/report/system-usage/2014-10-01?apikey=12345

The above URL requests data for October 1, 2014. You can also request data for a date range like October 1 through October 7, like so:

GET http://api.conductrics.com/undefined/agent-1/report/system-usage/2014-10-01/2014-10-07?apikey=12345

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  "data": [
    {
      // some basic information about sessions during the time period
      "sessions": 19, // # of sessions started (see important note below)
      "sessionsFromAgents": 1, // # of agents the "sessions" count came from
      // counts of authorized api calls received
      "calls": {
        "decisions": 2000, // number of decision API calls
        "goals": 200,      // number of goal API calls
        "expires": 50,     // number of explicit session expiration calls
        "webactions": 0,   // number of calls to the web-actions system
        "other": 10        // all other authorized API calls (reports, admin, etc)
      }
    }
  ]
}

Please Note The 'sessions' count in the above data structure is a total of the sessions started for each agent during the time period. If the same session id is passed to ten different agents, that session id is counted 10 times in the 'sessions' count. You can get a per-agent breakdown from the 'Agent Status' report data (see top of this document).

Reporting API

Agent Status Report Data

Option Confidence / Lift Report Data

Optional parameters

Data item fields

Targeting Features Report Data

Optional parameters

Data item fields

Decision Scorecard Report Data

Optional parameters

Raw Learning Report Data

Optional parameters

System Usage Report Data