Reporting API

Conductrics provides reports on agent learning, hypothesis testing, and the like. Most people will access those reports via the web by logging in to our administrative/reporting pages with their username and password.

We also provide a Reporting API, documented here, which can be used to access the report data programatically, so that it can be displayed in a different form elsewhere, or used for some other purpose.

This is the same API that our web-based reporting pages use internally.

Providing Your API Key: You must provide a valid API key to any of the methods in our Reporting API. You can provide it as a URL param called 'apikey' as shown here, or you can pass it in a HTTP Header called 'x-mpath-apikey' if you want to keep it out of the query string.

Contents:

Agent Status Report Data

High-level information about agents, suitable for at-a-glance widgets, etc.

To get the data shown in the Agent List portion of the Conductrics administrative pages, issue a GET to a URL like the following:

GET http://api.conductrics.com/demo/report/status?codes=agent-1,agent-2&apikey=12345

The data returned is for the past 14 days by default. For a different timeframe, pass a number of days:

GET http://api.conductrics.com/demo/report/status?codes=agent-1,agent-2&days=30&apikey=12345

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  data: {
    agent-1: {
      dates: [

        // one object like this for each date in the range
        {
          date: 1351555200, // seconds since 1970
          sessionCount: 12345, // number of sessions started
          goalCount: 1234, // number of goal events received
          liftOverDefault: 33.3, // expressed as a percentage
          liftOverRandom: 22.2 // expressed as a percentage
        }
      ],

      // same data as above for current date, as a convenience
      today: {
        date: 1351555200,
        sessionCount: 12345,
        goalCount: 1234,
        liftOverDefault: 33.3,
        liftOverRandom: 22.2
      }

      // same data as above, transposed for plotting charts, etc.
      trends: {
        dates: [1351555200, 1351641600, ...],
        sessionCount: [12345, 12345, ...],
        goalCount: [1234, 1234, ...],
        liftOverDefault: [33.3, 33.3, ...],
        liftOverRandom: [22.2, 22.2, ...]
      },

      totals: {
        goals: {
          count: 100
          dateFrom: 1351555200
          dateThru: 1351641600
        },
        sessions: {
          count: 1000
          dateFrom: 1351555200
          dateThru: 1351641600
        }
      }
    }
  }
}

Option Confidence / Lift Report Data

Data about the estimated mean value of your agent's options, with confidence scores

To get the data shown in the 'Option Confidence' portion of the Conductrics reporting pages, issue a GET to a URL like the following:

GET http://api.conductrics.com/demo/agent-1/report/confidence/2014-10-01?apikey=12345

The above URL requests data for October 1, 2014. You can also request data for a date range like October 1 through October 7, like so:

GET http://api.conductrics.com/demo/agent-1/report/confidence/2014-10-01/2014-10-07?apikey=12345
About Dates:

Dates must be specified in YYYY-MM-DD form.

Providing one date is just shorthand for providing the same date for the 'start' and 'end' dates.

Providing the special string 'today' is shorthand for the current date.

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  agent: "agent-1",
  dateFrom: "2014-10-01",
  dateThru: "2014-10-07",

  data: {
    bLo: 0.01615007001786055,
    bHi: 0.21812769810311980,

    items: [

      // one item like this for each combination of date/choice/segment
      {
        owner: "demo",
        agent: "agent-1",
        point: "point-1",

        date: "2014-10-01",
        choice: "decision-1:a",
        seg: "(none)",

        count: 29025,
        val: 434,
        vMean: 0.01713888406049018,
        bLo: 0.01615007001786055,
        bHi: 0.01812769810311981,
        confidence: 0.0189864628893,
        confidenceLevel: 1,
        vVari: 0.01712385378105195,
        stdDev: 0.13085814373225668,
        tScore: -2.0751947553348264,
        dgFree: 57446,
        pValue: 0.9810135371107048,
        signif: true
        ts: 1341522409717
      },

      // ... more items like the item above...
    ]
  }
}

Optional parameters

You can provide these optional parameters to this report. Each of these parameters can be provided as a query string parameter, or as a header with the prefix 'x-mpath-', such as 'x-mpath-confidence-measure'.

confidence-measure
An optional confidence measure to be used in the report calculations, from 0 to 1. The default value if not provided is 0.8. This main purpose of this measure is to change when the 'signif' flag on each item changes to true; if the pValue for a given item is greater than the confidence measure then signif will be true.
point
An optional 'decision point' code that you want report data for. Defaults to the first decision point in your agent. Many agents only have one point, so there is no need to pass this parameter unless you are working with a mulit-point agent.
comparison-decision
An optional 'comparison' code. When not provided, this parameter defaults to the 'default' option(s) at the decision point in question, which means that the report is conceptually reporting on each item's performance against the default option. Providing this parameter allows you to specify a different basis item that you want each item's performance compared to and reported on.
features
An optional 'feature' code (or codes, separated by commas). If your agent uses targeting features or auto-targeting, you can provide feature code(s) here to run the report for those features. If '(none)' or not provided, it is assumed that you want the report to be run for the population as a whole. If you want to run the report for all features, provide an empty string for this parameter.
segment
An optional 'segment' code. If your agent uses targeting segments, you can provide a segment code here to run the report for a particular segment. Otherwise, it is assumed that you want the report to be run for the default '(none)' segment.
use-bonferroni
Set this to 'false' to indicate that you do not want the Bonferroni Correction applied to the calculations. The default is 'true'.
ignore-goal-rewards
Set this to 'true' to indicate that you would like the report to ignore any explicit numeric values provided when goals occur (so, the system will consider the number of goals, but not the explicit values provided). The default is 'false', which means that any numeric rewards provided when goals occur are considered by the report. If your site or app doesn't provide explicit goal rewards, then this parameter has no effect.

Data item fields

Here is a brief description of the data fields you'll see for each item.

count
The number of times this combination was served (total, during the day in question).
val
The total reward value from goal events (total, during the day in question). Used internally to compute the other statistics. Typically not terribly important to end users in bare forms.
bLo
The lower bound for the estimate of this choice's value. Typically the lower and upper bounds are used to draw a horizontal bar that makes it easier to interpret the report data visually. The bar for each item is drawn from bLo to bHi, on a scale based on the overall bLo and bHi values (at the 'data' level).
bHi
The upper bound for the estimate of this choice's value (see bLo).
confidence
A confidence score that indicates the amount of evidence there is to support the value (val) prediction. The value will range from 0 (no evidence) to 1 (certainty, though in practice 1 is seldom achieved). The confidence score is currently computed as the inverse of the p-value, assuming that the options being compared have each been selected a reasonable number of times (if not, the confidence score remains at zero until they have). See also the confidenceMeasure value (below), which is a kind of best practice for how to interpret the confidence score.
confidenceLevel
A simple ranking from 0-5 based on the confidence score (above). Currently, level 5 means a confidence of .99 or better, level 4 means .95 or better, level 3 means .90 or better, level 2 means .80 or better, level 1 means .70 or better, and anything lower than .70 is level 0.
vMean
The estimated value of this choice, based on goal events attributed to the choice in question. Typically this value is displayed in the center of the bar for the lower/upper bounds. This value is the conceptual focus of the report for most users.
vVari
The statistical variance associated with the value of the goal events attributed to the choice in question.
stdDev
Standard deviation calculation, used internally to compute some of the other calculations (typically not important to end users).
tScore
Student's T calculation, used internally to compute some of the other calculations (typically not important to end users).
dgFree
Degrees of freedom, used internally to compute some of the other calculations (typically not important to end users).
pValue
Statistical P-Value, used internally to compute some of the other calculations. Conceptually important to end users, but more typically displayed by simply indicating whether the P-Value exceeds the basis confidence measure (see the 'signif' flag, below). Generally speaking the 'confidence' and 'confidenceLevel' values are preferred over the raw p-value.
signif
A Boolean flag that indicates whether there is evidence to support that the mean value has significance, based on the confidence measure in use. Will be true if the p-value is less than (1-confidence), where confidence is 0.8 (by default, or whatever confidence has been provided to the optional 'confidence-measure' parameter). See also the 'confidenceLevel' value, which may be easier to understand for many users.
ts
Timestamp of the last time this record was updated (typically not important to end users).

Targeting Features Report Data

Statistics and insights about targeting features (or combinations of them)

To get the data shown in the 'Targeting Features' portion of the Conductrics reporting pages, issue a GET to a URL like the following:

GET http://api.conductrics.com/demo/agent-1/report/targeting-features/today?apikey=12345

The above URL requests the report data for the current date only. It's generally more useful to request data for a date range like October 1 through October 7, like so:

GET http://api.conductrics.com/demo/agent-1/report/targeting-features/2014-10-01/2014-10-07?apikey=12345
About Dates:

Dates must be specified in YYYY-MM-DD form.

Providing one date is just shorthand for providing the same date for the 'start' and 'end' dates.

Providing the special string 'today' is shorthand for the current date.

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  agent: "agent-1",
  dateFrom: "2014-10-01",
  dateThru: "2014-10-07",

  data: {
    items: [
   	{
	  "feature": "my-feature-code",
	  "label": "My Feature",
	  "isCombination": false,
	  "decisions": 4,
	  "percentTraffic": 0.5,
	  "averageResponseValue": 0.19503401705181259,
	  "decisionValue": 0.10468782272804486,
	  "stability": 7.612046748871322,
	  "stabilityLevel": 0
	}

	// ... and so on for each feature ...

	]
  }
}

Optional parameters

You can provide these optional parameters to this report. Each of these parameters can be provided as a query string parameter, or as a header with the prefix 'x-mpath-', such as 'x-mpath-point'.

point

An optional 'decision point' code that you want report data for. Defaults to the first decision point in your agent. Many agents only have one point, so there is no need to pass this parameter unless you are working with a mulit-point agent.

feature-combinations

An optional set of one or more feature combinations to include in the returned report data. Indicate features to combine with the + sign, and separate combinations with commas. For example, vip+primetime,vip+afterhours+frequent would indicate two combinations of features (the first combination being the combination of fictional vip and primetime features, and the second combination being a combination of vip and afterhours and frequent).

feature-combinations-top-n

An optional number of feature combinations that you would like the report to automatically 'find' and report on. For instance, setting this parameter to 10 will retrieve the top 10 feature combinations. The Conductrics Console UI calls these 'Audiences'.

feature-combinations-top-by

If feature-combinations-top-n is provided, an optional field to 'find' the top-n feature combinations by. Accepted values are decisions (e.g., Top 10 feature combinations by the most number of decisions; could be labeled 'traffic' or 'decision traffic'), goals (e.g., Top 10 by most goal events), or val (e.g., Top 10 by most goal value; equivalent to 'goals' unless numeric goal values are provided to the agent).

segment

An optional targeting segment code that you want report data for. If not provided, the report data will be for the default (none) segment. Many agents don't use segments, so there is no need to pass this parameter unless you are specifically using them.

stability-rate

An optional factor to provide to the stability portion of the report calculations. We don't anticipate that you will have cause to provide a different stability rate; please contact us for advice if you think you need to.

Data item fields

Here is a brief description of the data fields you'll see for each item.

feature

The feature code that this item represents. It might be a feature code that you passed into the decision API, or a feature applied by Conductrics Targeting.

label

The friendly name for the feature, if there is one (if not, the label defaults to the feature code).

isCombination

Will be true for synthetic items generated via the 'feature-combinations' parameter. Otherwise will be false for normal features. You might use this to render a special icon for feature combinations, etc.

decisions

The total number of decisions issued to visitors with the feature.

percentTraffic

The percentage of overall decision 'traffic' that this feature represents over the date range.

averageResponseValue

The current reward prediction for visitors with this feature, based on rewards achieved.

decisionValue

A metric about the overall 'importance' of decisions made for visitors with this feature (how important is this feature to the agent's results overall?).

stability

A metric about the overall stability of the results from visitors with this feature (similar conceptually to a 'statistical confidence'). Higher numbers indicate greater stability (less variability); lower numbers indicate more inconsitency amongst responses from visitors with this feature.

stabilityLevel

A simple categorization of the stability metric. Values range from 0 to 6, where 6 means very stable. This value can be used for color-coding or similar visual presentation of the report data.

Decision Scorecard Report Data

Distilled information about which options work best for whom

To get the data shown in the 'Decision Scorecard' portion of the Conductrics reporting pages, issue a GET to a URL like the following:

GET http://api.conductrics.com/demo/agent-1/report/decision-scorecard/today/today?apikey=12345

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  "agent": "vh-36",
  "dateFrom": "2015-05-28",
  "dateThru": "2015-05-28",
  "data": {
    "choices": [{

      // One item like this for each choice
      "choice": "decision-1:experience-a", // decision/choice code
      "labelText": "Experience A", // friendly name, as provided in UI, etc

      // The features array highlights the visitor features  
      // (or combinations of features) that this choice is best for
      "features": [{
        "feature": "audience:metro+visit:first", // feature combination prefers this choice
        "labelText": "Metro", // friendly name
        "decisions": 15, // number of decisions (this will be a high number IRL)
        "value": 0, // predicted conversion value from this type of visitor, given this choice
        "percent": 0.6818181818181818, // percentage of visitors
        "stability": 100 // how stable is the data? (a confidence-type measure from 0 to 100)
      }, {
        "feature": "audience:rural", // this feature alone prefers this choice
        "labelText": "Rural",
        "decisions": 7,
        "value": 0,
        "percent": 0.3181818181818182,
        "stability": 100
      }],

      // always one item like this for the choice overall (for all visitors)
      "overall": {
        "decisions": 22, // total decisions during the date range
        "percent": 1, // what percentage of visitors prefer this choice?
        "avgValue": 0, // average conversion value per visitor
        "stability": 99.99999999999999
      }

    // ...remiaining choices follow same pattern
    // this choice happens to not be favored by any audiences
    }, {
      "choice": "decision-1:experience-b",
      "label": "Experience B",
      "labelText": "Experience B",
      "features": [], // just an example - IRL expect some feature combinations here
      "overall": {
        "decisions": 0,
        "percent": 0,
        "avgValue": 0,
        "stability": 0
      }
    }]
  }
}

Optional parameters

You can provide these optional parameters to this report. Each of these parameters can be provided as a query string parameter, or as a header with the prefix 'x-mpath-', such as 'x-mpath-point'.

point

An optional 'decision point' code that you want report data for. Defaults to the first decision point in your agent. Many agents only have one point, so there is no need to pass this parameter unless you are working with a mulit-point agent.

segment

An optional targeting segment code that you want report data for. If not provided, the report data will be for the default (none) segment. Many agents don't use segments, so there is no need to pass this parameter unless you are specifically using them.

Raw Learning Report Data

Raw data about the accumulated value of options

You can get raw information about the current internal score for each option (per targeting feature and segment, if any) via a URL like the following:

GET http://api.conductrics.com/demo/agent-1/report/learning/2014-10-01?apikey=12345

The above URL requests data for October 1, 2014. You can also request data for a date range like October 1 through October 7, like so:

GET http://api.conductrics.com/demo/agent-1/report/learning/2014-10-01/2014-10-07?apikey=12345
About This Data: This raw data is not displayed in direct form in of our web-based report pages, because it is not necessarily illuminating for most people without further analysis and normalization to account for variance in scales and so on. While this API call is provided and supported for certain specialized integration use cases, do not recommend using it unless you are well-versed in statistics, or have gotten further instruction from us as to how to interpret the data and/or present it usefully to your own users.

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  agent: "agent-1",
  dateFrom: "2014-10-01",
  dateThru: "2014-07-04",
  data: [

    // One record like this for each choice at each point, for each
    // targeting segment and feature (default "(none)" segment/feature shown here)
    {
      date: "2014-07-01",
      agent: "agent-103",
      owner: "acme",
      // targeting segment and feature
      seg: "(none)",
      feature: "(none)",
      // option selection
      point: "point-1",
      choice: "decision-1:a",
      // actual data
      count: 0, // number of times option was selected (rolling, accumulated total)
      val: 0 // internal score for the option, based on goals achieved after selection
             // NOT NECESSARILITY USEFUL without further analysis/normalization
    }

    ...
  ]
}

Optional parameters

You can provide this optional parameter to this report. A parameter can be provided as a query string parameter, or as a header with the prefix 'x-mpath-', such as 'x-mpath-point'.

point
An optional 'decision point' code that you want report data for. Defaults to the first decision point in your agent. Many agents only have one point, so there is no need to pass this parameter unless you are working with a mulit-point agent.

System Usage Report Data

Counts and other basic stats about your account's usage of the system

You can get system usage information about your account via a URL like the following:

GET http://api.conductrics.com/demo/-/report/system-usage/2014-10-01?apikey=12345

The above URL requests data for October 1, 2014. You can also request data for a date range like October 1 through October 7, like so:

GET http://api.conductrics.com/demo/agent-1/report/system-usage/2014-10-01/2014-10-07?apikey=12345

Assuming your API key has the appropriate access, the server will respond with a JSON data structure like this (comments and newlines added for clarity):

{
  "data": [
    {
      // some basic information about sessions during the time period
      "sessions": 19, // # of sessions started (see important note below)
      "sessionsFromAgents": 1, // # of agents the "sessions" count came from
      // counts of authorized api calls received
      "calls": {
        "decisions": 2000, // number of decision API calls
        "goals": 200,      // number of goal API calls
        "expires": 50,     // number of explicit session expiration calls
        "webactions": 0,   // number of calls to the web-actions system
        "other": 10        // all other authorized API calls (reports, admin, etc)
      }
    }
  ]
}
Please Note The 'sessions' count in the above data structure is a total of the sessions started for each agent during the time period. If the same session id is passed to ten different agents, that session id is counted 10 times in the 'sessions' count. You can get a per-agent breakdown from the 'Agent Status' report data (see top of this document).