Teneo Developers

TQL manual

Teneo Query Language

Inquire is the Teneo Platform's integrated suite of services for Natural Language Interaction (NLI) log data manipulation and exploration.

Querying in Teneo is carried out using the Teneo Query Language (TQL), a proprietary query language designed for retrieving information from the Teneo Engine log data, which is generated by a conversational AI application during its interactions with users.

In Teneo Studio, querying is carried out in the backstage (Solution tab) under Optimization > Log Data.

TQL syntax overview

A TQL query consists of a command, a selection, and, optionally, a set of constraints and/or transformers and meta-parameters.

The command is separated from the selection by a space, the selection is separated by a colon from the constraints, the constraints/transformers are separated from each other by commas, and finally the meta-parameters are included after a space, as in the pseudo-code example below:

tql

1<command> <selection> : [<constraint1>|<transformer1>],
2[<constraint2>|<transformer2>],… <meta-parameters>
3

Where:

  • The command is the operation that the user wants to carry out.
  • The selection indicates what is retrieved and computed by the query. The selection can hold more than one variable.
  • The constraints (optional) can be used to filter undesired data off results, e.g. by specifying a date range, or a required variable value; or it can be used to indicate a particular relation among the items, e.g. by indicating that an event should happen before another event.
  • The transformers (optional) can be used to generate new data from the source data, e.g. by manipulating date strings, obtaining top N items from ranking lists, or matching strings against Teneo Solution Language Objects.
  • Finally, the meta-parameters (optional) are additional operators used to indicate how to organize the query results.

The next sections provide a complete description of TQL commands, constraints, transformers and meta-parameters, as well as the support commands for sub-queries and saving query results.

TQL syntax is case sensitive, hence TQL queries must use the correct capitalization
TQL syntax supports commenting out parts of the query by using// and /* */.

Query constructs

The structure of data (sessions containing transactions, transactions containing events, and the three of them containing properties) is reflected in the TQL syntax, which requires and explicit construct marker indicating the path to reach any property in the data.

The following notations are used:

  • s to refer to the session
  • s.<attribute_name> to refer to an attribute of the session, e.g. s.beginTime
  • t or s.t to refer to a transaction
  • t1, t2, t3, etc. to refer to different transactions. The digit does NOT indicate the order of precedence.
  • t.<attribute_name> to refer to a transaction attribute, e.g. t.time
  • e, t.e or s.t.e to refer to an event
  • e1, e2, e3, etc. to refer to different events. The digit does NOT indicate the order of precedence.
  • e.<attribute_name> to refer to an event attribute, e.g. e.type.

Querying events

Events’ properties are identified using the e. construct.

For example:

tql

1e.fname
2

refers to the fname property of an event.

A notation digit can be added to the construct to differentiate different events. In the example below, fname and userInput are properties of the same event:

tql

1e.fname, e.userInput
2

whereas in the next example, fname and userInput are properties of two events that may not be the same one (although they can be):

tql

1e1.fname, e2.userInput
2

Note that the digits here do not imply order or sequence, merely difference. To specify the precedence order of events, use Skip-constraints, or the index property.

Querying transactions

Transactions are queried using the t. construct, which has two uses: to access a property at the transaction level, and to constrain a set of events to occur on the same transaction.

The example below specifies that duration is a property of a transaction:

tql

1t.duration
2

The following example refers to two event properties that co-occur in different events within the same transaction:

tql

1t.e1.fname, t.e2.userInput
2

Notation digits can also be added to the t. construct – e.g. t1, t2 – to refer to distinct transactions. The following examples specifies that duration and time are properties of two distinct transactions:

tql

1t1.duration, t2.time
2

Similarly, it is possible to combine independent transactions and events in a single construct. The example below refers to two independent transactions (t1 and t2). Properties fname and userInput are obtained from two independent events (e1 and e2) that co-occur in the same transaction (t1), whereas answerText property is obtained from event e3 in transaction t2.

tql

1t1.e1.fname, t2.e2.userInput, t2.e3.answerText
2

Note thought that the above TQL selection is not allowed in Inquire because it requires additional constraints. See the section Computational Costs of TQL below for more information about this constraint.

When the t. construct is not present, a query will search for all events matching a constraint, regardless the transaction they belong to. For example:

tql

1e1.fname, t1.e2.userInput, t2.e3.answerText
2

does not constrain the event e1 to belong to the same transaction as e2, and so all the events in the data will be considered.

Querying sessions

Session level properties are accessed by means of the s. construct. The TQL query is automatically scoped to a single session at a time, so this is the only construct that can be used to refer to session properties. For example:

tql

1s.beginTime
2

Note that this means that it is not possible to use digits – such as s1, s2 – to refer to two different sessions.

Querying backstage

Query scope

Teneo Inquire considers all elements in a single Log Data Source (LDS). When a query is executed, Teneo Inquire looks into all the all sessions, transactions, and events, as well as user-generated data, such as augmenters, saved results and solutions that are available in the LDS.

As aforementioned, a TQL query is automatically scoped to work within individual sessions. This means that an individual query can operate across transactions and events within a single session at a time (querying elements intra session) but does not perform searches in elements that belong to two different sessions (querying inter-session).

Computational costs of TQL and good practices

As mentioned above, it is possible to combine properties from independent events and transactions by adding digits to their construct – e1, e2, t1, t2, etc. This TQL ability is very powerful but it also has computational implications given that all events and transactions will be taken in consideration for every distinct construct. For example, in the following construct

tql

1t1.e1.fname, t1.e2.userInput
2

e1 and e2 are explicitly constrained to belong to the same transaction. Such construct takes into consideration all events having the property fname in a transaction and computes the cross product with all events having the property userInput within the same transaction.

In contrast, the following example

tql

1e1.fname, e2.userInput
2

computes the cross product between all events having the property fname and those having the property userInput, regardless the transaction they belong to.

Therefore, the more distinct constructs are used, the more computationally demanding the query is. Hence, indicating which transaction an event should belong to is highly recommended.

In fact, Teneo Inquire includes a query validation mechanism that checks for potentially dangerous queries that may be highly demanding. If such a query is identified, Teneo Inquire rejects it and does not execute it, and Studio displays a warning message informing the user the query has not been carried out. The user will need to review the constructs in both the query selections and the constraints and step them up where possible.

A good practice would be to include the event type in the queries, this reduces the number of comparisons required to compute the results. For instance, when selecting the userInput variable in the query, the type=="request" constraint could be added:

tql

1la e.userInput: e.type == "request"
2

When selecting the answerText variable, the type=="response" constraint could be added:

tql

1la e.answerText: e.type == "response"
2

Similarly, when selecting properties from a path event, it can be constrained by means of the pathType:

tql

1lu t1.e.fname : t1.e.pathType=="raise-flow"
2

Working with variables

There are several types of variables logged in the log data: flow variables (fv), session variables (sv), log variables (lv), and annotations variables (annotation.variables). These and how they are logged by Teneo Engine are further described in the Data Model section. This section shows the difference among them and how to query them.

Session and flow variables: Teneo Inquire logs the values of these variables according to their type: string, integer, etc. However, a variable can be assigned different types in different events. In order to enable Teneo Inquire to deal with varying types, variable names are prefixed according to the table below. Also, flow and session variables are prefixed in Inquire with fv: and sv: respectively. Hence, the full name of a variable in Inquire becomes fv:TYPE:VARIABLE_NAME in case of flow variables and sv:TYPE:VARIABLE_NAME in case of session variables.

TypePrefixVariables as declared in StudioVariable name in Inquire/Log dataQuery example
BinarybinmyBinaryVarbin:myBinaryVarla e.fv:bin:myBinaryVar
BooleanbmyBoolVarb:myBoolVarla e.fv:b:myBoolVar
IntegernmyIntVarn:myIntVarla e.fv:n:myIntVar
DatedmyDateVard:myDateVarla e.fv:d:myDateVar
DoublefmyDoubleVarf:myDoubleVarla e.fv:f:myDoubleVar
StringsmyStrVars:myStrVarla e.fv:s:myStringVar
ArrayamyArrayofDoublea:f:myArrayofDoublela e.fv:a:f:myArrayofDouble
Everything else, including map and nullomyMap, myObjecto:myMapo:myObjectla e.fv:o:myMap, e.fv:o:myObject

As seen in the table, arrays are prefixed with two values: a: which indicates that the variable contains an array, and the TYPE: which indicates the type of the values in the array, e.g. a:f:myArrayofDouble.

Log variables: These variables are logged at transaction level and they can only contain a String, regardless if their content represent an Object. Hence, they do not need to be prefixed and they can only be constrained by means of string operators. Example la t.lv:intVar : t.lv:intVar ~= ".*\d.* "

Annotation variables: These variables can only happen within an annotation. They are logged as a map where each property is a key-value pair of strings-object, i.e. the key is a string and the value is an arbitrary object. Hence, annotation variable names are also prefixed according to their type, the same way as flow and session variables are. Note that strings in JSON are stored with quotes. Hence, they should be considered when trying to match their content.

Examples:

(1) la e.annotation : e.annotation.variable.f:confidence > 0.5
(2) la e.annotation : e.annotation.variables.n:order == 0
(3) la e.annotation : e.annotation.variables.s:name ~= "NN.POS"

Working with annotations

Annotations are a special type of data in Teneo Inquire which are organized in a nested structure.

Map annotation { Long sentence-index, Map<String,String> variables, Long[] word-indices, String name, String action }

The String value in the attributes of the variables property represents a JSON object having nested structures such as numbers, strings, maps or arrays. They can be used the same way as session or flow variables. See the sections Data Model and Annotations for additional details.

It is possible to use log data to query some of the Annotation data, for instance:

Annotation properties

(1)

tql

1lu e.annotation.sentence-index, e.annotation.variables, 
2e.annotation.word-indices, e.annotation.name, 
3e.annotation.action
4

A specific property in the variables map

(2)

tql

1lu e.annotation.variables.f:confidence
2

Numeric conditions over annotation.sentence-index

(3)

tql

1la e.annotation : e.annotation.sentence-index == 0
2

(4)

tql

1la e.annotation : e.annotation.sentence-index == in {0,1}
2

String comparisons over annotation.name and annotation.action

(5)

tql

1lu e.annotation : e.annotation.action == "added",
2e.annotation.name == "VB.POS"
3

(6)

tql

1lu e.annotation : e.annotation.action == "added",
2e.annotation.name ~= ".*SENTENCE_WORDS.*"
3

Numeric comparisons on annotation variables properties having numbers

(7)

tql

1lu e.annotation.variables.f:confidence : e.annotation.variables.f:confidence > 0.8
2

(8)

tql

1lu e.annotation : e.annotation.variables.n:sentenceLength == 1
2

String comparisons on annotation variables properties having strings

(9)

tql

1lu e.annotation : e.annotation.variables.s:words ~= ".*[hH]ello.*"
2

(10)

tql

1lu e.annotation : e.annotation.variables.s:words == "hello"
2

Array comparisons over annotation word-indices properties. Note that word-indices is an array of numbers

(11)

tql

1 lu e.annotation : e.annotation.word-indices ~~ 1,
2 e.annotation.word-indices ~~ 2 
3

Note the use of the contains operator, this query returns annotations having the values 1 and 2 in the word-indices array. The above query can also be written in a more compact way as: lu e.annotation : e.annotation.word-indices ~~ {1,2}

(12)

tql

1lu e.annotation : e.annotation.word-indices == in {{0},{2}}
2

Search for empty values

(13)

tql

1lu e.annotation : notexists e.annotation.variables.s:words
2

Returns annotations which have no words property in the variables

(14)

tql

1lu e.annotation : notexists e.annotation.word-indices
2

Return annotations having empty arrays in word-indices

(15)

tql

1lu e1.id, !e1.annotation.word-indices : notexists e1.annotation.word-indices
2

Returns empty arrays AND events without annotations

Array comparison over annotation array type variables

(16)

tql

1la e.annotation : e.annotation.variables.a:s:words == {"hi","hello"}
2

Returns annotations having exactly the array {"hi", "hello"} as value for the variable a:s:words

(17)

tql

1la e.annotation : e.annotation.variables.a:s:words ~~  {"hi","hello"}
2

Returns annotations having "hi" and "hello" within the values of the variable a:s:words

(18)

tql

1la e.annotation : e.annotation.variables.a:s:words ~~ in {{"hi"}, {"hello"}}
2

Returns annotations having "hi" or "hello" within the values of the variable a:s:words

(19)

tql

1la e.annotation : e.annotation.variables.a:s:words ~= {"H.*", "h.*"}
2

Returns annotations where the variable value is an array of size 2, the first element matches "H." and the second matches "h."

However, the current Teneo Inquire version still has some limitation when dealing with annotations. It does not allow certain constraints such as:

Use of ranges with set constrain and array

(20)

tql

1lu e.annotation : e.annotation.word-indices == in {0..2}
2

Working with dates

Teneo Inquire data includes some date properties, such as the session begin time, which are all logged in Z zone using the format: yyyy-MM-ddTHH:mm:ssZ, e.g.: 2018-12-12T09:07:26Z. Dates are however displayed in the frontend using the locale of the host.

Furthermore, users can assign Date values to variables in Teneo using the scripting functionalities, for example, assigning a Date value to a flow variable using the on-drop script. These dates can be stored using the user preferred date/time format. However, note that Teneo Inquire will identify and store as Date those that comply with the ISO-8601, else they will be stored as a String.

TQL reference guide

Main TQL commands

Teneo Query Language currently supports five basic commands, as displayed in the table below – both full and abbreviated forms are valid TQL syntax:

CommandAbbreviated formReturn value
listAlllalist
listUniquelulist
countAllcainteger
countUniquecuinteger
distributedtable

Count All

The countAll function counts all values matching the query and returns an integer: example (1) below counts all user inputs, including duplicates; example (2) counts the total number of user inputs containing the string "hello".

(1)

tql

1ca t.e.userInput
2

(2)

tql

1ca t.e.userInput : t.e.userInput ~= ".*hello.* "
2

Count Unique

The countUnique function counts all unique values matching the query and returns an integer.

Example (1) below counts all unique user inputs, excluding duplicates, whereas Example (2) counts the number of dialogs, that is, of unique sessions.

(1)

tql

1cu t.e.userInput
2

(2)

tql

1cu s.id
2

Note: Uniqueness is determined by string comparison and is case sensitive.

Distribute

The distribute function counts the number of occurrences for all unique values matching the query and returns a 2-column table of un-ordered counts.

The example below counts the number of occurrences for each unique user input (including empty values).

(1)

tql

1d t.e.userInput
2

Example output:

t.e.userInputcount
Bonjour1
What time is it12
Hello102
Hi7
How are you4

Note: Uniqueness is determined by string comparison and is case sensitive.

List All

The listAll function retrieves all values matching the query and returns a list.

The example below lists all user inputs, including duplicates.

(1)

tql

1la t.e.userInput
2

Example output:

t.e.userInput
Hello
What time is it
Bonjour
Hi
hi
Hello
What time is it
Hi

List Unique

The listUnique function retrieves all unique values matching the query and returns a list.

Example (1) below lists all unique session ids, while example (2) lists all user inputs, excluding duplicates.

(1)

tql

1lu s.id
2

(2)

tql

1lu t.e.userInput
2

Example output:

t.e.userInput
What time is it
Hi
Hello
Bonjour
hi
how are you

Note: Uniqueness is determined by string comparison and is case sensitive.

Selection

The selection specifies what data does the TQL query return. Variable names in the selection are re-used as column headers in the output. Queries composed of a selection with no constraints retrieve all existing matching items in the data. In contrast, constraints in the TQL query are projected onto the selection, which may result in null or empty results if a constraint is not applicable to a variable in the selection, e.g. due to values belonging to different events.

Handling null results (! operator)

Due to the nature of the Teneo Platform and the large amount of data fields that exist in a solution, TQL results often come with lots of null and undefined values - for example, the query la e.userInput returns empty rows for all events not having the e.userInput property, which are the majority of them. This behavior slows down queries by producing a lot of meaningless results.

In order to obtain more convenient query results and enhance the overall performance, Inquire adds, by default, exists-like constraints for every item in the selection, which filters fields with data and removes nulls and undefined results.

Therefore, the queries listed below return equivalent results:

(1)

tql

1la s.id, e.userInput
2

(2)

tql

1la s.id, e.userInput : exists e.userInput
2

(3)

tql

1la s.id, !e.userInput : exists e.userInput
2

Retrieving null results

Nonetheless, since certain use-cases may require seeing undefined and null results, a short-hand syntax has also been introduced that allows to not add exists-like constraints. By using the exclusion operator !, query results also include null results.

Please note that if both exists and ! appear in the same query, exists prevails.

Aliases

TQL supports aliasing using the as notation. The example below:

(1)

tql

1lu t.e.userInput as inputs
2

produces the result:

Inputs
What time is it
Hi
Hello
Bonjour
hi
how are you

Aliasing can also be used as a shorthand for referring to outputs from transformers (see Transformers section). In the Trend example below, aliasing is used to access and display multiple elements of the result as specified in the selection:

(2)

tql

1lu t.e.userInput, tau, direction, p, z : 
2	trend(min-tau="0.0", max-p="0.05") t.e.userInput
3		as (tau, direction, p, z)
4

Example output:

t.e.userInputtaudirectionpz
What time is it0.2019696471802132-0.01-2.719869088995218
Bonjour0.1543033499620919-0.05-2.077960316065855
hi0.1497410977398209-0.05-2.016521733740252

Note: Inquire supports unique names only. This means that if two variables are assigned the same alias, only the last assignment is kept. In the example below, the result will be a single column containing the transaction time values. The session-level value from s.beginTime is not displayed.

(3)

tql

1lu s.beginTime as time, t.time as time
2

This behavior also applies to the variable name count which is automatically created by the distribute command; that is, only one variable called count can exist in the same output and only the last assigned value will be kept.

Also, aliasing cannot be used as a shorthand in the constraints. The following query would be invalid:

(4)

tql

1lu s.beginTime as time: time > 2019
2

Constraints

Constraints are used to specify conditions that sessions, transactions, and events properties must fulfil. They are separated from the selection by the colon symbol (:) and consist of multiple terms, separated by a comma. The relation between a property value and its condition is specified by means of one of eight operators below.

Core constraint operators

TQL supports the eight basic constraint operators listed in the table below. The table shows a brief explanation of their function, an indication of whether or not they support regular expressions, what field types they apply to, and example event properties they can be applied to.

OperatorFunctionApplies toExample
==equalsstrings, numeric, date, arrayuserInput, index, beginTime
!=not equalsstrings, numeric, date, arrayuserInput, index, beginTime
~=Matches (regex)Strings, arrayuserInput, folder, fname
!~not matches (regex)Strings, arrayuserInput, folder, fname
~~containsanalyzed text, arrayuserInputWords, answerTextWords word-indices
!~~not containsanalyzed text, arrayuserInputWords, answerTextWords word-indices
existsx existsanyany
notexistsx does not existanyany
>, <, >=, <=greater than x, less than x, greater or equal than x, less or equal than xnumeric, date, numeric arraytransactionsCount, beginTime,

The exists and notexists operators are special in that they only take one argument (the event property to check) and return a match if it exists.

The contains operator ~~ can be used to perform case-insensitive queries of userInputWords and answerTextWords, by checking if a field contains or does not contain a word or word sequence. This is possible because these two fields are stored as analyzed text, which is lowercased, tokenized, and normalized before being stored as an array of tokens (words).

Example

The example below, which retrieves a list of user inputs, consists of a constraint with three terms specifying that the user session must have been initiated in March 2015, one of its events must have raised a flow located in the "Safetynet" folder and the event that contains the userInput and the event indicating the folder name are related in the same transaction:

tql

1lu t.e1.userInput :
2s.beginTime == "2015-03", 
3t.e2.folder ~= ".*?[S|s]afetynet.*?",
4t.e1-{pathType=="flow-trigger"}>t.e2
5

Using core constraint operators with array type attributes

The eight basic constraint operators listed above can be applied against array type attributes. In this case the constraint is specified by means of one of the constraints operators and the possible values enclosed by curly brackets {}.

These are examples on how to use the constraint operators with arrays:

(1)

tql

1la e.sv:a:s:words : e.sv:a:s:words == {"hello", "hi"}
2

Returns all cases where the value of the variable e.sv:a:s:words is exactly the array {"hello", "hi"}. Note that the equals operator does an exact match, that means: the size, the order and the value of each of the items of the array must match.

(2)

tql

1la e.sv:a:s:words : e.sv:a:s:words ~~ {"hello", "hi"}
2

Returns all cases where the value of the variable e.sv:a:s:words contains "hello" and "hi". In this case nor the size of the array or the order matters. It only needs to contain both: "hello" and "hi".

(3)

tql

1la e.sv:a:s:words : e.sv:a:s:words ~~ "hello"
2

Returns all cases where the value of the variable e.sv:a:s:words contains "hello". Note that in this case it is not needed to enclose the possible value in curly brackets. The use of the operators contains/not-contains against arrays specifying just one single possible value is the only exception where the curly brackets are not required.

The above is equivalent to the following query:

tql

1la e.sv:a:s:words : e.sv:a:s:words ~~ {"hello"}
2

(4)

tql

1la e.sv:a:s:words : e.sv:a:s:words ~= {"h.*", "b.*"}
2

Returns all cases where the first element of the variable value matches "h.*" and the second element matches "b.*". Note that the size of the array matters, in the above example only arrays of size two will match the constraint, as well as the order of the regex expressions in the constraint.

(5)

tql

1la e.sv:a:d:dates : e.sv:a:s:dates ~= {"2018.*", "2019.*"}
2

Returns all cases where the first element of the array matches "2018.*"and the second element of the array matches "2019.*". Note that in this case the regex operator is used against an array of dates. Inquire allows the usage of the regex operators against any type of array; if the elements are not strings, their string-value will be automatically calculated before applying the operator.

(6)

tql

1la e.sv:a:n:numbers : e.sv:a:n:numbers > {1, 2}
2

Returns all cases where the first element of the array is greater than 1 and the second element of the array is greater than 2.

Set-constraints

Set constraint allows a condition to be evaluated against more than one possible value grouping the result of them in an OR. Set constraint is specified by means of one of the constraints operators above, the reserved word in, and possible values enclosed by curly brackets {}.

The example below will return user inputs that consist of only lower case "hi" OR lower case "hello":

(1)

tql

1la t.e.userInput : t.e.userInput == in {"hi", "hello"}
2

Note: the previous TQL query would be expanded as:

(2)

tql

1t.e.userInput == "hi" OR  t.e.userInput == "hello"
2

Set-constraints is currently the only way of constraining value ranges. For example, the TQL query below will return user inputs for sessions that were initiated in the range from January 2017 to March 2017:

(3)

tql

1la t.e.userInput : s.beginTime == in {"2017-01".."2017-03"}
2

The examples below are equivalents and will cover all of 2017:

(4)

tql

1la t.e.userInput : s.beginTime == in {"2017-01".."2017-12"}
2

(5)

tql

1la t.e.userInput : s.beginTime == in {"2017".."2017"}
2

It is also possible to put constraints on the hours and minutes as in the example below, which finds inputs for sessions initiated from midnight to 1 a.m.:

(6)

tql

1la t.e.userInput : s.beginTime == in {"2015-01-01T00:00".."2015-01-01T01:00"}
2

As well as specifying multiple ranges in the same set-constraint:

(7)

tql

1la t.e.userInput : s.beginTime == in {monday..tuesday, thursday..friday}
2

Set-constraints used in date ranges

When date ranges are specified in set-constraints, Inquire is expanding these dates into intervals taking into account the date of the first and the last stored sessions in its database. For example:

(8)

tql

1la t.e.userInput : s.beginTime == in {january..march}
2

Considering that the first stored session is from 2017-01-01T00:00:00 and the last stored session is from 2019-12-31T23:59:59, it would be expanded in the following intervals:

  • from 2017-01-01T00:00:00 to 2017-03-31T23:59:59
  • from 2018-01-01T00:00:00 to 2018-03-31T23:59:59
  • from 2019-01-01T00:00:00 to 2019-03-31T23:59:59

Thus, it would return all sessions from the specified month range from all years present in the stored data.

The same applies to weekdays.

Set-constraints can also be evaluated against array type values.

The example below will return user inputs present in events containing the variable a:s:words with either the arrays {"Hello", "hi"} or {"Bye"} as value.

(9)

tql

1la t.e.userInput : t.e.a:s:words == in {{"hello", "hi"}, {"Bye"}}
2

In a similar way, the example below will return user inputs present in events containing the variable a:s:words if the value of the variable contains {"hello" and "hi"} or {"Bye"}

(10)

tql

1la t.e.userInput : t.e.a:s:words == in {{"hello", "hi"}, {"Bye"}}
2

Skip-constraints

The skip-to syntax specifies the order of precedence among transitions or events by setting out a starting point and an end point, where the end point has some property. The syntax (in pseudo-code) is schematically as follows:

start -{end point constraints}> end point end point <{end point constraints}- start

The starting and end points can be transactions or events (not necessarily in the same transaction). The constraints enclosed by {} specify the constraints the end points must fulfil. The direction of the arrow specifies whether the constraint is skipping forward or backwards in the session.

The example query below lists non-empty user inputs alongside non-empty responses immediately following them:

(1)

tql

1lu s.id, t1.e.userInput, t2.e.answerText :
2t1.e.type == "request",
3t1.e -{type=="response", answerText~=".+"}> t2.e,
4t1.e.userInput ~= ".+"
5

The second example query lists non-empty user inputs alongside non-empty responses immediately preceding them:

(2)

tql

1lu s.id, t1.e.answerText, t2.e.userInput :
2t2.e.type == "request",
3t2.e.userInput ~= ".+",
4t1.e <{type=="response", answerText~=".+"}- t2.e
5

Regular expressions

TQL supports a subset of full Perl-style regular expressions, essentially the same subset supported by Elasticsearch. Regular expressions in TQL are by definition anchored, meaning that any regex must match the entire string from beginning to end. Therefore, the conventional anchoring symbols ^ and $ are not needed.

The table below summarizes the allowed regex symbols:

Operator/SymbolExplanation
`. ? + * { } [ ] ( ) " \`
.Any character
?Zero or one time
+One or more times
*Zero or more times
{1,3}Min / max number of matches
()Grouping
``
[a-z], [A-Z], [^a]Character classes (including negation)
\dAny digit
\wAny word character
\WAny non-word character
\sA space character
\SA non-space character
\\bA word boundary
\\tA tabulator character
\\nA newline character
\\rA carriage return character

Note that escape characters \b, \n, \r, and \t require double backslash escapes.

Regular expressions are supported for fields stored as strings only (not for numeric values or analyzed text). Property values are match against regular expression via the ~= (matches) and !~ (does not match) operators. The !~ operator evaluates to true either when the regex does not match or when the property is not present.

The first example below will identify user inputs that contain either "Hello" or "hello", whereas the second will identify all non-empty inputs.

(1)

tql

1lu t.e.userInput : t.e.userInput ~= ".[H|h]ello."
2

(2)

tql

1lu t.e.userInput : t.e.userInput ~= ".+"
2

The following examples are also valid. In the below cases, although the variables are declared as date and number, the use of regular expression is allowed. That is because, in the case of variables, Inquire will calculate the string value if needed before applying the constraint.

(3)

tql

1lu t.e.sv:d:dates : t.e.sv:d:dates ~= "2018.*"
2

(4)

tql

1lu t.e.sv:n:number : t.e.sv:n:number ~= "1.*"
2

(5)

tql

1lu t.e.sv:f:float : t.e.sv:f:float ~= "1.*"
2

Order of constraints

In the evaluation of a query, the TQL parser transforms the TQL expression into a directed graph that guarantees that each part of the query expression is evaluated in the correct order. Therefore, the order of constraints does not matter.

For example, the two queries below return the same results:

(1)

tql

1d e.userInput :
2s.beginTime == "2015-03",
3e.userInput ~= ".hello."
4

(2)

tql

1d e.userInput :
2e.userInput ~= ".hello.",
3s.beginTime == "2015-03"
4

Syntax and operators for diverse data types

Inquire holds diverse type of data, namely: string, integers, floats, dates, Boolean, arrays, and maps. The following table summarizes the operators that can be used with each of them.

Data typeExampleOperatorsRemarks
String"foo", "bar"==, !=, ~=, !~, exists, notexistsA quote sign followed by anything and ending with the same quote sign. The valid quote symbol is "
Date"2018-01-01", "2018-01", "2018"==, !=, exists, notexists, >, <, >=, <=A string with an ISO-8601 formatted date or date time. This includes the following java Datetime classes: Date, Instant, LocalDate, LocalDateTime, OffsetDateTime, Time, UtilDate.
Booleantrue, false==, !=, exists, notexistsA boolean
Integer1, 3094, -5==, !=, exists, notexists, >, <, >=, <=An Integer
Double/Float0.42, 1.5==, !=, exists, notexists, >, <, >=, <=A number with a '.' inside.
Array[0,1,2]~~, !~~, exists, notexists, ==, !=, , >, <, >=, <=, ~=, !~An array of some type.
Map Object{name: "john", surname: "doe"}==, !=, ~=, !, exists, notexistsMaps (and objects in general) are stored as json strings. It is possible to check if the map contains a particular value by using regular expressions, or if it is exactly the same as a given map.

Syntax for attributes with inner spaces

In general terms, querying Teneo Studio metadata variables follows standard TQL syntax rules, e.g.:

(1)

tql

1la t.e.userInput: t.md:CATEGORY
2

However, metadata name can contain a white space. In this case, the name in the query should be written using the formulas [‘<metadata name>’] or ["<metadata name"], as in

(2)

tql

1la t.e.userInput: t.md:['CURRENT PAGE']=="http://mysite.com"`  
2

(3)

tql

1la t.e.userInput: t.md:["CURRENT PAGE"]==http://mysite.com`
2

Engine-matching filters

Condition matching

The matches-condition transformer can be applied to a text field variable and transforms its values into a match or non-match against a solution Language Object (LOB) condition. The matches-condition transformer has the following parameters (them all are mandatory):

ParameterDefault valueDescription
solutionnonestring specifying the solution containing the condition(s)
conditionnonea string representing a language condition

The output is a Boolean value that discards those items that did not match the condition.

Examples

The example below returns only those user inputs that match the FOOD.LIST Language Object in the "en_tlr" solution (see the section Lexical resources for log data analysis for more information on how to import solutions to an LDS):

(1)

tql

1lu t.e.userInput :
2t.e.type == "request",
3t.e.userInput matches-condition
4(solution = "en_tlr", condition = "%FOOD.LIST")
5

Matching Language Objects

The matching-lobs transformer can be used to find Language Objects within a solution that match a given input text. An optional filter is available for partial matching of Language Objects.

ParameterDefault valueDescription
solutionnonesolution to use, e.g. "en_tlr"
filternoneOptional. Regular expression that the language objects must fulfil, e.g. ".*?PHR.*?" to use PHR lobs only.

The output values are:

  • lob: the name of the Language Object that matches the input text
  • used-words: an array of string that represent the words from the input text that contribute to the actual match

Examples

Example (1) below retrieves all Language Objects present the "en_tlr" solution that match any user input that contains the word "no" surrounded by spaces. Example (2) does the same but looks for PHR Language Objects only.

(1)

tql

1lu t.e.userInput, lob, used-words :
2matching-lobs( solution="en_tlr" )
3	t.e.userInput as (lob, used-words),
4t.e.userInput ~= ".* no .* "
5

(2)

tql

1lu t.e.userInput, lob, used-words :
2matching-lobs( solution="en_tlr", filter=".*?PHR.*?")
3	t.e.userInput as (lob, used-words),
4    t.e.userInput ~= ".* no .* "
5

Example matching

The matches-examples filter can be used to retrieve text field variables that match the Language Object condition generated by a set of (positive and/or negative) examples. The parameters are:

ParameterDefault valueDescription
solutionnoneThe solution to match example against
pnoneA string with a positive example to match against
nnoneA string with a negative example to match against
  • At least one positive or negative example must be specified.
  • More than one positive or negative example can be used at the same time.

Examples

Example (1) below identifies user inputs matching the condition generated by "can you tell me a riddle?" and "do you know any riddles?". The generated condition depends on the content of the solution being uploaded to the environment as en_tlr.

(1)

tql

1lu e.userInput :
2e.type == "request",
3e.userInput matches-examples(
4	solution="en_tlr",
5	p = "Can you tell me a riddle?",
6	p = "Do you know any riddles?")
7

Classification

The classify transformer can be used to classify variable values according to certain criteria expressed using a Language Object (LOB), e.g. for sentiment analysis identifier, or yes/no answers. The parameters are:

ParameterDefault valueDescription
solutionnoneThe solution name to profile against
cnoneOne or multiple c conditions having each a string with a language object to match against. E.g.: c=%YES, c=%NO
<alias>noneThe alias for the variable that will hold the results
<label>noneAs many labels as c parameters. The actual value assigned to the <alias> variable when the i-th LOB in c has matched.
<no-match-label>noneOptional. The value assigned to the <alias> variable when no LOB matches.

Examples

Example (1) below illustrates how user inputs can be classified as either "Positive", "Negative" or "Neutral". Let’s consider a solution named Sentiment that contains two Language Objects, SENTIMENT_POSITIVE.INDICATOR and SENTIMENT_NEGATIVE.INDICATOR. The classify operation below creates a new variable called "sentiment" which takes three values: Positive wen the first LOB is matched, Negative when the second LOB is matched, and Neutral when no LOB matches.

(1)

tql

1lu s.id, sentiment :
2classify(
3	solution="Sentiment",
4	c="%SENTIMENT_POSITIVE.INDICATOR",
5	c="%SENTIMENT_NEGATIVE.INDICATOR" ) t.e.userInput as
6	( sentiment, "Positive", "Negative", "Neutral" ),
7	t.e.type == "request"
8

Example (2), shown below, illustrates the use of the English TLR solution to annotate the polarity of Yes-No-Answers. There is no "non-match" label, so input text that don’t match any of the two LOBs will be discarded:

(2)

tql

1lu s.id, polarity :
2classify(solution="en_tlr", c="%YES.PHR", c="%NO.PHR")
3	t.e.userInput as (polarity, "Yes", "No"),
4t.e.type == "request"
5

Classify returns an array containing the matched labels.

Transformers

Transformers are a group of TQL functions that take a variable value, execute some data manipulation, and return one or more new data values. The returned value(s) and type of data manipulation differ depending on the transformer. Transformers are used as part of a constraint in the query, and the transformed output can be returned in the selection via aliasing.

With transformers, the Teneo Inquire API may throw a null pointer exception if the type of the variable to be transformer is not correctly specified.

Dates

The catd transformer can be applied to a date field and can be used to transform its time stamp value into other date formats or to group query results by time at different levels of granularity. It accepts the four parameters below, and at least either model or pattern are mandatory.

Parameters and values are listed below:

ParameterDefault valuePossible valuesDescription
modelnone"date", "day-of-week", "month", "quarter"A model for grouping results by time at different levels of granularity
patternnone"yyyy-MM-dd", "kk", "yyyy-'w'ww", etc.A date format string for time and date output. See Java manual for all possible values.
locale"en"as for Java LocaleA locale for formatting time and date output
timezone"UTC"as described by Joda-Time's DateTimeZone classA timezone id. Can be an offset relative to UTC, e.g. `"-05:00"

Examples

Example (1) below transforms the full time stamp as stored in Teneo Inquire into a conventional year-month-day string (and is equivalent to calling pattern="yyyy-MM-dd"). Example (2) counts the number of sessions per weekday. Example (3) shows how to use relative time zone offsets to obtain results according to another time zone.

(1)

tql

1lu date : catd(model="date") s.beginTime as date
2

(2)

tql

1d day : catd(model="day-of-week") s.beginTime as day
2

(3)

tql

1d date : catd(model="date", timezone="-05:00") s.beginTime as date
2

The trend transformer can be applied to a text field variable and transforms its values into a list of trending items. The trend algorithm is based on the Mann-Kendall non-parametric trend test, and the input arguments are:

ParameterDefault valueDescription
min-tau"0.5"string specifying the cutoff for the absolute value of the tau correlation coefficient (min-tau)
max-p"0.05"string specifying the cutoff for the p-value (max-p)

The output values are:

  • the text field variable values in the selection matching the input arguments to the trend transformer
  • tau: the absolute value tau correlation coefficient
  • direction: the direction of the trend (negative: "-" or positive: "+")
  • p: the p-value (significance) of the trend
  • z: the z-value (standard score) of the trend

The outputs can be referred to by means of aliasing.

Examples

Example (1) below returns any trending user inputs that are significant at the 0.05 level and that have a medium sized trend effect, alongside the tau value. Example (2) returns all user inputs alongside their tau value, direction of trend (if any), p-value, and z-value, ordered by tau in ascending order:

(1)

tql

1lu t.e.userInput, tau :
2t.e.type == "request",
3trend (max-p="0.05", min-tau="0.3") t.e.userInput as tau
4

(2)

tql

1lu t.e.userInput, tau, direction, p, z :
2t.e.type == "request",
3trend (max-p="1", min-tau="0") t.e.userInput
4	as (tau, direction, p, z)
5order by tau
6

Meta parameters

Meta parameters are optional parameters that work with all commands. They generally affect the behavior of TQL, rather than operating directly on the data.

order by

The order by meta parameter orders the output data according to a mandatory variable name argument and an optional argument specifying the direction of the ordering.

Ordering is ascending by default, which can also be stated using the expression asc, but can also explicitly be set to descending using desc. For string values, ordering is alphabetical.

Examples

Examples (1) and (2) order user inputs bottom up, i.e., from less frequent first to most frequent. In contrast, example (3) orders user inputs top to bottom, i.e., most frequents first.

(1)

tql

1d e.userInput : e.type == "request" order by count
2

(2)

tql

1d e.userInput : e.type == "request" order by count asc
2

(3)

tql

1d e.userInput : e.type == "request" order by count desc
2

limit

The limit meta parameter takes an integer argument specifying a cap on the number of results returned. The meta parameter is specified with the TQL query. The example below returns the 10 first results of the query:

(1)

tql

1lu e.userInput : type == "request" limit 10
2

sample

The sample meta parameter takes an integer argument specifying how many sessions to randomly sample and return results from. The example below shows how to return results from a sample of 100 sessions:

(1)

tql

1sample 100 la e.userInput
2

Examples combining several constraints and transformers

Example (1) lists all unique user inputs that ended up in the Safetynet flow. Note that, by specifying that the transaction must have an event e of type request, we ensure that there will also be another event e2 present in the same transaction having the property fname (flow name). Also, the regular expression alternation pattern [S|s] allows the constraint to match flow names written either with lower or upper case "s".

(1)

tql

1lu t.e.userInput :
2t.e.type == "request", t.e2.fname ~= ".*[S|s]afetynet.*"
3

Example (2) counts the number of times that each of inputs in example (1) occurs and list the top-10 inputs that are most frequent. Note that the operator order by used in this query orders the results in ascending order (the default value), whereas the operator limit picks the top-10 results (after being ordered):

(2)

tql

1d t.e.userInput :
2	t.e.type == "request",
3	t.e2.fname ~= ".*[S|s]afetynet.*"
4	order by count limit 10
5

Example (3) counts the number of times a use input occurs per day, and order the results by date:

(3)

tql

1d date, t.e.userInput : t.e.type == "request",
2	t.e2.fname ~= ".*[S|s]afetynet.*",
3	catd(model="date") s.beginTime as date
4	order by date asc
5

Example (4) shows how to combine matches-condition, which uses Teneo solutions to match Teneo Inquire data and Language Object expressions) with other constraints. For instance, find all Safetynet inputs that match the Language Object INSULTS.LIST in the English TLR:

(4)

tql

1lu t.e.userInput : t.e.type == "request",
2t.e2.fname ~= ".*[S|s]afetynet.*",
3t.e.userInput matches-condition (
4solution = "en_tlr",
5condition ="%INSULTS.LIST" )
6

Example (5) lists trending inputs that are matched by the Safetynet only:

(5)

tql

1lu t.e.userInput, tau, direction, p, z :
2	t.e.type == "request",
3	t.e2.fname ~= ".*[S|s]afetynet.*",
4	trend(min-tau="0", max-p="0.05") t.e.userInput as
5		    (tau, direction, p, z)
6	order by tau
7

Other TQL commands

Sub-queries

TQL supports sub-queries, where a complete TQL query can be nested and its results can be accessed in the selection part of a superordinate TQL query. The schematic form of the sub-query syntax is:

tql

1x = @( TQL query )
2

Example (1) below lists all session ids from a sub-query that results in a unique list of session ids from those sessions with a user input containing the word "good":

(1)

tql

1la result.id :
2result = @( lu s.id as id, t.e.userInput :
3t.e.type == "request",
4t.e.userInput ~= ".*good.*" )
5

Remark: It is not yet possible to use sub-queries as a set of constraints for exclusion.

Save and Source

Teneo Inquire supports two ways to save and export query results data: from the Teneo Studio frontend and using the save TQL command.

save stores query results using name as the data results identifier. When results have been saved, then the source command can retrieve those results using the parameter name and use them as the source data for another query.

For instance, query example (1) saves the results of the distribute command using the name myData:

(1)

tql

1save (name="myData") d date, t.e.userInput :
2	t.e.type == "request",
3	t.e2.fname ~= ".*[S|s]afetynet.*",
4	catd(model="date") s.beginTime
5	as date order by date asc
6

Note that the data saved using the above query contains two properties: the date and the user input. To make use of that data, you need to use the source command in combination with another TQL command. The example shown in (2) retrieves and distributes the user input, whereas example (3) retrieves and distributes both the user input and the date:

(2)

tql

1d (source="myData") t.e.userInput
2

(3)

tql

1d (source="myData") date, t.e.userInput
2

Remark: you may need to remember which variables were stored when the data was saved in order to provide correct constraints when using the source command. Note that it is not possible to simply write d (source="myData") and expect to get the date and userInput data.

Aggregate functions

TQL supports aggregation functions on saved data or sub-queries.

The aggregate functions are listed below:

CommandReturns
minminimum value
maxmaximum value
meanarithmetic mean
sumsum of values
numnumber of observations

Example (1) below returns the mean of the distribution of counts of user inputs per day from a sub-query, while example (2) returns the number of observations for a given date from a saved data set named "myData":

(1)

tql

1lu mean x.count as meanCount :
2	x=@( d date, t.e.userInput :
3	t.e.type=="request", catd(model="date") s.beginTime as date)
4

(2)

tql

1la (a="myData") num count, date : date == "2015-04-21"
2

Augmenters

Teneo Inquire Augmenters are applied to log data, either during import or once it has been imported, to enrich or summarize it in a format that can be later accessed by standard queries.

There are two types of Augmenters:

  • Adorners add new data properties to log data in a format that can be used along original properties
  • Aggregators are a pre-calculation of sum-ups of complex queries, which may have high computational costs on real time.

This section describes how to perform TQL operations with Augmenters. Note that Augmenters cannot be created using the Query interface in Studio, but the Data Manager Frontend. For further information on how to create and manage augmenters in Teneo Inquire, see Teneo Inquire: Data Manager.

Adorners

The adorn command adds new properties to sessions, transactions and events to facilitate simpler and faster queries. Adorner properties have the following syntax: <LEVEL.a.PREFIX:name>, where

  • LEVEL can refer to session s, transaction t, or event e
  • a stands for adorner
  • PREFIX is the data type, and
  • name (following the colon) is the adorner name chosen at creation time.

The example (1) below adorns sessions with the property issueComplete, a Boolean variable set to true where the meta data variable ISSUE_INFORMATION contains the string "Complete":

(1)

tql

1adorn s.a.b:issueComplete = true :
2	exists t.e.md:ISSUE_INFORMATION,
3	t.e.md:ISSUE_INFORMATION ~= ".*Complete.*"
4

Then, the issueComplete adornment can be queried the same way as any other property.

Example (2) shows how to use it in the selection construct to obtain all session ids that were completed, while example (3) shows how to use a as a constraint to retrieve all user inputs where the session was completed:

(2)

tql

1la s.id, s.a.b:issueComplete
2

(3)

tql

1la s.t.e.userInput:
2	exists s.a.b:issueComplete,
3	s.t.e.type == "request"
4
Aggregators

The aggregate command computes sum-ups over values of properties, grouped by date. Aggregations are stored aside the rest of the data, so they cannot be queried along them in a single query. Aggregations have a name (provided in the Log Data Manager frontend) and can generate one or more <key> along three additional properties: date, week, and count. The <key> property holds the list of items that matched the TQL query, count has the size of the items list, and date and week take the session date and week values.

Example (1) creates an aggregator named dailySafetynet with a single key dailySN holding the number of inputs that fall into a safetynet trigger per day:

(1)

tql

1aggregate s:dailySN = t.id :
2	t.e1.type == "request",
3	t.e2.fname ~= ".*[s|S]afetynet.*"
4

The syntax to query aggregated values takes the aggregator name as a mandatory parameter, and any of the other properties generated by the aggregator. Example (2) shows how to retrieve all the data stored by the aggregator, grouped by date by default. Example (3) shows how to obtain the same grouped by week, and example (4) obtains the total sum-up of the count variable:

(2)

tql

1la (a= "SNHits per day") date, week, count, SNHits
2

(3)

tql

1d (a= "SNHits per day") week, count
2

(4)

tql

1ca (a= "SNHits per day") count
2

Appendix: TQL cookbook

Lists

List all values of a variable

Problem: list all the values a variable takes on
Solution: use listUnique on the variable

tql

1lu t.e.sv:s:userCountry : exists t.e.sv:s:userCountry
2

Note: sv:s:userCountry is meant to be a string (s:) session variable (sv:) defined in the solution.

List unique inputs to a flow

Problem: list all the different (unique) inputs to a specific flow
Solution: use listUnique combined with a constraint on the flow name or flow id

tql

1lu t.e.userInput : t.e.type=="request", t.e2.fname == "User says yes"
2

List trending user input words

Problem: list user input words that are showing a significant positive or negative trend
Solution: use the trend transformer and set the cut off values to the appropriate values (recommended values below)

tql

1lu t.e.userInputWords, tau, direction, p, z :
2	t.e.type == "request",
3	trend (max-p="0.05", min-tau="0.2") t.e.userInputWords
4		as (tau, direction, p, z)
5	order by tau
6

Note: the trend algorithm is based on the Mann-Kendall trend test, which is a specific implementation of the general Kendall Tau correlation test

List variable values matching a pattern

Problem: list variable values that match a specific pattern, in the case of the example below: country names starting with a capital vowel
Solution: use regular expression matching on the variable value

tql

1lu t.e.sv:s:userCountry : t.e.sv:s:userCountry ~= ".[A|E|I|O|U].+"
2

Notes:

  • sv:s:userCountry is meant to be a string (s:) session variable (sv:) defined in the solution
  • a quotation mark may be part of a variable value, hence the leading dot in the regex above

List flows coming before safetynet

Problem: list flows that are coming immediately before a safetynet flow
Solution: use listUnique and skip-to for controlling the order events

tql

1lu t1.e.fname, t2.e.fname :
2	t1.e.pathType=="raise-flow",
3	t1.e-{pathType=="raise-flow"}>t2.e,
4	t2.e.fname ~= ".*[s|S]afetynet.*"
5

List review reason values

Problem: list the all the non-empty values of the review reason meta-data variable
Solution: use listUnique and regex constraint on the review reason variable

tql

1lu t.e.md:REVIEW_REASON : t.e.md:REVIEW_REASON ~= ".+"
2

Notes:

  • md:REVIEW_REASON is meant to be a metadata variable defined in the solution

List review reason values with user inputs and flow names

Problem: list the all the non-empty values of the review reason meta-data variable, alongside the user input and the flow name
Solution: use listUnique and regex constraint on the review reason variable, in combination with user input and flow name

tql

1lu t.e1.userInput, t.e2.fname, t.e3.md:REVIEW\_REASON :
2	t.e1.type=="request",
3	t.e1.userInput ~= ".+",
4	exists t.e2.fname,
5	t.e3.md:REVIEW_REASON ~= ".+"
6

Notes:

  • md:REVIEW_REASON is meant to be a metadata variable defined in the solution

List negative feedback comments

Problem: list the unique feedback comments that are accompanied by a negative rating
Solution: use listUnique with a constraint on the feedback rating variable

tql

1lu t1.e.md:FEEDBACK_COMMENT, t2.e.md:FEEDBACK_RATING :
2	t1.e.md:FEEDBACK_COMMENT ~= ".+",
3	t2.e.md:FEEDBACK_RATING ~= "negative"
4

Notes:

  • md:FEEDBACK, md:FEEDBACK_RATING, and md:FEEDBACK_COMMENT are meant to be metadata variables defined in the conversational AI application.

List user inputs matching a Language Object condition

Problem: list the user inputs that match a given Language Object or Language Object condition
Solution: use listUnique with the matches-condition transformer

tql

1lu t.e.userInput :
2t.e.type == "request",
3t.e.userInput matches-condition (
4solution="en_tlr", condition = "%WHO.FW.LEX + %BE.VB.LEX")
5

Notes:

  • the example above shows that the condition parameter accepts any valid combination of Language Objects and condition operators. However, the same condition could be rewritten using a single Language Object: WHO_IS.PHR
  • this query only works if a solution "en_tlr" has been associated with the log data.

List Language Objects matching user inputs

Problem: list the Language Objects matching every user input. This can be used to analyse how to improve conditions in the solution according to the matching LOBs.
Solution: use listUnique with the matching-lobs transformer

tql

1lu t.e.userInput, lob, used-words :
2	matching-lobs(solution="en_tlr") t.e.userInput
3		as (lob, used-words)
4

Notes:

  • this query only works fi a solution "en_tlr" has been associated with the log data.

List triggered flows per day from aggregated data

Problem: list the flows that have been triggered in a transaction for aggregated data
Solution: use listAll on the aggregated data

tql

1la (a="test") triggeredFlowsIdPerDay
2

Notes:

  • the example above assumes that an aggregator "test" has been created, with the properties triggeredFlowsIdPerDay which contains the relevant flow ids

List words matching pattern irrespective of case

Problem: list all user inputs containing "hello" irrespective of case
Solution: use listAll on userInputWords with the ~~ operator

tql

1la t.e.userInputWords :
2	t.e.type == "request", t.e.userInputWords ~~ "hi"
3

List inputs going to a particular trigger

Problem: list all inputs going to a particular trigger
Solution: use listAll and specify the name of the trigger, combined with skip-constraints

tql

1la t.e1.userInput, t.e2.vname :
2	t.e1.type=="request",
3	t.e2.vname=="Send SMS",
4	t.e1-{pathType=="flow-trigger"}>t.e2
5

Notes:

  • the example is based on a solution that retrieves inputs going to the Send SMS trigger in the flow Send SMS to X

List answers after a particular trigger

Problem: list all answers after a particular trigger has been activated
Solution: use listAll and specify the name of the trigger, combined with skip-constraints

tql

1la t.e1.vname, t.e2.answerText :
2	t.e1.pathType=="flow-trigger" ,
3	t.e1.vname=="Send SMS",
4	t.e1-{type=="response"}>t.e2
5

Notes:

  • the example is based on a solution that retrieves inputs going to the Send SMS trigger in the flow Send SMS to X

Count and averages

Count the number of dialogues

Problem: count the number of dialogues / sessions
Solutions: use countUnique or countAll on the session id

tql

1cu s.id
2ca s.id
3

Count user inputs

Problem: count all user inputs
Solution: use countAll on the user input (use countUnique to get types rather than tokens)

tql

1ca t.e.userInput : t.e.type=="request"
2

Count variable values

Problem: count the number of values found in a variable, excluding the default value
Solution: use countUnique in combination with a sub-query

tql

1cu x.city :
2	x=@(lu t.e.sv:s:userCity as city : t.e.sv:s:userCity ~= "...+")
3

Notes:

  • sv:userCountry is meant to be a string (s:) session variable (sv:) defined in the solution.

Count solution responses

Problem: count all outputs
Solution: use countAll on the output text (use countUnique to get types rather than tokens)

tql

1ca t.e.answerText :
2	t.e.type=="response",
3	exists t.e.answerText
4

Calculate mean transaction count for a time period

Problem: calculate the mean transaction count for a given time period
Solution: use the mean prefix in combination with a sub-query

tql

1lu mean x.count as meanCount :
2	x=@(lu s.id, s.transactionCount as count :
3		s.beginTime=="2015-03-11")
4

Calculate mean number of empty inputs for a time period

Problem: calculate the mean number of empty user inputs for a given time period
Solution: use the mean prefix in combination with a sub-query

tql

1lu mean x.count as meanEmptyCount :
2	x=@(d date, t.e.userInput :
3		t.e.type=="request", 
4		t.e.userInput == "",
5		catd(model="date") s.beginTime as date,
6		s.beginTime == in {"2015-03-11".."2015-03-17"})
7

Calculate mean number of results for aggregated data

Problem: calculate the mean number of observations per date for aggregated data Solution: use listAll in combination with the mean prefix

tql

1la (a="test") mean count, date
2

Notes:

  • this example assumes that an aggregator "test" with the properties date and count has been created

Frequency lists

To create top N lists from the solutions, use limit, e.g. limit 10 for a top-10 list (when sorted in descending order).

Most frequent user inputs

Problem: count the user inputs and sort in decreasing order
Solution: use distribute and order by count, specify request events

tql

1d t.e.userInput :
2	t.e.type=="request",
3	t.e.userInput ~= ".+" order by count desc
4

Most triggered flows

Problem: count the triggered flows and sort in decreasing order
Solution: use distribute and order by count, specify flow-trigger events

tql

1d t.e.fname :
2	t.e.pathType=="flow-trigger" order by count desc
3

Most frequent safetynet inputs

Problem: count the inputs going to a safetynet and sort in decreasing order
Solution: use distribute and order by count, combined search for request and flow-trigger events

tql

1d t.e.userInput :
2	t.e.type=="request",
3	t.e.userInput ~= ".+",
4	t.e2.pathType == "flow-trigger",
5	t.e2.fname ~= ".*[s|S]afetynet.*" order by count desc
6

Frequency of variable values

Problem: count the values of a variable and sort in decreasing order
Solution: use distribute and order by count, applied to variable

tql

1d t.e.sv:s:userCountry :
2	exists t.e.sv:s:userCountry
3	order by count desc
4

Notes:

  • sv:s:userCountry is meant to be a string (s:) session variable (sv:) defined in the solution
  • the default value of the variable (such as "") is also listed. Use regex matching to require a longer string if "" is to be excluded

Frequency of flows triggered by a specified input

Problem: count the flows that are triggered by a specific input and sort in decreasing order
Solution: use distribute and order by count, in combination with skip-to

tql

1d t.e1.userInput, t.e2.fname :
2	t.e1.type=="request",
3	t.e1.userInput ~= ".*hungry.*",
4	t.e1-{pathType=="flow-trigger"}>t.e2 order by count desc
5

Notes:

  • returns frequency list of user inputs containing "hungry" and triggered flows

Frequency of flow triggers

Problem: count the number of times flow triggers have been activated and sort in decreasing order
Solution: use distribute and order by count, specify flow-trigger events

tql

1d t.e1.fname :
2	t.e1.pathType=="flow-trigger",
3	exists t.e1.fname order by count desc
4

Frequency of sessions per day or week

Problem: count the number of sessions per day or week in a time period and sort chronologically by date
Solution: use distribute and order by date asc

tql

1d date :
2	catd(model="date") s.beginTime as date
3	order by date asc
4

tql

1d week :
2	catd(pattern="yyyy-'w'ww") s.beginTime as week
3	order by week asc
4

Frequency of sessions with an input triggering safetynet by day

Problem: count the number of sessions per day, where at least one input has ended up in the Safetynet
Solution: use distribute to count sessions, with a constraint on the flow name

tql

1d date :
2	catd(model="date") s.beginTime as date,
3	t.e1.type=="request",
4	t.e2.fname~=".*[s|S]afetynet.*"
5

Frequency of triggered flow folders

Problem: count the number of times the flows from various folders have been triggered and sort in decreasing order
Solution: use distribute with folder

tql

1d t.e.folder :
2	t.e.pathType=="flow-trigger",
3	t.e.folder ~= ".+" order by count desc
4

Frequency of traversed safetynet transitions

Problem: count the number of times the different transitions in the safetynet flow(s) have been logged, and list the names and ids of the transitions sorted in decreasing order
Solution: use distribute with the transition path type

tql

1d t.e.vname, t.e.vid :
2	t.e.pathType=="transition",
3	t.e.vname ~= ".+",
4	t.e.fname ~= ".*[S|s]afetynet.*" order by count desc
5

Frequency of review reason values

Problem: count the number of non-empty values of the review reason metadata variable, and sort in decreasing order
Solution: use distribute with a regex constraint on the review reason variable

tql

1d t.e.md:REVIEW_REASON :
2	t.e.md:REVIEW_REASON ~= ".+" order by count desc
3

Notes:

  • md:REVIEW_REASON is meant to be a metadata variable defined in the solution

Frequency of feedback rating

Problem: count the number of different feedback ratings, and sort in decreasing order
Solution: use distribute on the feedback rating variable

tql

1d t.e.md:FEEDBACK_RATING :
2	t.e.md:FEEDBACK_RATING ~= ".+" order by count desc
3

Notes:

  • md:FEEDBACK_RATING is meant to be a metadata variable defined in the solution

Frequency of inputs matching A OR B

Problem: count user inputs matching one of two possible values (or more)
Solution: use distribute in combination with set constraints to emulate a logical OR condition

tql

1d t.e.userInput :
2	t.e.type == "request",
3	t.e.userInput ~= in {"A", "B"}.
4