Teneo Developers

Automated testing

Positive and negative examples

Teneo Studio allows performing automatic tests in order to ensure that the proper Flow Triggers and Transitions are triggered by the given input. The Auto-test uses the positive and negative examples added to the Triggers and Transitions in order to test the flows, so before running an Auto-test, examples should be added to the Flows.

Run an Auto-test

To see and run an Auto-test, follow the below steps:

  • Go to the backstage of Teneo Studio (Solution tab)
  • Select Auto-test in the left-side menu
  • In the top of the Test page, click Run Test to run an Auto-test with the default settings.

Defining Auto-test settings

In the top of the Test page, it is possible to define the settings of the Auto-test to run.

The first three icons allows the user to specify which elements to include in the test: Triggers, Transitions, and/ Links. By default, all three elements are pre-selected. To deselect any, simply click the icon.

Predefined selection

Auto-test levels

Auto-tests can be done at three levels:

  • Flow (note that this test level is performed within the Flow)
  • Folder
  • Solution

The Solution button at the top of the Test page allows the user to browse for a specific folder and define the level of the test.

The Run Test button allows the user to choose between two different test options:

  • Run Test and
  • Run Test Using Flow scope

Run Test Using Flow Scope

When selecting Run Test, the Intent Triggers and Transitions in the scope (Flow, Folder or Solution) are tested in the following ways:

  • If the Positive examples match the Match Requirements of the Intent Trigger they belong to,
  • If the Negative examples does not match the Match Requirements of the Intent Trigger they belong to, and
  • if the Intent Trigger is actually triggered for the examples that match the Match Requirements, or
  • if the example(s) "is stolen" by another Intent Trigger with a higher ranking in Ordering.

When selecting Run Test Using Flow Scope, which is particularly useful when working with Match Requirements to ensure that these covers the added examples, the test performed checks:

  • How the Intent Triggers perform as a stand-alone, and
  • all other Intent Triggers in the solution are ignored.

Auto-test at Flow level

Follow these steps to set up and run an Auto-test at Flow level:

  • Open the Flow to be tested
  • Select the items to be included in the test: Intent Triggers - Transitions - URLs
  • Click Run Test or Run Test Using Flow Scope at the top of the view.

When the Auto-test has finished, the results can be filtered by Succeeded, Failed, Succeeded with warnings or by Non-testable items.

Auto-test at Folder level

Follow one of the below options to set up and run an Auto-test at Folder level:

  • Either, in the Solution Explorer, right-click the folder to test and select Auto-test in the context menu, or
  • go to the Auto-test page in the backstage of Teneo Studio (Solution tab > Auto-test)
    • Select the items to be included in the test: Intent Triggers - Transitions - URLs
    • Click the Solution button to open the drop-down menu and select Browse for folder
  • In the Folder Browser window, select the folder and click OK
  • Lastly, click Run Test or Run Test Using Flow Scope in the top of the Auto-test page.

Auto-test at Solution level

To test all Intent Triggers, Transitions and URLs set up in the solution, follow these steps:

  • Go to the Auto-test page in the backstage of Teneo Studio (Solution tab > Auto-test)
  • Select the items to be included in the test: Intent Triggers - Transitions - URLs
  • Lastly, click Run Test.

Use Stable Documents

The Use Stable Documents/Use Latest Documents refers to Version Flags and allows to choose between running Auto-test on all documents in the solution marked as stable or on the latest version of all documents within the solution.

Auto-tests run on the stable documents are marked with a Tested using stable versions flag in the top of the test results.

Cancel an Auto-test

As soon as an Auto-test has started, the Cancel test button appears in the page. The user can, by clicking it, cancel the run of the Auto-test. Note that the cancellation can take a little as it needs to be communicated to the server for the Auto-test to be cancelled.

Cancel test

Status of the Machine Learning Model

In the top of the Auto-test page, a Model was not ready during this test message may be displayed after starting an Auto-test. If this is the case, cancel the Auto-test and wait for the model training to finish to run the Auto-test to obtain accurate test results.

Model was not ready during this test

Review former Auto-test results

The History panel, at the bottom of the Auto-test page, displays a summary of former Auto-tests. It is possible to Open any of the previous Auto-tests and Reload the list by using the available buttons.

History section

Export test report

The user can export the Auto-test results in Excel format. To do so, simply click the Get Report button available in the top of the Auto-test page.

For further information on how to review the test results, please see the Auto-test results panel section.

Machine Learning in Teneo
Version Flags