Teneo Developers

Lexical Resources

Teneo NLU Ontology and Semantic Network

Natural language comes in many variances and varieties, with some expressions being more commonly used than others which can make it cumbersome to model each dialogue without some basic building blocks to get started.

The Teneo NLU Ontology and Semantic Networks are Artificial Solutions' proprietary Lexical Resources; these contain off-the-shelf building blocks - Language Objects and Entities - to be used for building Natural Language Interaction (NLI) solutions in Teneo Studio.

The NLU Ontology and Semantic Networks provide an extensive collection of Language Objects and Entities with a comprehensive amount of wordforms and synonyms which ensure a high natural language coverage. Moreover, using Language Objects and Entities in Match Requirements enhances the clarity and consistency of the Match Requirements and facilitates maintenance and later enhancements.

While the Lexical Resources of the NLU Ontology and Semantic Networks are mostly designed to cover general language vocabulary and common phrases, Teneo Studio provides the possibility to create additional lexical resources per project needs. Project-created lexical resources can be re-used in other projects, different conversational AI applications or at later project stages within the same domain just as the Teneo NLU Ontology and Semantic Networks.

The names of the Language Objects and Entities in Artificial Solutions' Lexical Resources are usually transparent and intuitive, but it can nevertheless become necessary to search among the available objects to, for example, know their exact names or to verify whether a desired object exists.

Working with lexical resources

Teneo Studio provides different options to search for or discover existing Language Objects and Entities: the Auto-Complete and Condition Building Assistant available in the Condition Editor, the Search functionality of Teneo Studio.

On top of this, Teneo Studio also provides an option to draft Condition Match Requirements automatically, which can be used to get condition suggestions which can later be fine-tuned by the Studio user.

Language Object and Entity descriptions

Each Language Object and Entity have a description field which, in Teneo Lexical Resources, is typically used to indicate the coverage and intended usage of the given Language Object or Entity.

The description field is visible either by opening the object or by hoovering over the object name, for example in the condition editor or list of search results.

Users building lexical resources are recommended to add descriptions so that other users can easily understand the scope of the object.

Generate condition suggestions

To get started using an assigned lexical resource, one can start by adding the positive examples which the Intent Trigger should cover, and then use the option to draft a condition (Match Requirements Add > Condition > Advanced options > Draft Condition) and then, based on the suggested condition re-adjust as per project needs.

For example, imagine a Flow in a solution where the English Lexical Resource is assigned, which recommends products to the customer, and the examples should cover the below variations of how to ask for recommendations:

Do you know any good (...)
Recommend a (...)
Can you recommend (...)
What can you recommend (...)
Would you recommend me (...)

Based on the above examples, the Draft Condition functionality suggests a condition with the Language Object WOULD_YOU_RECOMMEND.PHR, which covers an even wider range of ways of expressing questions related to recommendation.

Although the above example is rather simple, very complex conditions can be built this way and the user is afterwards able to fine-tune the condition to cover each specific project need.

Auto-complete

The user can search for the name of Language Objects or Entities in the Condition Match Requirement's editor when the name of the object is already known; this is done by typing the beginning of the object's name in the editor (preceded by the % sign) and pressing Ctrl+space on the keyboard.

This action opens a dropdown suggestions menu which displays available Language Objects and Entities that share the prefix which was typed so far.

Auto-complete suggestions for "%Lis"

If the given suggestion is too wide, further typing will incrementally reduce the options on display.

Hovering over an object in the suggestion list will display further information of the given object and thus allows to verify whether it is the right object for the current purpose without having to open it.

Once the desired object has been found, simply click on it and it will automatically be added to the editor.

Condition Building Assistant

When a Language Object or Entity containing a specific term is unknown, it is possible to search for the objects based on bare words; the Condition Building Assistant allows the user to type the bare word(s) which should be covered by either a Language Object or an Entity, and to do this, the user should simply starts writing the word in the editor and presses Ctrl+space.

When writing more than one word, the words need to be selected before clicking Ctrl+space for the Condition Building Assistant to suggest objects for all the words.

This action opens a dropdown suggestions menu which displays available Language Objects and Entities that contain the selected word(s).

The suggestion menu includes information about the word(s) searched for, and hovering over an object in the suggestion list will display further information of the given object and thus allows to verify whether it is the right object for the current purpose without having to open it.

Condition Building Assistant suggestions for "i want"

Once the desired object has been found, simply click on it and it will automatically be added to the editor.

Customization of Language Objects

The Language Objects and Entities in the Lexical Resources of the Teneo NLU Ontology and Semantic Networks are designed to cover general language vocabulary and common phrases and when working in a project it is often needed to create more project-specific Language Objects/Entities.

Both the objects coming from a Teneo Lexical Resource, other lexical resources, and project-specific objects located in the project's solution can be used and reused within a solution; and not only is it possible to create new Language Objects or Entities, but it is also possible to a certain extend to customize Language Objects or Entities coming from a lexical resource.

The benefit of using the approach where objects from the assigned lexical resources are reused within project-specific objects are many, but most importantly when new versions of a lexical resource is available any update from the new version will automatically be propagated through to the project-specific solution's objects and hence the project-specific solution will benefit from any improvements performed in the updated versions of the lexical resources.

Adapting, creating and modifying Language Objects

Despite the broad coverage, the Teneo Lexical Resources or other assigned lexical resources may not always cover all the words and concept required for certain domains. In these cases, new Language Objects or Entities can be created in the project's solution.

Other times a user might encounter that a Language Object or Entity coming from an assigned lexical resource doesn't cover all the terms and concepts needed for the specific project and will need to extent the coverage of the existing object.

For example, the English Teneo Lexical Resource contains the Language Object PLANETS.LIST with the following language condition:

tlml

1%MERCURY.NN.LEX /
2%VENUS.NN.LEX /
3%EARTH.NN.LEX /
4%MARS.NN.LEX /
5%JUPITER.NN.LEX /
6%SATURN.NN.LEX /
7%URANUS.NN.LEX /
8%NEPTUNE.NN.LEX /
9%PLUTO.NN.LEX
10

Now, imagine wanting to use a Language Object with the above list of planets but also Ceres, Eris and Sedna. To do so, the below steps can be followed:

  • First, the corresponding NN.LEX Language Objects should be created for Ceres, Eris and Sedna with the following Language Object names and conditions:

tlml

1CERES.NN.LEX ceres/ceres'
2ERIS.NN.LEX eris/eris'
3SEDNA.NN.LEX sedna/sednas'
4

To see how to create new Language Objects please follow the link.

  • When the above NN.LEX Language Objects are created, yet another Language Object should be created with the name, for example, PROJECT_SPECIFIC_PLANETS.LIST and the following condition:

tlml

1%PLANETS.LIST /
2%CERES.NN.LEX /
3%ERIS.NN.LEX /
4%SEDNA.NN.LEX
5

It is common practice to add a project-specific prefix to Language Objects (in above examplePROJECT_SPECIFIC) which belong to a certain domain within a project; this facilitates the retrieval and maintenance of the Language Objects and make them easier to identify and find when using them throughout the solution.

Following the above steps and using the Teneo Linguistic Modelling Language almost any kind of adaptation and modifications can be made to Language Objects and Entities coming from the assigned lexical resources to have the objects covering the project-specific needs.

Create a lexical resource

Along with the Lexical Resources of the NLU Ontology and Semantic Networks that the solution can use, the users are also able to create their own project specific lexical resources, which gives the opportunity to share and re-use often needed Language Objects and Entities.

This is done by creating the needed Language Objects and/or Entities in a solution and then exporting the solution as a lexical resource (.library file) as this functionality only exports the Language Objects and Entities residing in the solution (not Flows).

The steps to follow are outlined below:

Teneo NLU Ontology and Semantic Network