How SoftwareOne uses AWS language technologies to build Intelligent Decision Support Systems

Components of an Intelligent Search Engine

An IDSS tool is based on an intelligent search engine (ISE), which ISE offers natural language and semantic understanding of queries, going beyond simple keyword-based searching. Questions can be asked in natural language, and results can be more relevant to the user. Figure 1 shows a high-level diagram of an IDSS based on an intelligent search engine.

Data is ingested into the system by crawling various sources, such as news agencies, blogs, portals, reports, and social media. The ingested unstructured text is pre-processed and enriched to extract important information that assists the answering process later. This can include classifying documents into different categories and extracting relevant entities or metadata.

The enriched documents are then indexed into the ISE, and users can ask questions in natural language and receive direct answers or relevant excerpts that are related to their questions. Users can also provide feedback on the relevance of the answers (via thumbs up/down, for example) and allows the ISE to improve over time.

Solution requirements and approach

Following a similar approach to Figure 1, the Savills team needed to ingest daily CommSights news articles, in Vietnamese, covering different categories, including commercial leasing, industrial, residential, and disaster news.

The team needed to differentiate between categories and required a filtering mechanism that would enable users to quickly narrow down their search to any of these topics. They also required the search to be conducted using natural language to quickly and seamlessly provide the necessary insights to users.

To address these requirements, SoftwareOne opted to base its IDSS on Amazon Kendra, an intelligent search service that uses natural language processing (NLP) and advanced machine learning (ML) algorithms to return specific answers to search questions from data. Unlike traditional keyword-based search, Amazon Kendra uses its semantic and contextual understanding capabilities to decide whether a document is relevant to a search query. It returns specific answers to questions, giving users an experience that’s close to interacting with a human expert.

To categorize the incoming documents into different categories, SoftwareOne used Amazon Comprehend, which leverages NLP to extract insights about the content of documents. It develops insights by recognizing entities, key phrases, language, sentiments, and other common elements in a document. Using Amazon Comprehend, SoftwareOne trained a custom document classifier using a set of categorized historical documents.

The classifier exhibits an F1 score of 95% and can classify new incoming documents to different key topics. The output of the document classifier (class metadata) is included in Amazon Kendra, providing users with filtering options to easily narrow down their search to a specific topic.

For example, if the Savills commercial leasing team needs to know more about the latest commercial leasing news, it can narrow down its search to only this particular document category by utilizing the built-in filtering functionality of Amazon Kendra.

To translate the ingested Vietnamese documents to English, SoftwareOne used Amazon Translate, a text translation service that uses advanced ML technologies to provide high-quality translation on demand.

Implementation

Figure 2 shows a diagram of the IDSS architecture that SoftwareOne implemented for the customer.

In the first stage, the ingested documents go through a pre-processing and enrichment phase, before they are ingested to Amazon Kendra. An AWS Lambda function acts as an orchestrator during this phase.

1. Incoming documents are passed to Amazon Translate to be translated from Vietnamese to English. The translated documents are stored in an Amazon Simple Storage Service (Amazon S3) bucket.
2. The translated documents are also classified into different topics by a custom document classifier in Amazon Comprehend, which has been trained on historical documents across different categories. The extracted metadata (document topics) are also stored in the same S3 bucket.
3. The English documents, along with their extracted metadata, are ingested periodically to an Amazon Kendra index. Kendra offers a large number of built-in connectors including Amazon S3. Periodic ingestion of new documents can be easily scheduled from within Kendra based on the frequency of the new incoming documents.
4. In the next retrieval phase, users authenticate through Amazon Cognito and use the IDSS web application. Through the search user interface (UI), they can submit questions in natural language and narrow down their search by filtering specific topics of interest. Each question is propagated through Amazon API Gateway and Lambda to Amazon Kendra. Based on the filtered topics and the question, Amazon Kendra will either return a specific answer or multiple relevant text excerpts from the knowledge base. Optionally, a large language model (LLM) can be used to further improve the answers returned by Kendra. This could include summarization of all the text excerpts, or synthesizing an answer to the asked question, by using the consolidated text excerpts as context (similar to what you can learn in this AWS blog post). The LLM can be either hosted in an Amazon SageMaker endpoint, using SageMaker JumpStart, or interfaced through an API using Amazon Bedrock. The final answer (either directly from Amazon Kendra or from the LLM) is returned to the user.
5. Answer generation:
1. Based on the filtered topics and the question, Amazon Kendra will either return a specific answer or multiple relevant text excerpts from the knowledge base.
2. Optionally, an LLM can be used to further improve the answers returned by Kendra. This could include summarization of all text excerpts or synthesizing an answer to the asked question by using the consolidated text excerpts as context (similar to what you can learn in this AWS blog post). The LLM can be hosted in an Amazon SageMaker endpoint, using SageMaker JumpStart, or interfaced through an API using Amazon Bedrock.
3. The final answer (either directly from Kendra or from the LLM) is returned to the user.
6. Optionally, an Amazon QuickSight interactive dashboard can be used to display consolidated statistics about the ingested documents (topic ratio of the ingested documents) or search-related analytics that Amazon Kendra offers natively (click-through rate, top queries, top clicked documents, total queries).

As an alternate option to the pre-processing and enrichment phase, instead of using Amazon Translate and Amazon Comprehend, an LLM could be used to translate from Vietnamese to English and classify documents into different topics, thus simplifying the architecture.

The two figures below depict an example of the actual IDSS system built for Savills. After a user is signed in, they can ask any question in the main search box.

On the left, they can filter results across different topics (which were extracted during the enrichment phase). In the main part of the page, relevant documents (Figure 3a) or a direct suggested answer (Figure 3b) will appear. Users can click the title of each retrieved result and be taken to the original document.

Below each item, thumbs up/down icons allow the users to indicate whether this particular result is helpful or not. This direct feedback is important because it closes the loop, allowing Amazon Kendra to improve its responses over time.