Customers often ask us to process their inner normative database for the ChatBot to search it for answers to questions. Today we’ll consider a lifehack: how to transform a regular text into intellectual.
Let’s consider the first page of the document (here: bank regulation):
«1. Priority in operations
1.1. General provisions
1.1.1. The duration of the operational day for servicing the Customers is established by the Branch Office order.
1.1.2. The bank takes Customer’s monetary funds to bank accounts, call and savings deposits, deposit accounts in the currency of the Russian Federation and in foreign currency. The list of transactions against bank/ deposit accounts of the appropriate type (account mode) is governed by the laws of the Russian Federation and the Agreement. The Bank has the right to refuse to conduct an operation on a bank / deposit account if it does not correspond to the account regime (mode) established by the Agreement. Transactions against bank/ deposit accounts are made by both the Customer and his Representative in the manner established by the legislation of the Russian Federation and the Agreement.
1.1.3. In accordance with clause 5 of Article 7 of the Federal Law No. 115-FZ, credit organizations may not open bank / deposit accounts for individuals without the personal presence of the depositor opening the bank / deposit account or his representative.
1.1.4. Bank account opening and keeping, account transactions without opening an account shall be made by the Bank provided that the Customer is legally capable.”
After text processing we’ll upload the processed text onto the ChatBot created on the basis of Botlibre. So we use the syntax offered by Botlibre.
https://www.botlibre.biz/manual-chatlogs.jsp
One of the possible ways to process the text:
1. The order, numeration, in particular, may be ignored. It’s not important for intellectual search if the user looks for p.1.3 or right away p.1.4.5.
2. We create a topic and write it below every question-answer pair, until a new topic is found.
Our present topic: General provisions
(this title is to be written down below every question-answer pair)
3. Then we create a question-answer pair; to make it easier, you can use 3-8 words of the answer to the first question.
The duration of the operational day for servicing the Customers
The duration of the operational day for servicing the Customers is established by the Branch Office order.
topic: General provisions
4. Now we write keywords (nouns, verbs, adverbs) without prepositions, pronouns, adjectives; and as the result we have:
The duration of the operational day for servicing the Customers
The duration of the operational day for servicing the Customers is established by the Branch Office order.
topic: General provisions
keywords: duration operational day servicing customers
5. We add concepts and meanings to the keywords and get the following text:
The duration of the operational day for servicing the Customers
The duration of the operational day for servicing the Customers is established by the Branch Office order.
topic: General provisions
keywords: duration operational day how long last hours working servicing customers transaction performance operation
6. Now it’s important to delete doubles (one and the same synonym to different words). So we do and add necessary word forms
The duration of the operational day for servicing the Customers
The duration of the operational day for servicing the Customers is established by the Branch Office order.
topic: General provisions
keywords: duration operational operation day days how long last lasts hour hours work working servicing service customers customer transaction transactions performance performances perform operations
7. The first question-answer pair is ready!
8. We make up the second question-answer pair; if there are numbered or bulleted lists (bullets are to be replaced by numbers observing the sequence numbering), we repeat steps 2-6 and get the following result:
Monetary funds to bank accounts
The bank takes Customer’s monetary funds to bank accounts, call and savings deposits, deposit accounts in the currency of the Russian Federation and in foreign currency. The list of transactions against bank and deposit accounts of the appropriate type (account mode) is governed by the laws of the Russian Federation and the Agreement. The Bank has the right to refuse to conduct an operation on a bank / deposit account if it does not correspond to the account regime (mode) established by the Agreement. Transactions against bank and deposit accounts are made by both the Customer and his Representative in the manner established by the legislation of the Russian Federation and the Agreement.
topic: General provisions
keywords: money, monetary funds bank account accounts call deposit deposits etc.
9. It’s necessary to add to the question-answer pair one more character, called previous. It includes the full previous answer of the ChatBot.
Monetary funds to bank accounts
The bank takes Customer’s monetary funds to bank accounts, call and savings deposits, deposit accounts in the currency of the Russian Federation and in foreign currency. The list of transactions against bank and deposit accounts of the appropriate type (account mode) is governed by the laws of the Russian Federation and the Agreement. The Bank has the right to refuse to conduct an operation on a bank / deposit account if it does not correspond to the account regime (mode) established by the Agreement. Transactions against bank and deposit accounts are made by both the Customer and his Representative in the manner established by the legislation of the Russian Federation and the Agreement.
topic: General provisions
keywords: money, monetary funds bank account accounts call deposit deposits etc.
previous: The bank takes Customer’s monetary funds to bank accounts, call and savings deposits, deposit accounts in the currency of the Russian Federation and in foreign currency. The list of transactions against bank and deposit accounts of the appropriate type (account mode) is governed by the laws of the Russian Federation and the Agreement. The Bank has the right to refuse to conduct an operation on a bank / deposit account if it does not correspond to the account regime (mode) established by the Agreement. Transactions against bank and deposit accounts are made by both the Customer and his Representative in the manner established by the legislation of the Russian Federation and the Agreement.
Thus, the context is retained, and the user can go from one point to another.
Because of the heuristic structure of the ChatBot memory the user may not know exactly the content of the next or previous bot response, so we suggest that the numerical help (numeration) of the user’s location in the document should be used, for which we add a duplicate question-answer pair where instead of the question we use its number, enclosed in square brackets, and add the same number to the answer at the end); delete the points (they are logical periods).
10. Add a duplicate question-answer pair connected by means of the exact numeration. Add the number to the response
[112]
Monetary funds to bank accounts
The bank takes Customer’s monetary funds to bank accounts, call and savings deposits, deposit accounts in the currency of the Russian Federation and in foreign currency. The list of transactions against bank and deposit accounts of the appropriate type (account mode) is governed by the laws of the Russian Federation and the Agreement. The Bank has the right to refuse to conduct an operation on a bank / deposit account if it does not correspond to the account regime (mode) established by the Agreement. Transactions against bank and deposit accounts are made by both the Customer and his Representative in the manner established by the legislation of the Russian Federation and the Agreement. [112]
topic: General provisions
keywords: money, monetary funds bank account accounts call deposit deposits etc.
previous: The bank takes Customer’s monetary funds to bank accounts, call and savings deposits, deposit accounts in the currency of the Russian Federation and in foreign currency. The list of transactions against bank and deposit accounts of the appropriate type (account mode) is governed by the laws of the Russian Federation and the Agreement. The Bank has the right to refuse to conduct an operation on a bank / deposit account if it does not correspond to the account regime (mode) established by the Agreement. Transactions against bank and deposit accounts are made by both the Customer and his Representative in the manner established by the legislation of the Russian Federation and the Agreement.
Conclusion
Thus, in addition to the heuristic structure of the questions and answers, the logical one is created so that the user could go from one part of the document to another. All the question-answer pairs are composed in the same way.
At the end the help command is made, where the content of all the parts of the document is described, so that the user could scan the contents.
To solve this task we used the free Paraphraser service, which also has a more complicated commercial version with a more complicated processing.
About the service
https://paraphraser.ru/api/form
The text processing program is written in Python.
