https://discord.gg/botpress logo
Join Discord
Powered by
Incorrect information
# 🤝help
f
*My AI task is giving me information that is not located on my KB. * **When I ask about a service such as teeth whitening it says it costs $200 when it is supposed to be $299. I don't know where it is getting this information from. It used to work. But it seems the more complicated I made the bot it just doesnt work anymore. ** __**This is its task instructions:**__ Input: @workflow.KBanswer Task: You are a helpful sales expert for a dental practice. Your task is to take the input and convert it into a concise answer/output for the potential client by following the rules below. Rules: - If the input is related to a service, then follow the rules listed under "Service rules". - If the input is unrelated to a service, then follow the rules listed under "s1 rules". Service rules: 1. Include the price of the service in your output. 2. Encourage the client to make a booking for the service. 3. Hyperlink the booking page using this format [text](link). s1 rules: 1. Choose a service within the "Dental services" knowledge base that is most relevant to the input and include this in your output. 2. In 1 sentence suggest the service chosen in rule 1 to the client in an enticing tone. 3. Include the price of the service. 4. Hyperlink the booking page using this format [text](link). __**I am using: **__ - Query knowledge base (manual agent enabled) - AI task with input variable from query knowledge base. My KB for "Price list" description: "Price list" is a knowledge base that accurately lists all the prices for all the services that the dental practice offers. This knowledge base provides a detailed list of services and their corresponding price. The chatbot will prioritise this knowledge base over all other knowledge bases when referencing a price. This knowledge base can answer questions like: What is the price for veneers? What is the price for a clean-up? What is the price for a check-up?

https://cdn.discordapp.com/attachments/1145642290224832602/1145642290426171392/1.png▾

https://cdn.discordapp.com/attachments/1145642290224832602/1145642290682019881/2.png▾

When I ask it directly what the price is, it is correct. However if I ask about a service it will give incorrect information:

https://cdn.discordapp.com/attachments/1145642290224832602/1145646914340143187/image.png▾

b
I have the same problem. I am including the KB in the AI task (instructions). If I ask for information not included in the KB, sometimes the AI makes up an answer. I even included this in my prompt: (You must not, by any means, create an answer to the user's question. All your responses must come from the knowledge base.). Still, it does the same thing. I think the problem is from the AI side of things (probably a bug), and I don't know if there is a way to work around it. Maybe a staff member can help us.
a
Hey @famous-raincoat-58603, AI Tasks don't inherantly know what is in the knowledge base or not, so you need to pass a list of the available knowledge base sources into the AI Task Input
You might also want to consider storing the prices in a table. GPT models are pretty weak on math, but you can use code to look up and find the exact prices from a table
b
Isn't it better to include the KB in the AI task for this reason?
a
If you include the entire KB in an AI task, that's what the Knowledge Agent is 😁
But there are other ways to have fun with AI task & KB sources, depending on what you want to do
Looking at your prompt, it could be worthwhile to split your AI task into two diferent tasks
One to either extract the service or guess the most relevant service
and then another to take that service draft and draft a Knowledge base question that could answer the user's question
b
That's what I am doing as well. I have an AI task to understand the intent of the user, and another to answer their question if the intent is not matched by the other possibilities.
But regarding the problem with giving made up answers
This is my prompt: You are a helpful AI assistant from Hotel. You receive chunks of documents that help you answer User questions. When a link is given in the knowledge base, provide the link in your answer. Always answer succinctly in no more than 2-3 sentences, and always try to provide the answer in a way that makes Hotel sound like the perfect hotel. If the user asks a question about the hotel you don't have an answer to, you must reply with "I don't know.". These exact words must be sent as your response. You must not, by any means, create an answer to the user's question. All your responses must come from the knowledge base. Consider yourself as a part of the staff of Hotel, and thus say "we" in your messages. You have full access to the conversation so far. You should be able to answer a question that is a follow up of the previous one.
@acceptable-kangaroo-64719 can you take a look please?
a
Are you including the knowedge base results as inputs for this AI Task?
b
No, i have the KB text below this prompt
It is in a Q&A format
a
That must be a pretty long AI Task prompt, then
Are you doing any kind of KB search before running this task?
b
I am just using the AI task for the KB
I was experimenting with having a KB agent to return the answer but it doesn't run as smoothly.
So I kept it this way
a
Gotcha. You can experiment with some prompt engineering techniques to try and get the bot to use the knowledge instead of making things up: * Tell the model to first find a relevant chunk, than give an answer * Tell the model to back up its answer with facts from the document * Tell the model to understand the user's question, then find a relevant chunk, then draft an answer
b
Okay, I will follow the steps. Also, I tried using the same prompt in GPT, and it works as intended.
It is probably a bug with the API or something
I changed the prompt: You are a helpful AI assistant from ------- Hotel. Your job is to understand the user's question and find a relevant chunk in the knowledge base below. Then, double-check with the rest of the knowledge base to see if anything else is relevant. Finally, create an answer using the instructions below. If the user asks a question about the hotel you don't have an answer to, you must reply with "I don't know.". These exact words must be sent as your response. Instructions: 1) When a link is given in the knowledge base, provide the link in your answer. 2) Always answer succinctly in no more than 2-3 sentences, and always try to provide the answer in a way that makes ------- Hotel sound like the perfect hotel. 3) You must not, by any means, make up an answer to the user's question. You will say "I don't know", if you don't find the answer. All your responses must come from the knowledge base. 4) If the user asks a question about the hotel you don't have an answer to, you must reply with "I don't know.". These exact words must be sent as your response. Consider yourself as a part of the staff of -------- Hotel, and thus say "we" in your messages. 5) You have full access to the conversation so far. You should be able to answer a question that is a follow up of the previous one. 6) If the answer to "How can I help you?" or "Is there anything else I can help you with?" is no, end the conversation.
(I had to remove the name of the hotel for their privacy)
It still answers incorrectly
In the KB there is no mention of room types, when I ask it about the room types they offer, it says: We offer a variety of room types including standard rooms, deluxe rooms, and suites. Each room is designed to provide comfort and relaxation for our guests. You can find more information about our room types on our website:
a
If you go into advanced settings on the AI task, is the temperature at 0?

https://cdn.discordapp.com/attachments/1145642290224832602/1145681725347856465/image.png▾

b
Yes
And version 1
a
And, just to be clear, after step 6 in your prompt you have a bunch of question-answer pairs about the hotel, right?
b
Yes
Where there is no mention of the room types
a
It might also be helpful to include an example of the bot both answering a question and saying "i don't know" to a question
b
Okay, I will add this and test
Do you mean in the prompt or the build in task example?
a
in the prompt
b
Okay I will try it
I added the examples, still it changes nothing
Is there another way to work around this?
a
the other way would be trying to get the knowledge base to work
b
Okay, I will experiment with it and send a message if I have a problem. Thanks for your help so far.
12 Views
PreviousNext
Powered by