https://www.loom.com/share/bc785e8ba169480392fb56bc7e2ca66c?sid=7a744eab-48d0-4ccd-bf3b-a79150029bcb
See video for demonstration.
Problem: Knowledge base will answer differently for documents compared to static text. When responding from docs, it is much more likely to be wrong. I'm positive that the way that documents are being pulled in must be messing with the text.
Example: The example from the video shows KB set up with both a text doc and a static text doc. It is the same exact text. I disable th KBs to test them individually.
Real Answer from text: "The Status Effect Cleanse is a Unique Type Effect with a Stack Limit of 3 that does the following: Spend to remove one status effect token: A player with this token may spend it at any time to remove a single status effect token from themselves. Cleanse belongs to the following characters: Monk, Seraph"
When Using Text Doc
User Question: What does the status effect cleanse do?
Bot Answer: The status effect "Cleanse" allows a player to remove a specific number of other status effects from themselves or another player. When using cards or abilities that remove a specific number of status effects, the player can remove that many tokens of status effects.
When Using Static Text
User Question: What does the status effect cleanse do?
Bot Answer: The status effect Cleanse is a Unique Type Effect with a Stack Limit of 3 that allows a player to spend the token at any time to remove a single status effect token from themselves.
As you can see, even using the same text, KB hallucinates the wrong answer only when using docs. This makes it very concerning to trust Knowledge Agent.