Trying to understand the KB Agents ability to use relevant information from a KB to answer questions about a rule of the game. I provide a few simple rules for the KB Agent to read and then ask a question that to see if the agent understands. So far, I'm having very mixed results. Attached is the chat and KB as well a short text summary below: KB: Shield: Prevents 3 damage from Ranged Type abilities, Sword: Melee Type ability that deals 2 damage User Question: Does shield effect the sword ability? Bot Answer: Yes, the shield does affect the sword ability. The shield prevents 3 damage from **Ranged **Type abilities, and the sword is a **Melee **Type ability that deals 2 damage. Since the sword is a **Melee **Type ability, it falls under the category that the shield can protect against. Therefore, the shield will prevent 3 damage from the sword ability. Correct Answer: Sword is a melee ability, so shield does not effect it.

Yes. There's also a ton you can do with your prompt, especially for reasoning-heavy questions like this. If you're into reading papers, I recommend this one https://arxiv.org/pdf/2305.04091.pdf about the best methods for prompting LLMs for best reasoning performance. TL;DR tack this on the end of your prompt:

Let's first understand the problem, extract relevant variables and their
corresponding numerals, and make a plan. Then, let's carry out the plan,
calculate intermediate variables (pay attention to correct numerical
calculation and commonsense), solve the problem step by step, and show
the answer.

It works!!! All hail Gordy and the bot press team 🙇‍♂️ For anyone else, coming across this, this was how I set it up for testing purposes but it can certainly be fleshed out more: I created a workflow that pulls info from KB where each ability had its own KB plain text with the ability name in the title and the description in the text field. In the Rule Agent Workflow, the rules are pulled from the top facts using an Execute Code card and then passed to an AI Card where it interprets the rules and provides the answer with higher accuracy.