For example in a Q&A situation I have a bunch of questions with their respective answers. The bot would just hallucinate and combine other answers together to form it's own.

That's a question I've been wanting/seeking an answer to mysel👍 f!