https://discord.gg/botpress logo
Join Discord
Powered by
Using Query Tables with Find Records
# 🤝help
b
Hey there, this should be a simple question for most, I created a bot and had a conversation of 64 messages to test it. Those 64 messages cost me 2$ in AI spend and thinking of a realistic deployment, even 10$ would never be close to enough for a live bot. How do people manage their AI spend? I checked this article: https://botpress.com/blog/how-to-optimize-ai-spend-cost-in-botpress But I have not really found a definitive implementation of something like searching query tables to print responses. If there's a tutorial out there, please let me know. I would also love to hear other strategies too! Thank you for the read and apologies if this was answered somewhere else, I checked wherever I could.
Also, to add to this, I am also considering using an open source LLM instead of ChatGPT for the inference to reduce the costs but am unaware on how to use the knowledge base with that LLM along with the personality agent. Would be helpful if someone quoted this problem too.
b
currently, we don't allow you to bring your own models to perform inference on Botpress
your logs will show you how much your queries are costing, but at this price point it sounds like you're using GPT-4 when you could be using a less expensive model
b
Thank you for the response, I remember setting gpt 3.5 in the options. Can you please point out where exactly the model selection is?
b
it's in a few places, depending on which function you're using
b
and how much should I expect to spend on AI tokens on a bot for let's say 2000 messages a month
b
you can start by selecting it in the knowledge agent menu
otherwise, it's difficult to predict, because the price varies per message based on what actions the bot is taking
so I can't give you a solid estimate
b
Under 50$?
b
you can play around with : https://botpress.com/ai-spend-calculator
b
This wasn't accurate for my case
I'm pretty sure I set gpt 3.5
b
that's what I'm saying - it's difficult to guarantee any kind of estimate. your best bet is to test it out for a few hours/days and see how your usage is
b
But it makes sense that gpt 4 was being used considering the cost
b
yeah there's no way 64 messages cost 2$ using 3.5
b
i'll take a look and see what's up
b
you'd have to be ingesting and then writing entire books at that price 😛
your logs should give you detailed token usage
b
Right?
I'll start with that then, thanks a lot for your time 👍
b
no worries, let me know how you get on
while you're testing in the emulator, pay close attention to the event debugger and see how your token usage is
b
this was already selected
rest are all disabled, only using these 2.
and it still cost me tokens for some reason
pretty sure I have the knowledge agent disabled and im just printing the text the bot said
b
the knowledge agent isn't the only thing consuming tokens
b
yeah I managed to figure most of it out, the tokens were mostly used due to the personality agent earlier
but after removing the personality agent, the bot cannot answer questions like "hey bot_name, how are you?"
"how are you?" is a defined question in the knowledge base but the bot gets confused by the "hey bot_name," string
any solution to this?
b
hahaha I think the problem might be
that it's not being interpreted as a question?
that the bot can answer
when you ask this question, what do your logs say?
b
no helpful answer generated by kb
the only solution I can think of is trimming the user input before passing it to the kb query
going to be annoying to implement though so I wanted to know if there is a defined solution to this in botpress
125 Views
PreviousNext
Powered by