https://discord.gg/botpress logo
Join Discord
Powered by
Please help! Our AI spending is too much high!
# 🤝help
r
f
Hey there, It will be lower if you use GPT-3.5 it's around 30x cheaper
Maybe we can separate these posts? So this post is about AI spending, and you can make a new post about the issue with the Query of the KB
r
done it
We wanted to have all types of situations but looks like we need to use less values
f
Why not just include the answers to all of those in the KB, instead of there?
r
because some of them traffic to another nodes the handle this types of situations
like some one says he wants to cancel i different ways so it will send him to where it needed
c
can you post a bit more of your flow with higher resolution ?
generally it is a good practice to minimize as much possible within your flow
this includes the options for the AI task node.
instead of branching out to every possible solution with the AI expression node you can try to "catch and parse" the response using an AI node e.g.: - if a user says "I would like to buy a product" set "workflow.intent = buy" - if a user "wants to order a coffee" set "workflow.intent = coffee" - then we can set an expression node that triggers the flow if it matches one of these outputs.
r
@User Someone can assist us?
f
I think Remo and I have given you enough to start changing some things, and trying those 🙂
f
Hey there , from where can I check my AI spending??
f
f
Thanks 👍🏻
f
You are very welcome
r
@User @bumpy-butcher-41910 anyone here can assist us? each response is now 0.1$ even from the KB (all organized there in a Q&A) and its really lot in a basic conversation...
the ai spend can go to end i only one conversation
we tryied to switch to GPT 3.5 in KB , and downed the prices to 0.01 but it cant answer nothing for our KB and skips to our ai fallback like nothing is there
Hi , the main cost is on the knowledge search and not even the ai cards
its 0.1 for each response
d
Please show me your KB what type of files u have
r
only rich text file
and a website
homepage
f
How big is your rich text file?
And are you using the Website as your source or the Search the web?
d
Maybe its language issue, because i work with polish and ai spend for polish is a litle bit higher than for english
r
tryied both, search was more cheaper
what are your prices for polish? and what kind of model you run?
not too much big, just the details we need to sell the product
f
So are you still using Search?
r
yes
what do you recommend?
f
I don't think I can do much more here unfortunately. I have given you the tips and tricks that I know.
b
just to confirm, it sounds like you're using GPT-4?
r
Now its on GPT 3.5, On 4 all the prices are going to 0.1$
b
if you've been using GPT-4 during your tests, that's just the cost of the model
you can learn more at https://openai.com/pricing
as I show in the video you linked above, it's possible to reduce the cost of your knowledge base queries using the tips we outline there
r
Hi Robert thank you for your reply, We have been working on it for the past few days and trying to fix it. I would like to describe it hope you will know the answer. *• When we go on 'Fast (GPT3.5)' * All our query knowledge base cards are getting ignored, or with bad responses price for each response is around 0.00X$. • When we go on 'Hybrid (GPT3.5\4) reads the kb query knowledge base cards working much better The bot knows how to answer and do it better, price for each response is around 0.08$ on GPT 4 and when it is on GPT 3.5 price for each response is around 0.00X$ • When we go on 'Slow (GPT4)' Good response but the price for each response is 0.08 on GPT 4 after this 2 days Hybrid is the more accurate one, but the prices are very high, with calculating of a typical chat of 30 responses, it can cost around 1$ for each conversation.
all the tips that are here are very welcomed and big thanks for them but as you can see @bumpy-butcher-41910 @fresh-fireman-491 the problem is not the AI classification or any other thing, Its just the Query KB agent that spends 10000 tokens for small responses It is a lot...
All KB is organized in one rich small text file with Faq and explention. Also, @User I am not this type of person but I must say that we have been working on the Botpress platform for about a year, learning all the features and techniques and we are huge fans of it that looking forward. but each time we think it's perfect and it's working for about a week it pulls us back again and again by messing with our responses randomly after a week or those high prices of conversation. we really don't know how to sell the bots to our customers until we figure out how to solve those things.. And we see that a lot of bot builders are working. so why we can't? (Have just checked the Claude LLM, the prices are much lower, work great with other languages like Hebrew and etc I think it can be an amazing option to add it to botpress)
b
what is giving you the sense that the token usage here is higher than expected?
GPT-4 is just an expensive model, unfortunately
I typically don't have problems with KB queries on GPT 3.5
the ideal solution would be to use GPT 3.5, since it's about 30x cheaper than GPT-4
r
Because it a lot of time costs 0.07 instead of 0.0043 which takes one simple conversation more than 1$ as I mentioned before in the images
From your experience, is that the ideal price? I gave a try to check claude haiku with the same amount of data and tokens price was there about 0.0000X, its a big difference..
We try to understand how to do that it will use the GPT 3.5 Prices and also answer good from the KB, sadly it works normal only on 'HYBRID' mode that takes us to those prices. We realy have done a well explained (not too much) KB that has all the data we need to get answers about. So we cant figure it
This what we trying to do 🫨
f
You can't really compare GPT-4 or GPT-3.5 to Haiku. They are different models, from different companies with different pricing. Haiku is incredibly cheap and fast. You could compare GPT-4 to their Opus model.
r
Will check that thank you!
f
You are very welcome
r
I am trying to do the table part with the faq in my KB
The thing with the tables is getting little messy with translation and finding it
f
The tables are for structured data. It will yield the as good results if the data is not structed, but it will yield better results if the data is
r
We have done some changes but It is still expensive when we use the 'Hybrid' KB model because of using those Query knowledge cards, does anyone here have a recommendation for a way to reduce the cost of it somehow? we trying to figure it out for long time
There is a way after we used the Query knowledge card one time to save the data somewhere for not to use it any time we need the kb? to make all the processes cheaper and not used each time we need the KB with this expensive card Each time a client asks a question we use the KB, but we need to use each time the Query knowledge card? or there is a way to use it one time only at the beginning for scanning all the kb? https://cdn.discordapp.com/attachments/1222453583728087090/1225460833924943934/image.png?ex=66213656&is=660ec156&hm=7115883d5677b5ca216d07dd38ad39e5bffa51a76b1b8a4f2bd2efec1e0514f4&
the main reason of those prices is this query knowledge bases cards...but there is no other way to answer with KB
f
I would recommend you to take a look at this video, which explains the basics of how RAG works

https://www.youtube.com/watch?v=2mDsuzty4fw▾

Hey there @rhythmic-lizard-47527 You can now reduce the spending as you wish in the KB agent settings. You can pick how many chunks it should retrieve from the Knowledge Bases. Less chunks = cheaper, but less accurate.
r
how we can do it?
f
You can edit it in Knowledge Agent settings. There is a slider
r
I want to marry you @fresh-fireman-491
PreviousNext
Powered by