Following a conversation with @square-evening-72346 and @prehistoric-airplane-85682
We noticed a major cost contributor that I believe could be optimized significantly: non-configurable default settings in Botpress AGI tools (like agi/connor).
Problem:
- These tools currently default to GPT-4o , which is great for quality but overkill (and expensive) for many things, including the ones that are required in the connor integration (transcript analysis...)
- Users have no control over:
- The model being used.
- The row factor, which multiplies table usage (the 30 row factor is not always/never justified )
-> What I’d Like to See:
- Let users choose the LLM model used inside AGI/Improvement tools (fast, best, or custom config). Even just defaulting the model to "best model" or "fast model" would be a first improvement as it would allow the user to choose it, in a way
- Allow users to customize the row factor for conversation tables and computed columns.
The new "by source" AI spend tracking is great, but as you can easily see, my whole usage comes from the computed columns of the agi/connor integration (attached image), while i'm speding time to optimize AI costs in autonomous nodes
Thanks for considering this!
Happy to help test or validate if you’re exploring changes here
https://cdn.discordapp.com/attachments/1390614863440646226/1390614864317386864/image.png?ex=6868e684&is=68679504&hm=f6afe88e27185d5d652cf513d2844996cc3494669520103045a579350501a762&https://cdn.discordapp.com/attachments/1390614863440646226/1390614865109848095/image.png?ex=6868e684&is=68679504&hm=97f9feb4d1777d30de3b8e09806be26a989f194df913c4539e08c4b7ef140a55&
s
square-evening-72346
07/04/2025, 10:31 AM
Really appreciate you raising this concern with me @wooden-beard-40210