https://discord.gg/botpress logo
Join Discord
Powered by
Increase timeout for AI tasks
# 👀feature-requests
a
It's usual that GPT 3,5 / 4 takes more than one minute to reply even to easy tasks, the limitation of 60 seconds stops the process and the response never appears, having the user waiting forever. I know is not your fault but 60 is too sort too many times If you can not increase the limit at least create a way to capture the error and show a custom message
i
I'll add: I'm tormented in GPT 4 by "Execution time of 60 seconds exhausted while processing card card:ai-generate / node:XXXX / flow:YYYYY. This card will be re-executed when the conversation is resumed." It's technologically unbearable...
a
Maybe you can move from Open-AI to https://groq.com/ .... the improvement in velocity is huge. Right now you have to contact them to increase API limits but in your case I think make sense and is also cheaper than OpenAI. They use Mistral and LLama models... are not as good as GPT4 but very similar to GPT3.5 I am already working with Groq using custom code cards but they have not increased my API rate limits yet so it only solves some requests per minute... for the rest of queries I have to use your AI cards or OpenAI API and is very common to receive this 60s error. OpenAI api is not very stable, it does not depends only on the number of tokens etc, sometimes it just work slower
i
Thanks for the reply, ivangvillar This is another reflection that I will have to make even if I am perplexed by the lower quality in Groq's answers... In the meantime, I will hope that Botpress will propose solutions to this problem (increasing the limit or offering other providers) which I consider to be a serious limitation that other tools do not present...
a
Groq is not an LLM; it is merely an infrastructure that allows you to choose the model it uses behind the scenes. In fact, it was possible to use OpenAI models, but this now seems to be deprecated
i
Thank you! Now I understand better... It's an interesting option. Using Groq could I choose any model? Can you provide me with information to delve deeper into the issue and specifically how to connect Botpress Studio with Groq? I would be grateful...
a
Groq does not support any model, and the number or requests/tokens per minute is very limited at the moment . You can ask them to increase you account limits as part of the beta program but is not guaranteed. They specialize in hardware, not in providing their infrastructure as a service. However, they are theoretically working on it and it will be available soon. Create an account and start testing the API is free, and they are working on a "on demand / pay per token" plan but is not available yet. If they approve your project, this is the price per token and the available models: https://wow.groq.com/ If you are not approved you can use it but you need another solution as a backup. If the API give you any response code different to 20X you can redirect the query to other system like OpenAI or the native AI cards in botpress
i
Thanks for your directions... I have done some research on the perspective you suggested, even though my technical expertise is limited. For now, I saw that a solution that seems to be in line with my needs could be the integration: AI Stack + Zapier + Botpress Studio... What do you think? Thank you!
f
Seems unnecessary. What do you need Zapier for?
a
Well... I can imagine you have watched the latest openai event... change your model to GPT4o ASAPP!!! 😁
f
i
excellent! I'm doing some testing on problematic prompts (timeouts) and I'm seeing a significant improvement, especially in speed (it seems like processing is reduced by at least 30%)!
Unfortunately, the improving effects of GPT4o in consultation are only manifested in the emulator. IN THE USER CONSULTATION THE ANSWERS ARE NOT PROVIDED AND THE CONSULTATION IS BLOCKED!!! Card took too long to execute A card has been executing for at least 30 seconds, which is too long for a single card
Timeout Issues with AI Generate Text in Botpress Cloud using GPT-3 and GPT-4 Turbo For several months now, I have been experiencing significant issues with the efficiency of responses generated by AI Generate Text in Botpress Cloud, despite the use of detailed and complex prompts. Initially, using GPT-3, the responses were quick but not of the desired quality. Moving on to GPT-4 Turbo, and recently GPT-4o, performance improves dramatically in emulation, but not in production due to timeouts. Despite increasing the timeout from 30s to 60s, more complex requests are often not fully processed, resulting in error messages such as: "60 seconds run out of time while processing card:ai-generate/node:XXXX/flow:YYYYY." The integration of GPT-4o showed significant speed improvements in the emulator, reducing processing time by 30%. However, these benefits don't extend to production, where consultations regularly stall with the message: "Card took too long to execute." I tried various solutions, including segmenting prompts and using the "Query Knowledge Bases" card to lighten the processing load. However, the improvements have only been partial. Question: Are there any configurations or techniques to overcome these timeout limits imposed by Botpress? Or should I consider integrating other models via external APIs (e.g. AI Stack) or move my implementation to another platform that can better handle such processing loads? I would appreciate any insights or experiences you could share!
a
I moved long running processes outside of botpress and implemented asynchronous processing using queues to receive the requests and worker processes that call back botpress webhooks. Complex approach to overcome the timeout issue, but the only one that worked and fulfilled my business requirement. We have a feature request open to make this effort simpler: https://discord.com/channels/1108396290624213082/1215742095625687060
f
Any advice about how to avoid the timeout issue (more than 60 seconds), Is there is something that can be done to increase it? Any contribution to reduce the timeout of the card will be more than welcome...
i
I am reaching out to Botpress as a customer to express some concerns regarding the timeout limits currently set for processing sessions. My practice frequently requires the processing of complex prompts that exceed currently imposed time constraints. I understand that timeouts are necessary to ensure efficient and stable service for all users. However, I believe that the approach taken does not adequately take into account the specific needs of professional users like me, who require more processing time for in-depth research and analysis tasks. Below, I list a few reasons why I think a review of timeout limits would be beneficial: - - Requires Complex Processing: My research activities often involve analyzing large amounts of data and generating complex answers which take more time than standard activities. - Efficiency and Competitiveness: Having more time for processing would not only improve the efficiency of my work, but would also increase the competitiveness of your service, attracting professionals with similar needs. - Customization Options: It may be useful to consider service customization options, allowing users to choose plans that offer more flexible timeout limits based on their specific needs. Additionally, I would like to hear about any temporary solutions that can be implemented in the meantime, such as implementing extended or priority sessions for users with special needs. Thanks
Question: If I activate "Check always alive mode" (currently not active) i.e.: Always Alive Adds "Always Alive" functionality for 1 additional bot, will my user session timeout problems be solved?
e
Nobody on the user side will chill for 60 seconds homie
20 Views
PreviousNext
Powered by