Greetings! I would like to have information on using the GPT 4 turbo option when using Botpress Studio. In building the flow, using the GPT 4 turbo option in complex text generation processes that require significant dimensions, I almost always receive the error: Cognitive - AI Generate failed: timeout of 30000ms exceeded (Status Code: undefined) without no results. Instead, with the GPT 3.5 turbo option, in the same processes, I obtain satisfactory results. What should I do to overcome this problem and be able to use the GPT 4 turbo option for better results? Thank you very much for the answers and indications you want to give me.

What is your prompt to it?

Thanks for your answer. I don't think the problem is a matter of a specific prompt. The problem persists with several prompts that require an elaborate response... and anyway, with GPT 3.5 turbo I have no problem. I'm sorry I can't take advantage of GPT 4 turbo: how could I solve the problem?

The reason why I asked for the prompt is that you have a timeout. If you asked it to do something that would take more than 30 seconds for it to process it would give you that error. Have you tried exporting your bot to a new workspace?

Thanks for your answer! How can I disable timeout? Also: I tried exporting to a new workspace. Again: I developed the bot in the new workspace that enables the use of GTP 4 Turbo...

I don't think you can disable it. Can you ask it to generate something really simple like in the picture

I understand. But the goal is to obtain a more elaborate and extensive result from GPT 4 Turbo than GPT 3 Turbo. If I ask for "simple" answers I don't get what I would like and therefore I would continue to use GPT 3 Turbo... Furthermore, if the problem is "Invocation Timeout" (The maximum time for a bot to process a message or an event before timing out) whose Quotas are Amount 60s, why is the timeout in my case 30s?

Trying it with a simple prompt is just to make sure that it works and doesn't get time outed every time

With a simple prompt GPT 4 works... the problem is that with elaborate prompts it times out... While GTP 3 Turbo has no problems...

I think this is out of my reach. Hopefully someone from the Botpress team can help you in your new thread tomorrow

Hey @icy-battery-22041 let me investigate. Could you please send over your bot ID? Is there a specific node where this happens?

Thank you, Sabina. my bot ID is: botId": "3cb7b4b4-ef1d-4449-9bfc-c214432f6d06" but the problem happens to me in every bot. Also, the problem happens to me in every node with AI Generate... I await your kind feedback.

Could you kindly take a screenshot of the error message you get?

@icy-battery-22041can you please check your Workspace Limits (Dashboard --> Usage and Subscriptions)? And please confirm the usage under AI Spend. I'm sure this is unrelated, but taking all info in. Thanks

Here you are! but it's very different from the one you sent me...

Thank you @icy-battery-22041 !

You are really very kind and efficient! Thank you very much!👏

I'm always happy to help!!! 🙂 have a great rest of your day.

Hi Sabrina! I would like to give you feedback about the issue: Cognitive - AI Generate failed: timeout of 60000ms exceeded (Status Code: undefined) I have verified that currently the limit has been increased to 600000ms. The results are that the use of CHATGPT 4 has improved (whose processing has a higher quality than CHATGPT 3.5) and yet sometimes the problem reoccurs. In this case, by restarting the conversation locally in botpress studio, I can obtain the result. So, the situation has improved but it still suffers from the problem that prevents me from being able to propose the chatbot to the public... What could be done to definitively solve the problem? Thank you very much for your reply.

I have transfered over your message to the product team. Will update you as soon as I hear more.

Thanks to you, Sabrina! For completeness of the report, I add that in the first processing there are 2 cases: 1) undefined error; 2) elaboration that contains the previous text, but with a correct heading. In both cases, by restarting the conversation locally in botpress studio, I can obtain the result.

@witty-football-93730 if you have any questions for @icy-battery-22041 , feel free to jump in. Thanks 🙂

@witty-football-93730 Thank you! I am at your complete disposal to provide all the necessary information and support!

Any update on this @icy-battery-22041 ? Is it working fine, now?

Thanks, Sabrina! You are very kind in asking for feedback... Now it seems to work (in numerous tests the problem only recurred once). Unfortunately, I am currently stuck in my work by another problem that has manifested itself since yesterday: https://discord.com/channels/1108396290624213082/1203965336559288340 Queries to the knowledge base fail because the documents in the knowledge base do not appear to be operational and indexed (see image). I have not had any feedback on the matter. However, I see that the problem is not just mine and it seems to be the server's... do you know anything about it?

Alright, so solved for this issue! I'll see the other thread.

Dear Sabrina, I wanted to update you about the problem: GPT 4 turbo generates the error: Cognitive - AI Generate failed: timeout of ... After numerous tests, I confirm that with the limit of 60000ms the phenomenon has been reduced in Botpress studio even if it sometimes recurs. In this case, by restarting the conversation locally in botpress studio, I can obtain the result. This is not possible for a user and for this reason I fear that I cannot, unfortunately, release chatbots for users as the consultation is blocked. And I'm very sorry about this. Do you think something can be done?

Thanks for letting me know @icy-battery-22041 ! Appreciate it. The team went ahead and increased the timeout even more. Please let me know if it works out now.

👏

Pushed to prod 🚀 @crooked-van-25152 @icy-battery-22041

Amazing! Thanks @witty-football-93730 💎

Could you be kind enough to give me an answer to solve the excessive latency problem when I use GPT 4? Why does GPT 4 Turbo in Botpress have unbearable response latency? I use Botpress Studio with satisfaction but unfortunately the response latency problem with GPT 4 prevents me from aiming for more satisfactory results (this does not happen with GPT 3.5). In particular, if this happens with the emulator but I still acquire the response, with the user chat it even freezes... and this prevents me from using GPT 4 for bot diffusion. The chatbots I create use "full" prompts and requests which certainly require complex processing and which return complex texts but this, for example, with other platforms (ChatGTP, Poe, ..) there are no speed and latency problems. Should I assume that the problem I'm having is due to the Botpress platform? How could I solve the problem? Thanks for your answers.

Do you always get a response with GPT4? Are all answers generated by GPT4 taking a long time or mostly the first message? How long are we talking here? @icy-battery-22041

Thanks Sabrina! The prompts of my workflows are generally complex and require mainly text processing responses which are also complex and full-bodied. With GTP 3.5 I receive quick responses both with the emulator and with the user bot, while with GTP 4 the processing times with the emulator are extended (about a minute) while with the user bot the response is often not received and the conversation it blocks. So with GPT 4 all the answers take a long time... Please note that using the same prompts with other platforms that using GPT 4 (ChatGPT, POE, ...) I get quick responses, while with Botpress Studio I don't. How can I solve it? This issue prevents me from producing bots for users using GPT 4 and this is very frustrating for me...

GPT-3.5 is generally just faster, but it shouldn't be that much faster. Is this issue still persisting, if so could you share some more information about the prompt and the response it gives for both GPT-3.5 and 4

Can I please get a report ID? I'll see with the team what can be done.

Thanks Decay! I think something has changed but only for the EMULATOR. IN PREVIOUS TESTS between GPT 3.5 and GPT 4, with the same prompts, the results were generally the following: WITH EMULATOR: AI Task Completed – 51709ms: GPT 4 AI Task Completed – 9929ms: GTP 3.5 WITH THE USER BOT, the times with GPT 4 were much longer and the response was often not shown. IN THE CURRENT TESTS the results are reported in the attached document in which I also reported the prompts I use. FOR THE EMULATOR: As you can see the response latency seems to be reduced and the difference between GPT 3.5 and GPT 4 now seems reasonable. HOWEVER, WITH THE USER BOT, IN TEST 2, while with GPT 3.5 I have no problems, with GPT 4 the responses are blocking and (after 10 minutes of waiting) anomalous. report_01HQTVJCT2HK30BE72TXMKBZTK

I don't understand Italian at all, but it looks like your prompt, is very complex. The AI cards don't have access to the KB. You can't query it through them. You would have to use a capture information card or the Query Knowledge Bases card, and then process the response via the AI card.

Thanks, Decay. In relation to the methods that I use to address the KB, I insert an instruction in the prompt: I will reply it in English below: // You are "JOB SYSTEM 5C", an assistant specialized in providing support to the user who has a role of "@workflow.scope" to know, understand, apply, exemplify the contents of the knowledge base "THE 5C MODEL". ## Task: - Analyzes the request of the user "@workflow.richiestachat" and identifies in the knowledge base "THE 5C MODEL" all the contents consistent and relevant with the request. ## Using Knowledge Bases: - Draw from the knowledge base "THE 5C MODEL" to form your answer. In relation to costs, it is not a problem. I aim for quality and quick responses. AM I SOMETHING WRONG? WHAT SHOULD I HAVE DONE?

So you got all of the knowledge base text directly in the AI Task?

Yes!

That is most likely the reason why it takes so much time

Forgive my ignorance... what should I have done?

You could use a capture information card or the Query Knowledge Bases card, and then process the response via the AI card.

I think I understood my systematic error. I will use Query Knowledge Bases cards for my tests and will let you know. Thank you very much!

You are very welcome