Streaming response from API
# 🤝help
e
Hi team, I'm making an api call to DANTE AI streaming endpoint. Attached is the streaming response from DANTE AI. How can process the streaming response and show the data captured in the chat. I'm able to get the response from within botpress as well with the attached code. How to I process the response in chunks so that I can display it to the user in chat. https://cdn.discordapp.com/attachments/1223290590461169825/1223290590611898468/apicode.txt?ex=66195124&is=6606dc24&hm=2782f5181116a5d8148230c7ccae25b49e2bd0664793340f8edc0803736aaafe& https://cdn.discordapp.com/attachments/1223290590461169825/1223290590960287764/stream_response.txt?ex=66195124&is=6606dc24&hm=b55099c76748436595edb1f06847b3bc945abf714b80f2bb6c3a77347cfe2352&
a
what is the problem exactly? I see you get the response in response.data and asign it to a variable: workflow.dante_response Then just put that variable in a text card
e
I'm using a text card to display the result however it's taking almost 27s to get the response. So I wanted to process the response in chunks and display the chunks immediately so that the user doesn't have to wait for the whole response to be returned by Dante API.
a
yeah, I had the same problem with Cohere that allows get the response by stream so you can show the user each sentence of the response but I din't find the way to do the same with botpress cards.
f
You cannot stream responses in Botpress. You can do a workaround https://discord.com/channels/1108396290624213082/1211400458329923636
e
I'm not familiar with React.This workaround wouldn't work as our application doesn't need react.
f
It doesn't have to be React. It can also be written in HTML and CSS. Just more work, at least for me
e
I didn't get the workaround correctly, if you don't mind could you please explain it here. and can I use the execute code card to achieve the workaround
f
I have just looked at his explanation and I don't think I can explain it any better:)
Can you ret and go through the post again, and look at the files
e
The code he shared is in react and as I mentioned I don't know react
He mentioned " If you create a new React app, replace the App.jsx file with this code, and it will work like this." If I'm using execute code card I do not know where this file comes in.
f
Fair enough. You receive the event sent from Botpress, then you stream that payload in your custom UI, then you continue from there. That should be the primary React code. You are not going to use the execute code card to send the data. Just for your API call. You can download his bot and see how he did it
e
I making an API call to Dante API which returns the long text response in streams like word by word. I'm making the API call in the execute code and I donot have any UI for now. So how this "You receive the event sent from Botpress, then you stream that payload in your custom UI, then you continue from there." apply to my use case
f
Make the API call to Dante AI without streaming, then send the whole message in an event to the custom UI
e
Since botpress can't handle stream directly, the workaround is to use a custom UI as an intermediate stage. I need the conversation to be continued within the chat so this wouldn't be a feasible solution for my use case.
Thanks for looking into this
f
Fair enough, and you are very welcome!
4 Views