https://discord.gg/botpress logo
Join Discord
Powered by
Keep state of conversation
# 🤝help
f
We have been trying to figure out how we can keep the state of the conversation with the Webhook Trigger. We have tried creating a conversation with the [Create Conversation endpoint](https://botpress.com/docs/api-documentation/#create-conversation), but that didn't help. Devmik reminded us of this message https://discord.com/channels/1108396290624213082/1215469047773143050/1217873161970909265 from @microscopic-butcher-43176 about using the Make integration to keep the state, but the explanation of how to do so was a bit vague. All ideas and inputs are very welcomed! People who might also know @famous-jewelry-85388 @limited-pencil-78283 ?
@quick-musician-29561 and @fresh-article-37001 can add something if I forgot to mention it
f
I tried using the conversationId of the Webchat channel (Since I cannot create one for Webhooks) and it was no longer giving me errors in the botpress logs that the conversationId should be provided. But I am still unable to keep the state, all messages are send as if they are the first. I am sending the conversationId of the Webchat conversation in my request to the bot and I am saving the conversationId as shown in the images. https://cdn.discordapp.com/attachments/1222855520524701777/1222859215157198888/image.png?ex=6617bf64&is=66054a64&hm=304b513365b912f81426e4dc556dd5e1f1fa3030475dfeffcc32a62977b41a7d& https://cdn.discordapp.com/attachments/1222855520524701777/1222859215429963786/image.png?ex=6617bf64&is=66054a64&hm=efad58e03e4b8b71da107d3a75727fe52d02e24fda9268524e1b377845648be6&
l
The problem you will face with webchat and I have not tried it yet is to have an identifier to a webchat user. I mostly work with WhatsApp, where I have the user's phone number as the identifier and it has a linked conversationId. Whenever a user comes to the Bot , I capture the phone number and its associated conversationId. So when an input is received from the webhook, I know the phone number and can pull up the conversationId from the table, where i stored it If there is a way to identify the user, in webchat, this can be achieved easily
The other option would be that you send the conversationId wherever you are sending data, like to make and then make will return this information in the payload
I was reading the general chat, have a question, is "webchat:id": "a991f618-58c5-48e6-a042-24245723d0f6" part of the event?. If yes, you have your unique identifier
f
So what I'm doing is a bit different. I have an application in flutter which is a mobile and desktop app. I want to be able to have a normal conversation with the bot through the app's interface. I am not performing any requests to the bot from anywhere other than the app. But the issue is that when I send a message from the app to the bot through the Webhook integration the conversationId isn't present by default. And when the bot tried to send it's message it was giving me an error that the request/body must contain a conversationId. We tried using the conversationId of a Webchat conversation since the Webhook didn't have any conversations stored. But now the only progress achieved is that I am no longer getting the error that the conversationId isn't provided. However, I am still unable to keep the state of the conversation flowing, no matter what I send the bot will respond as if it's the start of the conversation
l
conversationId of webchat will not work as the interaction is not the same. if your mobile app needs to connect with the bot, then there has to be a way for the bot to know the interaction. Why don't you send a heartbeat to the bot's webhook, with an api uri, the webhook should accept the heartbeat and send a heartbeat response to the api uri and in that payload the conversation id should be present. Then as long as you are sending data to the webhook with that conversationId, you will have a dialog. you periodically do a heartbeat to check if the conversationId is the same, if not, re-initate the heartbeart to get a new conversationId, similar to a token refresh Hope this method works
I forgot to state that when the webhook receives the conversationId, you have to manually set it event.converstationId = event.payload.data.conversationId (assuming this how the payload is received at the webhook)
f
I haven't used a heartbeat before but from what I understood I can send a callback URL to the webhook and the bot will be able to then send the messages to it? How can I provide the heartbeat and how will the bot know that it should use it?
I think I am doing this already
Second image
l
where is event.payload.body.conversationId value coming from? I mean I see you are fetching it from the payload received by the webhook, but from where are you passing the value to the webhook. If manually captured from webchat, then it definitely wont work
lets do a DM
a heartbeat is just a jazzy word. Yes send a callback URL and in that send the conversationid
f
So what would the request body look like?
I am currently sending the conversationId of a Webchat conversation I don't have a Webhook conversation that I can use.
l
from the screenshot image2, comment the line event.conversationId = event.payload.body.conversationId and just print event.conversationId?
f
I would really like to tag along 🙂 Preferably here so others also can, if it's okay
f
We are looking into ways to create a conversation with an ID that can be used for the webhook but still didn't find a solution yet
f
Alright. Let me know if you do
l
I tried to manually generate and conversationId from a guid and did not work
So somehow the conversationId has to be there system generated for a conversation and it cannot be over ridden
Unfortunately that does not help 😔
@fresh-article-37001 why do you need a conversationId, since you are calling Botpress from an app
f
I need to be able to continue the conversation from where it left off
l
Is it like you ar creating a chat stream
f
I have an accounting application and the user can chat to the bot and he can go back and forth with a certain task like creating an invoice
l
On WhatsApp my prescribed method will work
However, I am not sure from the app
f
I already can make whatsapp work but for production I need to use the app
Whatsapp isn't as professional and it also has high costs when scaling
f
One idea is to create a webhook trigger for each node. You can use a filter to decide which one to trigger. You would add an identifier of where to go next. This identifier would be used in the next request to the Webhook. https://cdn.discordapp.com/attachments/1222855520524701777/1222900791845851136/image.png?ex=6617e61d&is=6605711d&hm=a1726fb77dc42a69e86685fc0a49c757ae57c22d4dca38880e5797ee2acba298&
Its a bad way of doing it, but its a way
l
From the app, can you send the users WhatsApp number? If yes, then it can work
f
But my system is huge
f
Then no. I was about to write that it should only be done if you had very few nodes.
f
I have 4 document types and each type has 7 operations that can be performed on average and each one requires multiple stages
Yeah
I don't understand
l
Try this
When you are calling the webhook pass the phone number
f
Inside the app the user doesn't have a number they login with their emails
l
Shit
f
I guess I can change the login method to phone number
But even then
How would that work
l
In this case there is a problem, the user has to first start the conversation from WhatsApp
If that is doable system will automatically generate the conversationid, store that in a table and from the app when you pass the phone number, reference it to the conversation id
f
I want everything to be done from inside the app. The thing that could solve this would be if a conversationId was automatically generated on the first message sent to the Webhook but this isn't the case.
l
I mean you can create an OTP method using WhatsApp for registration that will send the first conversation from WhatsApp
f
Okay I want to understand one thing
Can a conversation be reused?
If the user first signs up with a phone number and then I can somehow use Whatsapp to create a conversation id can I keep using this inside the Webhook?
I want to note what I've discovered so far. I already had webchat and whatsapp conversations and I tried using the IDs of these conversations but that didn't keep the conversation state. I tried creating a brand new conversation via the API for webchat and same issue. If I don't use a conversationId I get an error, if I use a conversationId that I randomly generate I also get an error. The only way is to use a conversationId generated by botpress. Even though when I pass a conversationId for another channel such as Webchat I don't get an error, I end up getting an error when the bot goes inactive after 2 minutes (I set that in botpress studio). And that error is caused by a user variable not being set which was set on the time of the request.
l
Yes, I am doing that today
f
Please update me if it works
I am looking at the documentation and watching the entire video again now that I understand more

https://youtu.be/0p4x7RqkKj0?si=hgNZdyRK68KBjF4t▾

l
check bot. I created a simple bot that creates a conversationId on a webchat. I used the same conversationId and send it to the webhook and added a text card to the trigger, the text message was displayed on the webchat https://cdn.discordapp.com/attachments/1222855520524701777/1222913309137702982/excellent-hawk_-_2024_Mar_28.bpz?ex=6617f1c5&is=66057cc5&hm=247c7d99da406f61022dab1baebc6d8aa5609647c579ea2e81fcc174a0df7210&
f
But the conversation won't happen on the webchat. It will happen on the app via the webhook, if I use the conversationId of the webchat conversation is doesn't keep the state. At least it didn't work for me...
l
try the app, it works Instead of webchat, start from WhatsApp, it will work. Instead of manually copying, add the whatsapp number and conversationId to a table and retrieve from there. But when calling the webhook, pass the phone number, it will work
f
And I create a whatsapp and webhook integration like normal and then text the test number and it should save the conversation id and phone number right?
The version you sent here doesn't have the insert record card
How do I access the phone number and conversation id that are sent when a user texts the bot via whatsapp
I just thought of something, wouldn't it be different if I sent a message through the Botpress API instead of through the webhook? https://botpress.com/docs/api-documentation/#message
@quick-musician-29561 @limited-pencil-78283 @fresh-fireman-491
Btw me and @limited-pencil-78283 discussed this on a call and he was thinking similar to what @fresh-fireman-491 suggested by using filters to go to a separate node for every webhook call. But that won't work for my case.
I am currently trying to make this work but the messages endpoint is giving me this error:
Copy code
{
  "id": "err_20240328154107x1E455BF7",
  "code": 400,
  "type": "InvalidPayload",
  "message": "The provided UUID is invalid"
}
My request body looks like this:
Copy code
{
  "userId": "a1f7b996-9679-430d-b2f8-7792a8d06e6b",
  "conversationId": "dbd7576f-111f-4bf2-b8e4-2f5e64d5c92e",
  "payload": {
    "text": "Hello"
  },
  "tags": {},
  "type": "webchat"
}
And I obtained the userId and conversationId from thier respective endpoints and picked a user and conversation on the same channel but it didn't work
f
I am about to head off, but I can look into it later today. I can't find the cUrl right now, but I am pretty sure I have one that is working laying around
f
Yeah I added the headers
That would be very helpful thanks
m
henlo, need my help??
f
Yeah
You were able to maintain the state of a conversation using webhooks?
m
nah, i changed all of my webhooks to the Make integration
let me get to my pc and i'll explain it to u, is pretty easy
f
Ok and does it matter that I need to send messags from an app to botpress or does it work only for make?
m
last time i tested that integration worked with any webhook
see, in the make integration you can put any Webhook URL, i happen to use a Make webhook but you can put anything in there https://cdn.discordapp.com/attachments/1222855520524701777/1222947435312320582/image.png?ex=6618118d&is=66059c8d&hm=29ae4ae61d4ede5ba6c62bac0c19e1b34f42a7802a81b8f6e2c800cee943e70a&
then, inside the bot you can use the Send Data card from the make integration to send any request to that webhook,
that card has a Data field inside, if you have a complete object to send you will need to JSON.stringify it, like this:
now, make sure that the webhook you are using generates a response and not another POST request or something, because if you dont generate a response it wont work
the fact that the Send Data card is INSIDE the workflow and not in a trigger or something like that is the key to storing the actual convo state https://cdn.discordapp.com/attachments/1222855520524701777/1222948976937013399/image.png?ex=661812fd&is=66059dfd&hm=2882bae378e81b7cd1a1bb37d9e1468094b12421f2f31c43d8b54b65f1ff52d3&
of course, you'll need to make all the processing for the request inside the workflow (including error handling), otherwise you'll do nothing with that data
hope that helps u my man @fresh-article-37001
f
Hey thanks alot will check this out in a couple of hours kinda busy now. Thanks alot man
m
np, let me know if that helped u fam
also @fresh-fireman-491 thats how I solved my state management error, feel free to use that info as you wish
f
Fantastic, that is exactly why I pinged you!
m
no problemo my hermano
l
Interesting, I think Send Data being part of a workflow and not a trigger point is the key difference. Have to try this... great insight @microscopic-butcher-43176
f
I am currently doing your implementation but this step is going to be a hurdle for me
Since my app cannot directly receive messages from botpress since Botpress doesn't allow installing additional libraries like firebase-admin which I need.
Currently my conversation flow is I send a message from the app directly to the bot but if I want to recieve a message I need to send it back to my server which is then able to send it back to the app.
So the server will not be returning any data back to botpress which could be modified but even then what would I return?
I also am wondering where should I position each send data card, everytime I have a text card?
Hey guys I think I figured out the solution, it's through the botpress API. This me talking with the bot through postman, I was able to maintain the state. The file is the Get Messages endpoint response https://cdn.discordapp.com/attachments/1222855520524701777/1223282254042759238/Response_JSON.txt?ex=66194960&is=6606d460&hm=bbb127904fa95c81ad4f78fa696737066f9ac6fd74e4d94b96ba0876582e34ab&
I will be playing around with the API to see how to manage the state and stuff like that but I am very confident that I maintain the state through the API, and I will provide the code that I come up with.
Hey guys I figured out what the logic will look like and I will give the code once I finish it. But in general this is how it will work: When a user creates an account inside my app, I call the botpress API and create a botpress user and conversation for that user and I store them in the database alongside the user's login data. (The conversation and user can be created for webchat or any other integration, it will still work). Everytime the user logs in I fetch the userId and conversationId which are essential for the messages API endpoint. The conversation can be reused as much as needed and it is managed by botpress, meaning that if you set a conversation timeout of 5 minutes a new conversation will start after 5 minutes of inactivity and you don't have to manage that manually. Now that you have the userId and conversationId, you can use those to make a call the messages API and create a new message with whatever the user texts the app. The API response WONT contain the bots response so after that you need to make a GET request to get the new outgoing messages. Getting the new outgoing messages can be done in different ways but to keep it simple I decided that I will perform periodic GET requests every 0.5 seconds and check for a outgoing message that has the current conversationId and is 10 seconds old at most. (The logic might be changed later). After you get the new message(s) you can display them to the user and then rinse and repeat.
@fresh-fireman-491 @microscopic-butcher-43176 @limited-pencil-78283 @quick-musician-29561 Thank you everyone for your help
I will post the code here as soon as I'm done for anyone that's interested
m
sure man, i'd love to
l
Share the reference document you used @fresh-article-37001
f
This is a request that I made to the API if you want to see what it looks like in postman:
Copy code
{
  "userId": "user_01HSZQM2B6CD7A4XHY0ETKKQ22",
  "conversationId": "conv_01HT29SS60G9T39BGG1NACNG0Y",
  "payload": {
    "text": "Create invoice"
  },
  "tags": {},
  "type": "text"
}
Don't forget to include the x-intergration-id, x-bot-id in the header and the botpress API key in the authorization
The tags can be empty I actually still have no idea what tags do
For anyone that's interested this is the code I wrote that uses the API to maintain the state, it's in the dart programming language. https://cdn.discordapp.com/attachments/1222855520524701777/1223351825164533840/botpress-api-dart.txt?ex=66198a2b&is=6607152b&hm=3968de2a98d402a34b6bddd157c3d5fc6fbccedb57475e0c341cabf12b68a27d&
The function
Copy code
checkNewBotpressMessages
get's called once every second and this allows a delay that's very minor with not much load on the application.
I am aware that this solution might not be the most efficient but it gets the job done. If I come up with a more efficient way to conversate with the bot through the API I will post here.
@fresh-fireman-491 @microscopic-butcher-43176 @limited-pencil-78283 @quick-musician-29561
q
This is one of my favorite threads on this server, offering so much to learn from! 🔥 🔥 🔥 🔥 🔥
f
@quick-musician-29561 @fresh-fireman-491 Guys, Can you give me a summary? Or should I use ChatGPT 😄 ?
That topic is extremly important 🙂
f
I can do my best, but I think the best one to do it would be @fresh-article-37001 so maybe he can correct me when he gets online.
What Joe wanted is to use something like a messaging API that would keep the state of the conversation. The initial issue was that it was always starting the conversation from the start when he attemped to do it. After a lot of trial and error he found a way to do it! The start of the solution is here https://discord.com/channels/1108396290624213082/1222855520524701777/1223282254365855834, and then down to our messages.
That is a very short summary
q
We continued this topic here, which earlier started in developers channel with this message https://discord.com/channels/1108396290624213082/1121494527727902891/1222802001532817499 and continues there for a while. Unfortunately, I only participated with emojis 🫣 Others did a really good job here, and I also think that one complete tutorial with the solution in #1132038253109837994 section about this would be so useful 🛠️ 💎
⬆️ those messages in developers channel starting from this https://discord.com/channels/1108396290624213082/1121494527727902891/1222802001532817499 were a good starting point and some useful links there also.
f
Thanks guys 🙂
f
I have a mobile app that I want to use to have a conversation with my bot. Initially I was trying to use the Webhook integration which didn't work because the conversation states weren't saved automatically by botpress similar to other integrations. So I thought of another way to do what I wanted which was through the Botpress API. I basically used the API to send messages (I modified the code now and can also now send and receive images and documents). The hurdle with using the API was receiving the messages and I thought of a way which was to call the GET endpoint of the messages API periodically to check for new messages and then display them. With this solution I would simply format the payload in my app and then send it to the API and then check for new messages every second and extract that data and display it.
Once I've fully tested my code I will post the modified file here which will contain how to send images and documents as well because there is another major hurdle that I overcame with that
Basically when you want to send and receive document and images the payload needs to have a URL. To have a URL that the server and the app can both use you need a place to put the documents and I chose AWS S3. And you also have to make it secure so you can't just simply use the URL of these files you need to generate a Pre-signed URL that gives temporary access to them. I implemented this in my server and I exposed an endpoint for the app to be able to generate these Urls and download and upload files.
The server is in JavaScript and the app (client-side) is in dart. I will post the codes for this task when done.
This is the file in the app (client-side) which handled sending and receiving text, images, and documents to/from botpress and also uses the server to store the images and documents to allow sending the URLs: https://cdn.discordapp.com/attachments/1222855520524701777/1224784383148294254/botpress-service-dart.txt?ex=661ec058&is=660c4b58&hm=85347cc8c488d15765c2ba36f888ada98879caa91a7dc267263119467c7ebf7e&
@microscopic-butcher-43176 @limited-pencil-78283 @fresh-fireman-491 @quick-musician-29561 @famous-jewelry-85388 This above code allows images, documents and text to be sent. And I tested it and it all works. I am also thinking about adding support for showing dropdowns and multiple choice in my app and I could post it if someone is interested.
m
doin' things with dart, a fellow connoisseur i see hahah
p
Hi @fresh-article-37001 , Thanks so much for sharing this strategy. Ill be implementing the same to initiate my botpress conversation via a webhook (from Twilio IVR). Where did you find the x-integration-id? What exactly does that x-integration-id identify in the use case described in this topic? Im Not sure why its needed, and again i can't locate it.
f
When you go to an integration on botpress such as webchat if you look at the URL in the search bar there is an ID at the end
p
Oh interesting, I had to set the x-integration-id to be my webchat integration id. That was counter-intuitive to me because i figured i was interacting with the api without the need of an integration. So the "webchat" integration is also known as the "Botpress API" integration
I include had to include "channel": "channel" in the body
s
Hey, this was some time ago, but do you think you can send me a bigger picture of the flow? Including the "start" and "webhook event"?
f
Hey there👋 You can now use the Messaging integration https://app.botpress.cloud/hub/integrations/intver_01HVH28K660275MXFX741YDDTQ
p
Do we know if its possible to not specify the Response Endpoint URL in the integration setup, and instead programmatically set the url via a code card in the botpress editor? I guess a workaround would be to send responses to a different url manually via a get or post HTTP request.
f
Hey there @proud-beach-98934 It might be better to start a new thread about this. I like the idea so feel free to @fresh-fireman-491 in your new help post, if you make one
I have played a bit around with the integration. I managed to chat with my bot from a terminal, and keep the state of the conversation. https://discord.com/channels/1108396290624213082/1229868904034074675
l
@fresh-fireman-491 this seems to be the longest and the most interesting thread 😄 I am sure team Botpress should review this thread and look for product enhancement. The ability to control the workflow or be able to, through a trigger back into the workflow (from where all of this started) would be a huge advantage
f
I think it is the longest. I think that they fixed it all with the messaging integration
l
@fresh-fireman-491 messaging integration? please share details
f
There's a new integration for it. It's not that hard to use
14 Views
PreviousNext
Powered by