Hello everyone, I’m developing a real estate chatbot using Botpress and I’m facing a challenge with the knowledge base’s handling of web page content when it’s segmented into chunks. To illustrate, let’s say I have a web page for “Residence A”, listing available apartments in that residence which are from apartment 1 to apartment 10. This page is segmented into chunks when scraped into the knowledge base using ‘website’. The first chunk includes the residence’s name “Residence A” along with “Apartment 1” to “Apartment 5.” And, the second chunk starts with “Apartment 6” and goes to “Apartment 10” but does not repeat the “Residence A” name since it’s already mentioned in the first chunk. The issue is that when I query the bot about the available apartments in “Residence A,” it only gives me from Apartments 1 to 5 (the ones in the first chunk) and doesn’t give the second chunk’s apartments (Apartment 6 to 10) since they lack the residence name “Residence A” in their chunk, so the bot doesn’t really know to which residence they belong since it only sees their names, and therefore doesn’t include them with the available apartments for residence A. Does each chunk really gets treated independently ? If so, how can we ensure that all relevant data in subsequent chunks is still correctly associated with “Residence A”? Any suggestions on configuring the knowledge base or the scraping mechanism to maintain linkage between related chunks would be very helpful. Thank you for any insights or advice!

Did you find an answer for this?