Problem with indexing in Knowledgebase
# 🤝help
s
I've been experiencing issues with the indexing of websites using the knowledge base for the past two weeks. As shown in the screenshot, it only indexes the title tag. One month ago, the indexer was working perfectly fine. Could you please resolve this issue? I need to deliver some work to a client of mine.
b
did it work on this website previously?
the indexer is working fine for me just now on botpress.com, for instance
s
Yeah, it worked perfectly fine on the website 1 month a go and indexed the whole site. This is the site: "https://respace.dk/"
@bumpy-butcher-41910 Do you have a estimate to when it gets fixed?
I need to provide to my client
b
this is likely an issue with the way the information is set up on the client's website, so it might not be indexable by our tool
there isn't currently an issue because it's working correctly on other websites
can you try using a "search the web" on this specific website?
you might have more success that way
s
@bumpy-butcher-41910 I have tried with the "search the web" and the answer from the knowledgebase isn't that good. I have built a web-scraper that can scrape the data correctly, with a open source called Scrapy. So i know it is possible to scrape data from their site. Alternative i will use this data, but it just makes it so more complex rather than using your intern scraper built in Botpress. If your try to scrape Apple.com using your scraper, it also just get limited information, and not all of the information on their website.
b
gotcha - thanks for that!
it's possible there's an issue with our indexing process - I'll have the team look into it
s
Thank you so much @bumpy-butcher-41910 . Will you please update me when you have figured it out, so i can get back to my client?
a
Are there a update to this? @bumpy-butcher-41910
81 Views