https://discord.gg/botpress logo
#🤝help
Search the web Kb problem
# 🤝help
a

agreeable-accountant-7248

02/15/2024, 1:20 PM
Hi Sorry if something related is explained before, I have searched and didn't find the same question. I have a website knowledge base that works perfect but those websites update their content daily so I want to explore the Search the web option to be able to have the content updated every time the user uses the bot. I have changed the KB in the Node to the KB that uses Search the web but every time I ask something i get this: Knowledge Query No content matched the query Searched in Planes y Eventos Web Logs (1 items ) Skipping KB agent because none of the 1 KBs provided search results I don't know what I'm doing wrong, anyone could help me? Thanks in advance!
f

fresh-fireman-491

02/15/2024, 7:14 PM
Hey Did Robert help you in #1108396291060408352 with this issue? if yes did he manage to solve it?
a

agreeable-accountant-7248

02/16/2024, 12:40 AM
Yes, he helped me but still don't understand the reason of my problem. It seems it's a indexation issue but don't know how can I solved it as I only want to be able to scrap content from webs that are not only the main domain like malagamycity.com but malagamycity.com/eventos-en-malaga. I really would love to understand why the second url does not work because in the first url, even it has indexed all the content of the web, has indexed content that I don't want to show to the user because it's old. I will keep looking for the best way to do it Thanks
f

fresh-fireman-491

02/16/2024, 6:17 AM
If you use "Search the web" you can only use the domain url not the sub domain url. There should be one other you can use, and I think it's called "website". If you enter the domain url there you can pick the pages you want to use.
a

agreeable-accountant-7248

02/16/2024, 1:12 PM
Yeah, that is what I did first time, and works really good, but once you put the website you create a static image of the content of the website so if I upload new content to my web, it will not appear in my Knowledge Base. That’s why I started to look for alternatives like Search the web. At the moment the best solution (not the optimal) it’s to load every day all the websites I need to have updated my Knowledge Base
f

fresh-fireman-491

02/16/2024, 1:19 PM
What is the issue with using the whole website as the KB?
a

agreeable-accountant-7248

02/17/2024, 7:01 PM
as my website has a lot of events and plans, when those plans are outdated still are indexed so I don't want to index whole website. I only want to scrap the page where appears plans that are active right now or in the future. Don't know if I am explaining myself correctly
f

fresh-fireman-491

02/17/2024, 7:11 PM
You are explaining yourself perfectly and I understand that. As of now you would have to index the pages every time when using a specific page.
2 Views