Web Search integration
# 🤝helpc
careful-dream-42201
07/21/2023, 6:17 PMI am trying to build a small personalized knowledge base out of some blogs I really enjoy.
It seems that i am not allowed to add any websites that have multiple pages.
On other bot builders ive been able to add the posts that are hosted on sub pages and rock on.
It appears that botpress does not allow me to do this? am i mistaken? or are there some rules that i am not seeing in the documentation?
It will let me add the top level of the domain, but none of the lower pages.
Please help!
https://cdn.discordapp.com/attachments/1132013384972378143/1132013385148547133/image.png▾
b
bumpy-butcher-41910
07/21/2023, 6:17 PMif I'm right this, it's because inputting the top level domain will crawl all nested URLs
bumpy-butcher-41910
07/21/2023, 6:18 PMso there might be some redundancy here?
c
careful-dream-42201
07/21/2023, 6:18 PMit does not
careful-dream-42201
07/21/2023, 6:18 PMwhen asked what the blogs are, it tells me the H1
careful-dream-42201
07/21/2023, 6:18 PMbut when asked about info inside the blog, it doesnt know
b
bumpy-butcher-41910
07/21/2023, 6:18 PMhmm
bumpy-butcher-41910
07/21/2023, 6:18 PMinteresting
c
careful-dream-42201
07/21/2023, 6:19 PMhttps://cdn.discordapp.com/attachments/1132013384972378143/1132013954927956048/image.png▾
careful-dream-42201
07/21/2023, 6:19 PMeven with the tide.com example. if it has a basic sub url its not functional
careful-dream-42201
07/21/2023, 6:20 PMon other platforms the solution was to add in by hand, or by zap, the individual blog sub domains
careful-dream-42201
07/21/2023, 6:44 PMany advice on this matter would be very helpful, this is a major roadblock for me