Website Scraper Information
# 🤝help
Question - does the web scraper actually scrape the whole page or does it just scrape metadata? For some reason, it feels like the knowledge base I have with website data (which there are over 1,000 articles on) returns nothing valuable.
Hey @careful-kitchen-31087 , Can you drop me the link to the website?
Also you can use the "Search the Web" if your website is indexed by Bing
c it loads all 1000 pages but the knowledge content is title and meta only
But the bot starts from the root of the sitemap, can you see the links in the sitemap?