Is there a limit to the amount of text that can be...
# 🤝help
w
Working with a KB, I added some data to a Web page source, and deleted and re-ingested the page within the KB interface. However when I queried the KB, the new data was not reflected, and when I viewed the source text (by clicking on the KB website source name), the text appeared to be truncated shortly after the new data began. Is there a limit to the size or text volume of a single KB web page (and/or website) source? If so, what is that limit? EDIT: I mean, a limit aside from the overall 5000-vector general KB limit per bot
b
no, there isn't a set limit aside from vector storage
I'm not sure why the responses to your queries would be truncated
are you able to replicate this issue on a new bot?
w
@bumpy-butcher-41910 thanks mucho for the reply! Since there's no page-length limit (beyond general vector storage limits), the issue is probably on the source end (Goog sites, in this case) -- thanks again for your help, I'll troubleshoot it!
b
is the information all text? you could try saving the page as a PDF and seeing if it's working properly that way
it won't search the web anymore, but just for debugging
w
@bumpy-butcher-41910 yeah it is all text, and it's a static "web page" as opposed to a "search the web" -- good idea on the PDF -- While I have you "on the line," do you happen to know whether it's possible to programmatically trigger a "refresh" (something like a re-indexing of the page) on a KB source that is a "website" or "web page?" I'm trying to figure out how to handle a case where a source page is updated on an ad hoc basis
b
it's a good question
currently we don't natively support dynamic caching as you've described here
but you're not the first person to ask for it
it's coming "soon" in an upcoming release
keep an eye on #1111016943370113174 or our livestreams if you want to be the first to know when it's released!
w
awesome, thanks again for your help! Molto appreciate it! I'll keep an eye out for that upcoming feature!