do i need to connect to openai API to use openai chatgpt as a fallback if bot cannot find answer in knowledge base?

No you don't you can use the AI Task card

thanks