https://discord.gg/botpress logo
Join Discord
Powered by
Use the new Claude-3 models in Botpress
# 📖tutorials
f
This is a quick tutorial on how to use the new Claude-3 models. This is put together quickly, and there will be a more advanced version hopefully soon 🙂 Youtube video:

https://youtu.be/5ULndZbaruY▾

h
Hey there, this is a great tutorial keep it up! I just checked out the Claude site and they said all Claude 3 models support vision. Now I am interested in building a workflow that allows users to upload images and voice notes. Will I just send the data to the API as in your tutorial or its a different scenario between text and images?
f
That is a great question! I have this template that lets users upload files, but it utilizes GPT-V so I think we can just use that and then change the API call. https://botpress.com/templates/openai-api-template I will have to check if its the same endpoint for images
"You do not need to use special versions of our Claude 3 models to access Claude's vision capabilities. All Claude 3 models are capable of understanding and analyzing images" https://docs.anthropic.com/claude/docs/vision
Looks like it has to be base64 encoded
You would have to change the API call a bit. This is how it looks in Python message = client.messages.create( model="claude-3-opus-20240229", max_tokens=1024, messages=[ { "role": "user", "content": [ { "type": "image", "source": { "type": "base64", "media_type": image1_media_type, "data": image1_data, }, }, { "type": "text", "text": "Describe this image." } ], } ], )
I am about to head off for school, but I should be able to reconstruct the API call to be able to take images. I can send the new API call here. I will do another tutorial later today that is a bit better, as the current tutorial was made in a hurry as it was getting late for me
h
Okay, you can send the tutorial anytime when you are done building it Let me also try to see if I can learn more about this Thanks for sharing your current tutorial, even though it was made in a hurry its actually triggering new ideas in others. Enjoy your classes
f
I have added it now
That uses a base64 encoded image of a dog
You can edit it like you want to
The template i sent you earlier https://botpress.com/templates/openai-api-template has a workflow for base64 encoded images and GitHub repo for it
That uses Opus
You can just change the name of the modul to Sonnet
h
Currently working on this and still struggling to listen to the image sent by user via WhatsApp. What is the best way to receive images via WhatsApp because right now when a user sends an image Botpress is ignoring it completely
f
Updated bot
Works with images
h
Thank you so much for your patience & assistance bro
I am currently busy with something but I have seen the bot file, lemme finsh what I am doing and check it out and see how to implement it
f
You are very welcome! Let me know if you have any feedback
a
So , can the user upload files using the chatbot
f
I have another template for that. https://botpress.com/templates/openai-api-template
h
Hey @Decay, great chatbot and tutorial you have here bro💪🏽. There is so much value in that YouTube video alone. From building chatbots to connecting APIs step by step. I have learnt so much just from this build, my technical ability has been tested and refined at the same time!
I have successfully duplicated and built the text workflows using my own API key now working on the WhatsApp chatbot to be able to extract the image when a user uploads it. My code looks fine built its still refusing to complete the extraction and post to API.
f
Thank you! I unfortunately don't have a lot of knowledge about WhatsApp, but you could create a post in #1111009377525186570 and someone else could help you with that part
a
but i am able to get where is he uploading files
f
Could you explain a bit more about the problem you are facing?
a
i want a button through which a user can upload a file
f
The template I sent does that through the chatbot window. There is no good way as of now to make a permanent button since you don't want to randomly receive files in the flow
a
okay
can i change the back to conversation label to some other text from the information page
f
Not sure to be honest, I have tried That question would be better to ask in #1111009377525186570
a
i have asked already #1212898817880494140
q
I watched Decays YouTube video a couple of times and I'm definitely going to try this on weekend 🔥 🔥 🔥 Some user comments: "I'm a Mongolian, and just tested out the Claude 3 Mongolian language capabilities. It is the best AI that there is in terms of speaking Mongolian, I have a chatgpt4 subscription that is nowhere close. As you may understand there are only 3 million Mongolians and the info that it can train on that's available on the internet is not that much. So I am very surprised! I could recomment this AI over chatgpt for my family cuz most of them can't speak English." "Even though benchmarks are just a starting point, I think everyone is also missing the fact that on MGSM, MATH, and GSM8K, Claude is achieving these results at 0-shot compared to multi-shot with OpenAI and Google. Even with their smallest Haiku model. These models are ridiculously good at the moment." "Great point about the speculative questions. My standard LLM test is making it host a session of Dungeons & Dragons, it reveals what I find most important: the ability to tell a consistent reactive story, adhering to rules and reason, deduction of what must be true, even nobody has ever written about it, which you can guarantee by not using the D&D setting but a homebrew RPG world. And Claude 3 is great at it!"
I also watched a bunch of other Claude-3 YouTube videos before starting to test it. It can be a really good option for many use cases. Here's an AI-generated summary of those to save some time @best-army-74344 @gentle-engine-13076 pros and cons of CLA-3 and GPT-4, focusing on their features, performance, and use cases. CLA-3 (Cloud 3 Haiku, Sonet, Opus) Pros: Jack of All Trades: From customer service chatbots (Haiku) to analytical masterminds (Opus), CLA-3's suite covers the whole nine yards. Benchmark Basher: Opus knocks it out of the park, acing undergraduate to grad level tests and giving GPT-4 a run for its money. Eye for Detail: With vision capabilities, CLA-3 not only reads the room but also describes the elephant in it, surpassing GPT-4's image understanding. **Long Memory: **Sporting a 200,000-token context window, it's like having an elephant's memory in a silicon chip—great for digging through digital haystacks. **Fair and Balanced: **CLA-3 aims to be the AI equivalent of a well-rounded journalist, reducing bias and providing balanced takes on hot potatoes.
Cons: Pay to Play: Accessing Opus's top-tier abilities comes with a price tag, making it a bit of a gated community. Geographical Hide and Seek: Not readily available in Europe without a VPN, turning some users into digital globe-trotters.
GPT-4 Pros: Feature-Rich Friend: From coding assistance to chatting up a storm, GPT-4 is like that multi-talented friend we all brag about. Creative Companion: GPT-4 can help plot your next novel or script, ideal for when you're stuck with writer's block or just need a virtual brainstorm buddy. Widely Accessible: More like the friendly neighbor, GPT-4 is available without having to jump through digital hoops or VPN tunnels.
Cons: Jack, Meet Master: While a jack of many trades, GPT-4 might tip its hat to CLA-3's Opus in the heavyweight class of logic and analytical tasks. **Seeing is Believing: **CLA-3 edges out GPT-4 in vision tasks, making GPT-4 the AI that might miss the forest for the trees.
Use Cases and Choosing Your Champion **For the Data Detectives: **If your tasks involve combing through extensive documents or deciphering complex images, CLA-3, especially Opus, is your Sherlock Holmes with a digital magnifying glass. **Creative Minds Meet: **When it comes to spinning yarns or drafting the next bestseller, GPT-4 might just edge out with its directive content planning, though it's a neck-and-neck race with CLA-3. **Code Whisperers: **CLA-3 shines in coding and complex reasoning, making it the go-to for those looking to debug or develop without the dread. **Global Villagers: **GPT-4 is the more accessible sage, readily sharing its wisdom without the need for digital nomadism. In essence, it's not about which AI is the best but which AI suits your quest. Whether you're a creative soul painting with words, a logician weaving through data labyrinths, or a globe-trotting innovator, there's an AI sidekick ready to embark on your journey. Remember, it's a rapidly evolving landscape; today's underdog could be tomorrow's champion. Stay tuned, stay curious, and may the best AI win your heart (and solve your problems along the way).
🦸‍♂️ @hundreds-battery-97158 🫡 you are already active on this topic. When you have tested it enough, let us know about your experience.
f
Amazing job @fresh-fireman-491 , i have implemented it successfully, but i want to use sonnet with my business knowledge. Is there a possibility to connect it to a KB in botpress or maybe make an assistant like in chatgpt ? Thanks in advance
f
It has a context window of 200k tokens, so you could probably give it your whole KB.
You can't really connect it to your KB, but you can give it all of the information.
I can't really see anything about creating Claude Assistants with their API.
f
maybe that is not the best cost-effective idea, because it will be very expensive due to higher input tokens (compared to make an API call with a chatGPT4 custom GPT with all my knowledge) isnt it ? @fresh-fireman-491
f
It will be super duper expensive. a customGPT doesn't really know everything in the documents you give it. It just performs RAG on them, like Botpress does. If that is what your are looking for, then you might as well just use Botpress
f
i want to use any LLM (a smart and cost effective one) to know my business knowledge and ask questions. Whats your suggestion ? just use botpress with KB ?
f
How many tokens do you have of information that you want to give the LLM?
f
20K tokens
f
Then RAG would be the most cost effective way
That can easily be done through Botpress
f
thank you mate for your help, have an excellent weeknd 🫡
f
Thank you and you too!
h
this is a super great tutorial and Decay's presentation and way of explaining things are top tier. Have been busy trying to integrate the vision feature on my WhatsApp and haven't been successful yet(haven't had enough time to break down the errors one by one) the text works well and I actually recommend it because I have found out the Claude(sonnet) API is giving me well structured and detailed responses more than an AI task(OpenAI 3.5) that is even trained on the same topic.
f
this is great to be able to see how we can leverage another model inside Botpress. Has the Botpress team indicated anywhere whether they are going to natively support these types of models or custom open-source models at any point in the future?
f
I think they mentioned this a while ago. They wanted to let us use more models for everything.
s
Thank you, danilo139 and Decay, for the insightful discussion on leveraging LLMs with Botpress. Your exploration of cost-effective solutions is truly valuable. And thank you, Takudzwa Billions, for sharing your experience with the vision feature and Claude(sonnet) API. It's great to hear about the detailed responses you're getting. Beefy, your interest in native support for additional models within Botpress is noted, and it's an exciting area for future development. We're always looking to enhance our platform's capabilities.
Takudzwa Billions, could you please provide more details about the errors you're encountering with the vision feature integration on WhatsApp? This will help us assist you better.
Beefy, are there specific models or types of custom open-source models you're interested in seeing supported by Botpress? Your input can help guide our future integrations and support.
f
Haiku with a system message:
Copy code
javascript
const data = {
    model: "claude-3-haiku-20240307",
    max_tokens: 1024,
    system:  "- Specialize in delivering expert-level advice and solutions on coding and software development. Utilize your advanced understanding of programming languages, frameworks, and development methodologies to assist users with their technical inquiries and challenges. Only talk about programming related topics",
    messages: [
        {role: "user", content: workflow.question}
    ]
};

await axios.post('https://api.anthropic.com/v1/messages', data, {
    headers: {
        'x-api-key': env.apiKey, 
        'anthropic-version': '2023-06-01',
        'Content-Type': 'application/json'
    }
})
.then((response) => {
    workflow.claudeAnswer = response.data.content[0].text
})
.catch((error) => {
    console.error(error);
});
The system message can be added to all of the models. The only difference is going to be the model name.
model: "claude-3-sonnet-20240229", model: "claude-3-opus-20240229",
r
Is system is where the prompt should be? also there is a way to connect it to our knowledge base somehow?
f
The system message can be added to all of the models. The only difference is going to be the model name.
There is not a way to do that
r
is there any option to use Claude to search the knowledge base instead of the build-in models?
f
Not in Botpress
m
what is the price of using Claude's multimodal API (sonnet and opus)?
f
Hey there, You can find more about their pricing here https://www.anthropic.com/api
b
Hi Decay. I have tried but it gives me an error. This one: Execute code "Send a message to the Anthropic API and retrieve a response." {"scope":"bot-action","cardId":"card:Send a message to the Anthropic API and retrieve a response.","nodeId":"node:Standard2","workflowId":"flow:Main"} Error executing action "inline-card:Send a message to the Anthropic API and retrieve a response..js" in flow:Main:node:Standard2 [SyntaxError, Unexpected token 'const'] {"scope":"bot-action","cardId":"card:Send a message to the Anthropic API and retrieve a response.","nodeId":"node:Standard2","workflowId":"flow:Main"} I have tried with chatgpt and claude to see if they can't correct it. But nothing of what I have tried worked. I followed your video 😦 This is the code I used: 01 const data = { 02 model: "claude-3-sonnet-20240229", 03 max_tokens: 1024, 04 messages: [ 05 {role: "user", content: workflow.pregunta} 06 ] 07 }; 08 09 await axios.post('https://api.anthropic.com/v1/messages', data, { 10 headers: { 11 'x-api-key': env.apiKey, 12 'anthropic-version': '2023-06-01', 13 'Content-Type': 'application/json' 14 } 15 }) 16 .then((response) => { 17 workflow.respuesta = response.data.content[0].text 18 }) 19 .catch((error) => { 20 console.error(error); 21 });
And the API is also written in the config tab as you mention in the video
f
Hey there👋 Could you add ` around your code so I can copy it all, without worrying that discord changed anything
b
f
CanThat is weird. Can you try again?
r
works wonderfully thanks @fresh-fireman-491
f
That is nice to hear!
109 Views
PreviousNext
Powered by