Hey fellow developers! This is a project that I have been working on, and planning, for a time now and I need your suggestions! So I am in the process of developing an AI agent that can scan a person using OpenAI Vision and then recommending sizes for clothing based on the image. So I have been thinking and talking a lot with partners about it and our idea is that we want to capture measurments from the user within Botpress, send them with Make to GPT vision along with a few pictures (all from the same distance and angle) to use them as reference. Then we will basically have a "KB" of images with corresponding sizes that the AI can use as reference in determining the sizes. I also have a full spreadsheet with measurements for each size. So my questions are: Do you know how I can send images to Make using Botpress? Idk if that is possible, but do you have any work around? Secondly, do you know how I can create a "KB" of the images that I will use as reference so that each run, the AI will look through the library of images and try matching the user's image to one from the KB? Any help, tips, insights, and aspects to consider would be highly appreciated!

Maybe @quick-musician-29561 aka the Goat can help😂

Hey there, I am not sure if GPT-V is the correct choice here. I would use ML for this. It's a topic I would advise you to try out in any way so I think this project is perfect for you. Not the easiest first project, but you would learn a lot!

Thanks for your feedback! Do you have any idea on how to store the images to use as reference?

If you decide to use GPT-V: I would just use a bucket on AWS and then add them to the API call

And how about with ML?

Are you familiar with ML?

No

You could use something like PyTorch or TesnorFlow. You should consider using feature extraction and similarity scoring techniques. You could do it traditional machine learning methods or newer deep learning approaches. The pros with developing a custom model using machine learning, especially one based on convolutional neural networks (CNNs) is that it would likely be more effective. You could tailor it specifically to recognize and interpret various body shapes and sizes more accurately. You would need a robust dataset of body images tagged with precise measurements to train such a model. You could use tools like [TensorFlow](https://www.tensorflow.org), [PyTorch](https://pytorch.org) or even specialized services like [Google AutoML Vision](https://cloud.google.com/automl) can be used to train a model capable of interpreting body measurements from images.

Okay so @fresh-fireman-491, I am doing this and Ramón wants to help me out a bit as well, you kinda know our levels I guess, what would be your advice on an approach?

Is this a paid project or for fun?

Paid

You got it! I recommend starting with a proof of concept using OpenAI Vision and a small KB of reference images. As you gather more data and insights, you can iterate and refine your approach, potentially exploring machine learning techniques if needed.

Alright I get it

You should educate yourself on ML, specifically image processing, even if you decide to stick with OpenAI.

Alright bro

How precise should it be?

So the idea is that this brand sells extreme working clothes for first responders, etc. The price of an item is high, like 2-3G for a jacket. They have difficulty in their business because customers order the wrong sizes due to how their body is built. So the idea is that the AI can estimate the measurements (such as height, width, etc.) while also considering other factors such as "oh this guy has a short build but very wide shoulders, so a small jacket would fit him to tight, then recommend a medium size"

ML would be ideal to be honest.