Unlock Web Data with Ease: Introducing Browse AI f...
# 📖tutorials
q
@agreeable-accountant-7248 Browse AI is an AI-powered web automation software that allows users to extract and monitor data from websites with ease. Extract or monitor data from any website without the need for coding. Great tool for building Botpress Knowledge Bases.
There are three ways to use Browse AI: 1. Choose a prebuilt robot: Browse AI offers a range of prebuilt robots for popular use cases, which typically take 2 clicks and less than a minute to use. 2. Do it yourself (no coding needed): Users can extract or monitor data from any website using Browse AI's user-friendly interface. The platform has made it as simple and quick as possible for anyone to use. 3. Have a specialist set up your web automation: Browse AI offers free setup and onboarding support for users on Team and Company plans to ensure they have a great experience. To create a custom automation using Browse AI, follow these steps: 1. Start recording: Click on the Browse AI Recorder icon and press "Record new task". 2. Record actions: Perform the actions you want the robot to learn, such as logging into a website, searching for specific keywords, or capturing screenshots of the results. **3. Finish recording: **Once you've completed the task, stop the recording. Browse AI will then train and build a new robot based on your actions. **4. Set up the automation: **After finishing the recording, you'll be directed to the Browse AI dashboard to set up the new automation. Name the robot, schedule it to run at specific intervals, and configure any necessary input parameters After scraping the website data, you can store it and use it in CSV, webhooks, Google Sheets, Airtable, Make.com, Zapier, Pabbly, etc. Browse AI offers various features, including email notifications when the robot detects changes, before/after screenshots with highlighted changes, robot actions and video player, and robot input parameters that can be adjusted per task or per monitor.
For information and tutorials, Browse AI Help Center includes articles on getting started with web data extraction and monitoring, robot studio guides, and common technical questions. Browse AI's YouTube channel offers demos and guides on using the platform for various purposes, such as monitoring websites for changes or extracting data from specific websites.
Take 22 seconds to see how Browse AI can save you hundreds of hours.

https://www.youtube.com/watch?v=tXdp4m3y5KU

https://www.browse.ai/ Why people use Browse AI? Start for free, pay as you grow Learn in 5 minutes Scrape structured data with no code Bulk run 50,000 robots at once Emulate user interactions Extract data on a specific schedule Extract location-based data globally Handle pagination and scrolling Solve captchas Integrate with 7,000+ applications Orchestrate robots using Workflows Auto-adapt to site layout changes https://cdn.discordapp.com/attachments/1232364125728931900/1232367625074638849/image.png?ex=6629334a&is=6627e1ca&hm=8411bfae60b8418ef4cf1075c5f64ae4c94bbe062348cbaab43a21f0c9cc8c19&
a
Hi @quick-musician-29561 thank you so much for the information!!! I have just tested Browse AI and, for the moment, works really nice and it's very easy to use. Even I have seen that you can integrate directly the scraped information into google sheet! Tomorro I will try to do create bot for most of the urls I have right now and I will notice you the results. Thanks for your help!!
q
Great! ⚡ After scraping the data from the websites I need, I'm going to try different variations of storing the data and getting it into Botpress with webhooks, Google Sheets, Airtable, Make.com, and also Zapier.
a
What about convert the csv from the scrapper to vector database in pinecone and use langchain + llama3? Yesterday I did some test with the info scrapped in botpress and each query cost me 0.08€, too much for a massive use
j
WHy not just use a custome built GPT and deploy it onto Botpress ?
a
a custom built GPT for web scrapping?
j
Yeah..
Thats what i did.
I did a custome function with java or pytho, cant remember
I provided the link of a website, gave it a prompt and then it did what i told it to do.
I used an API from a software
q
@jolly-policeman-82775's idea could be one option to consider! ⭐🦾 💡
j
🙂
https://www.scraperapi.com/ That website has an API key for scraping
ill provide some code later.
Some javascript: try { String apiKey = "APIKEY"; String url = "http://api.scraperapi.com?api_key=" + apiKey + "&url=http://httpbin.org/ip"; URL urlForGetRequest = new URL(url); String readLine = null; HttpURLConnection conection = (HttpURLConnection) urlForGetRequest.openConnection(); conection.setRequestMethod("GET"); int responseCode = conection.getResponseCode(); if (responseCode == HttpURLConnection.HTTP_OK) { BufferedReader in = new BufferedReader(new InputStreamReader(conection.getInputStream())); StringBuffer response = new StringBuffer(); while ((readLine = in.readLine()) != null) { response.append(readLine); } in.close(); System.out.println(response.toString()); } else { throw new Exception("Error in API Call"); } } catch (Exception ex) { ex.printStackTrace(); }
But if you want to scraper info quickly, i advice you to use a google extension called "Open Web Scraper" url: https://webscraper.io/documentation/open-web-scraper
a
this is the one I am using right now. I have found that is very powerful to get data even if you have to paginate. Thanks for the info!!!
j
I'm so glad to hear that the scraper is working out well for you and proving to be powerful, especially with its pagination capability! If you have any questions or run into any challenges as you continue to explore its features, feel free to reach out. Happy scraping, and I look forward to hearing about your success and any interesting data you uncover.
chatgpt lol
a
thnks. I have some troubles to join different data into the same file. For example: I have a website with different events so I want to iterate through all of them and extract main info: - Date - Description - Place Sometimes when I create the selector they get grouped automatically so I have in the same row, for each event, Date, Description and Place But sometimes I get Date in one row Date, in other row Description and in other row the Place. Do you know the best way to get different information fields from the same element in just one row?
j
please provide me with your website url
I will help u out.
a
this is my sitemap: {"_id":"Culturama","startUrl":["https://www.malaga.es/culturama/2157/com1_ct-0/com1_fb-3/com1_cb-0/agenda"],"selectors":[{"id":"Elemento","multiple":true,"parentSelectors":["_root","Paginas"],"selector":"_parent_","type":"SelectorElement"},{"id":"Fecha","multiple":false,"parentSelectors":["wrapper_for_Fecha_Plan"],"regex":"","selector":".fecha > time","type":"SelectorText"},{"id":"Plan","multiple":false,"parentSelectors":["wrapper_for_Fecha_Plan"],"regex":"","selector":"span.h4","type":"SelectorText"},{"id":"Lugar","multiple":false,"parentSelectors":["wrapper_for_Fecha_Plan"],"regex":"","selector":"span.lugar","type":"SelectorText"},{"id":"Paginas","paginationType":"clickMore","parentSelectors":["_root","Paginas"],"selector":".listado > nav .next a","type":"SelectorPagination"},{"id":"wrapper_for_Fecha_Plan","multiple":true,"parentSelectors":["Elemento"],"selector":"div.elementoListadoPortada","type":"SelectorElement"}]} It seems like itearate over the different pages but always get the same info: https://cdn.discordapp.com/attachments/1232364125728931900/1232995387665813564/message.txt?ex=662b7bf0&is=662a2a70&hm=498e7c57d06fd3101793e7961e97c1789d2505b1762046f1d251c13fecfbc6fa&
j
What are you tring to exttract lol.
Provide me with your site url please
a
https://www.malaga.es/culturama/2157/com1_ct-0/com1_fb-3/com1_cb-0/agenda When you put your mouse over eache event you can get the date, the name of the event and the place. I try to extract that info from each event
c
webscraper.io is another one that might be cheaper to use 🙂
5 Views