LangChain Automate

This workflow is ideal for:

• Web Developers: To automate the extraction of social media links from company websites, saving time in manual data collection.
• Digital Marketers: To gather insights on competitors' social media presence and improve marketing strategies.
• Data Analysts: To compile and analyze social media data for reporting purposes.
• Small Business Owners: To easily collect and manage social media profiles for their businesses.
• Researchers: To efficiently gather data for academic or market research.

This workflow addresses the challenge of manually collecting social media profile links from various company websites. It automates the process of crawling web pages, extracting relevant URLs, and organizing them into a structured format, significantly reducing the time and effort required for data gathering.

1. Manual Trigger: The workflow begins with a manual trigger, allowing users to initiate the process when ready.
2. Get Companies: It retrieves company names and websites from a Supabase database, ensuring the data is up-to-date.
3. Set Parameters: It extracts only the necessary fields (name and website) for processing.
4. Crawl Website: The workflow utilizes an AI agent to crawl each website, extracting social media URLs while following links on the page.
5. Process URLs: It retrieves and processes all URLs found on the website, filtering out duplicates and invalid links.
6. Convert HTML to Markdown: For text extraction, the HTML content of the website is converted to Markdown format for easier handling.
7. Merge Data: The workflow combines the extracted social media links with the original company data, creating a comprehensive dataset.
8. Insert into Database: Finally, the structured data is inserted back into a Supabase database for easy access and management.