HttpRequest Automate

Data Analysts: Need to automate file retrieval and processing to save time and reduce manual effort.
Developers: Looking for a way to integrate AI capabilities with document management systems like Supabase.
Businesses: Want to enhance their document handling processes by using AI for content extraction and querying.
Researchers: Require efficient methods to analyze large sets of documents quickly.

This workflow automates the retrieval, processing, and analysis of files stored in Supabase, addressing issues such as:

Inefficiency: Manually retrieving and analyzing documents is time-consuming.
Duplication: Prevents duplicate processing of files by comparing newly fetched files with existing records.
File Type Handling: Supports multiple file types (e.g., PDF, text) and processes them accordingly.
Contextual Retrieval: Enables AI-powered querying of documents, making it easier to extract relevant information based on user requests.

Manual Trigger: The workflow starts when the user manually triggers it.
Fetch File List: Retrieves a list of files from Supabase storage.
Aggregate Data: Combines and prepares the retrieved file data for processing.
Loop Over Items: Iterates through each file to handle them individually.
Condition Check: Evaluates if a file has already been processed to avoid duplication.
Download Files: Downloads files that need processing based on their type.
File Processing: Uses different nodes to handle various file types:
- PDF Extraction: Extracts text from PDF files.
- Text Handling: Directly processes text files.
Text Splitting: Splits large text into manageable chunks for better processing.
Generate Embeddings: Creates vector embeddings for the processed content using OpenAI.
Store in Supabase: Inserts the vectorized data into a Supabase vector store for later retrieval.
AI Chatbot Integration: Allows users to interact with the system by querying the documents through an AI agent.
Result Return: Outputs the relevant information based on user queries.