Automated Real-time Notion Integration for Dynamic RAG
For the RAG on living data workflow, automate the integration of real-time data from Notion into a dynamic knowledge base. This workflow utilizes advanced AI capabilities to process and embed content, enabling efficient retrieval and interaction through a chat interface. By running every minute, it ensures that your knowledge base is always up-to-date, enhancing accessibility and responsiveness for users seeking information.
View Large Image
Workflow Overview
For the RAG on living data workflow, automate the integration of real-time data from Notion into a dynamic knowledge base. This workflow utilizes advanced AI capabilities to process and embed content, enabling efficient retrieval and interaction through a chat interface. By running every minute, it ensures that your knowledge base is always up-to-date, enhancing accessibility and responsiveness for users seeking information.
Who should use this workflow
- Data Scientists: To automate the embedding and retrieval of data from Notion for analysis and insights.
- Developers: To integrate Notion data with machine learning models seamlessly.
- Business Analysts: To ensure that the latest information from Notion is always available for decision-making.
- Content Managers: To keep track of document updates and ensure that the latest content is embedded for easy access.
What problem does this workflow solve
This workflow automates the process of updating and embedding data from Notion into a vector store, ensuring that the latest information is readily available for querying. It addresses issues such as:
- Data Redundancy: By deleting old embeddings, it prevents duplicate entries in the vector store.
- Timeliness: Automatically retrieves updated pages from Notion every minute, ensuring data is current.
- Efficiency: Processes documents in batches, making it scalable for large datasets.
Detailed explanation of the workflow process
- Trigger: The workflow is initiated either by a schedule every 1 minute or by updates in the Notion database.
- Get Updated Pages: Retrieves pages that have been updated in the last minute from the Notion database.
- Input Reference: Serves as a placeholder for processing each updated page individually.
- Delete Old Embeddings: Removes any existing embeddings related to the updated pages to maintain data integrity.
- Get Page Blocks: Fetches all blocks of content from the updated Notion pages.
- Concatenate to Single String: Combines all fetched content into a single string for easier embedding.
- Token Splitter: Splits the combined content into manageable chunks of 500 tokens for embedding.
- Embeddings OpenAI: Generates embeddings using OpenAI's model.
- Store in Supabase: Inserts the new embeddings into the Supabase vector store for retrieval.
- Vector Store Retriever: Retrieves the embeddings for use in a question-and-answer chain.
- OpenAI Chat Model: Facilitates chat interactions, allowing users to ask questions based on the embedded data.