Import Productboard Notes, Companies and Features into Snowflake

Target Audience

• Product Managers: To keep track of insights and features in product development.
• Data Analysts: For analyzing product data and generating reports.
• Marketing Teams: To stay updated on product features and customer feedback.
• Developers: To integrate product insights into development processes.
• Business Executives: To monitor product performance and strategic decisions.

Problem Solved

This workflow automates the process of importing product notes, companies, and features into Snowflake, ensuring that data is consistently updated without manual intervention. It addresses the challenges of:

• Data Silos: By integrating various data sources into a single repository.
• Time Consumption: Reducing the manual effort needed to collect and input data.
• Data Accuracy: Ensuring that the latest information is always available for analysis and decision-making.

Workflow Steps

1. Schedule Trigger: The workflow is triggered weekly on Mondays at 8 AM.
2. Empty Tables: Clears previous data from the PRODUCTBOARD_NOTES, PRODUCTBOARD_COMPANIES, and PRODUCTBOARD_FEATURES tables in Snowflake to ensure fresh data.
3. Fetch Data: Retrieves data from the Productboard API for notes, companies, and features, handling pagination to ensure all data is captured.
4. Data Mapping: Maps the fetched data into a structured format suitable for Snowflake:
   • Notes are mapped to include details like note_id, note_title, note_created_at, etc.
   • Companies are mapped to include company_id, company_name, and company_domain.
   • Features are mapped with feature_id, feature_name, and feature_status.
5. Data Insertion: Updates the Snowflake tables with the newly mapped data.
6. Count Insights: Counts the number of notes added in the last 7 days and those that remain unprocessed, preparing this data for notification.
7. Slack Notification: Sends a summary of the updates to a designated Slack channel, highlighting the number of new insights and unprocessed notes.