Intelligent Data Dictionary – AI-Powered Multi-Database Discovery Platform

Intelligent Data Dictionary (IDD) is an AI-powered metadata intelligence platform that helps data teams stop flying blind across their databases. It connects to multiple heterogeneous data sources — PostgreSQL, MySQL, Snowflake, MongoDB, and BigQuery — and transforms raw schema information into a rich, searchable, and conversational knowledge base.

The Problem We Solve: Data teams waste hours hunting for what tables exist, what columns mean, where data comes from, and whether it can be trusted. This tribal knowledge lives in Slack messages, outdated wikis, and people's heads — not where engineers need it.

What IDD Does:

• 🔌 Multi-Database Connectivity — Connect to PostgreSQL, MySQL, Snowflake, MongoDB, and BigQuery from a single unified dashboard using a secure, plug-and-play BaseConnector architecture with Fernet-encrypted credentials stored at rest.
• 🤖 AI-Powered Chat Interface — Ask questions in plain English. IDD intelligently detects whether to run a RAG (documentation lookup) or SQL query, generates dialect-aware SQL, executes it safely, and returns a summarized result with a confidence score.
• 📊 Automated Schema Syncing — One-click or background Celery sync extracts tables, columns, relationships (foreign keys), and data types directly from source databases.
• 🧬 Data Quality Profiling — Built-in profiling surfaces null percentages, distinct counts, min/max/mean values, and flags anomalous columns automatically.
• 🗺️ Data Lineage Tracking — Visual lineage graph traces how data flows through your systems, enabling impact analysis for schema changes.
• 📖 Auto-Documentation — LLM-generated descriptions for every table and column, stored as a searchable vector knowledge base (pgvector) namespaced per connection.
• 🔐 Secure by Default — All credentials encrypted at rest; role-based access patterns built into the API layer.
• 🐳 Fully Dockerized — Six-container stack (React frontend, FastAPI backend, Celery worker, Celery beat scheduler, Redis, PostgreSQL) deployable in a single docker-compose up command.

Tech Stack: FastAPI · React + Vite · PostgreSQL + pgvector · Celery + Redis · SQLAlchemy · Nginx · Docker

IDD is built for modern data teams who need to spend less time hunting metadata and more time building.