Case study
Cross-Service Data Lineage Engine
A distributed platform that auto-discovers how data flows across Spring Boot microservices, builds a lineage graph, and answers impact, compliance and “where does this field go?” questions.
Architected a distributed data lineage platform with 5 microservices that auto-discovers data flows across Spring Boot codebases using JavaParser AST analysis, builds lineage graphs in Neo4j, and enables impact analysis, GDPR compliance discovery, and anomaly propagation tracking.
Implemented an impact analysis engine using Neo4j Cypher queries and BFS traversal, letting developers simulate schema changes and identify every affected downstream service, database and data transformation — with auto-generated migration checklists.
Engineered an async scan pipeline on Apache Kafka with a fan-out consumer pattern (Graph Service and Search Service consume the same topic via different consumer groups), plus Redis caching and Redisson distributed locks for safe concurrent graph updates.
Built a query interface with Spring AI and an Ollama LLM so users can ask “Where does user email flow?” — with automatic intent classification, Cypher query generation and human-readable answers.
Designed a module that finds PII fields across all registered services using both annotation scanning and AI-powered field-name analysis, then generates deletion plans and compliance reports.