Taroon Kumar Ray

Provide hands-on technical and delivery leadership to two data engineers, guiding technical design, work decomposition, implementation, testing, code reviews, and production readiness. Partner with engineering managers, Product, Analytics, and business stakeholders to refine requirements and translate PRDs into technical design documents, architecture decisions, and executable delivery plans. Lead technical estimation and sprint planning for data-platform initiatives, allocate technical ownership, track delivery progress, resolve engineering blockers, and communicate risks and dependencies to stakeholders. Defined and implemented enterprise-grade data architecture for a near real-time analytics platform using AWS DMS, Kafka, and Spark Streaming. Designed end-to-end lakehouse architecture (Bronze Silver Gold) using Iceberg on S3, enabling schema evolution and ACID guarantees. Reduced data latency by 70% and compute cost by 65% through architecture optimization and EMR Serverless tuning. Architected One Big Table (OBT) strategy for self-service BI (Looker), improving business adoption by 50%. Implemented data governance and fine-grained access control using AWS Lake Formation. Evaluated Athena vs Pinot for serving layer and drove architectural decisions based on latency vs cost tradeoffs. Architected a RAG-based intelligent Q&A system for facility operations - designed the full pipeline including LLM-powered ingestion (Claude API), semantic embedding generation (sentence-transformers), cosine similarity retrieval, and role-aware response generation with conversational memory. Leverage AI-augmented engineering (Claude Code, Copilot) daily across the full development lifecycle - achieving measurably faster delivery on data platform tasks.

Designed scalable data models and warehouse architecture in Snowflake to support analytics workloads. Led schema evolution strategy for 20+ pipelines, ensuring zero downtime and backward compatibility. Built data transformation architecture using dbt, leveraging modular design and incremental models for efficient processing. Implemented event-driven data workflows using AWS services and Python-based orchestration to enable reliable data processing.

Led the day-to-day delivery of a 10-member data engineering team, including work allocation, progress tracking, dependency coordination, and delivery-risk management. Organised regular team huddles to review progress, identify blockers, coordinate priorities, and maintain visibility against delivery commitments. Mentored engineers through technical guidance, design discussions, code reviews, and hands-on troubleshooting across AWS, PySpark, Airflow, Snowflake, and Data Vault 2.0. Contributed to performance discussions by providing structured inputs on delivery, ownership, technical quality, collaboration, and development areas. Partnered with product, architecture, and business stakeholders to translate requirements into technical designs and executable delivery plans. Owned data platform architecture and scalability roadmap for an AWS-based analytics platform. Designed and implemented Data Vault 2.0 models to ensure enterprise-grade data consistency and auditability. Reduced manual effort by 90% through architecture-led automation using custom Airflow plugins. Led migration of legacy systems to cloud using AWS Glue and PySpark, ensuring reliability and performance.

Led data migration architecture from Oracle to AWS Aurora (PostgreSQL), ensuring minimal downtime and resolving performance bottlenecks during transition. Redesigned and optimized 30+ stored procedures and ETL workflows, improving query performance and supporting scalable analytical workloads. Developed data models for complex reporting use cases, enabling efficient querying and downstream analytics. Identified and resolved performance and data consistency issues, ensuring reliability of business-critical systems. Collaborated with cross-functional teams to deliver enhancements and stabilize production workloads.

Designed and developed analytics data models and marts using dbt, achieving 99% data accuracy through comprehensive testing and validation frameworks. Built end-to-end data pipelines using AWS Glue and PySpark, consistently meeting 100% SLA requirements for business-critical reporting. Engineered scalable data ingestion and transformation pipelines for retail systems, enabling enhanced BI and reporting capabilities. Developed custom data extractors and API integrations (Airflow + REST/ORDS) to onboard external vendor data (e.g., payment and logistics systems). Streamlined order processing workflows by integrating OMS and order booking systems, improving operational efficiency and data consistency. Ensured data reliability and pipeline stability through monitoring, debugging, and continuous enhancements in production environments.

Designed and implemented custom monitoring and automation solutions extending beyond Guardium's native capabilities, improving operational efficiency. Analyzed large-scale Guardium audit data (Oracle DB) to generate actionable insights for both operational and business stakeholders. Optimized data processing workflows using Python (multi-threading) and SQL, improving performance across Oracle and SQL Server environments. Developed automated alerting systems for STAP agent failures, enabling proactive issue detection and reducing downtime. Maintained and governed whitelist data controls, ensuring compliance and minimizing false positives in security monitoring. Collaborated with cross-functional teams to enhance data-driven monitoring and operational reliability frameworks.

Optimized performance of 40+ analytical ETL pipelines, improving processing efficiency and reducing execution time across enterprise data workflows. Analyzed and resolved performance bottlenecks in data pipelines, enhancing scalability and reliability. Developed and maintained data transformation logic to support reporting and analytics use cases. Collaborated with cross-functional teams to deliver data-driven solutions aligned with business requirements. Built foundational expertise in ETL optimization, SQL tuning, and data processing frameworks.