Taroon Kumar Ray

Revamping the data architectural footprint, moving away from Snapshot based processing to near real-time processing using AWS DMS to ensure faster data delivery to the downstream systems. Leveraging EMR Serverless and Spark Streaming for faster jobs and ready Bronze and Silver layers. Using Iceberg tables to build a unified data platform, and Athena to build data marts with necessary business transformations. Glue jobs orchestrated via Airflow for timely data delivery. Built Dapr Consumer Client to consume Application Messages from Kafka topic and trigger Airflow jobs. Collaborated with other teams to deliver solutions. Leveraging Looker to build reports integrated into the product for customers.

Snowflake is the data warehouse. Leveraged Snowstream, Snowpipe, Snowtask, Dynamic Tables, and Materialized Views for ETL tasks. Built SnowProc using Python for HouseKeeping, rolling data into S3, using AWS SNS, SQS, and Dagster job for Slack notifications. Used Fivetran to pull data from multiple sources. Dagster jobs perform tests and monitor Snowflake tables for data correctness. Slack integration with Dagster for alerts. Used DBT for data transformation models. Optimized queries to improve performance and reduce server load. Used Snowflake's Zero Copy Cloning for internal testing.

Developed and implemented data engineering strategies to improve scalability, performance, and accuracy of data pipelines. Managed design, development, and implementation of new data models. Promoted customer satisfaction by resolving problems. Built airflow plugins and enhanced automated solutions, reducing manual effort by 90%. Reviewed code for quality standards. Used AWS Lambda for file metadata registration. Onboarded Legacy Supplier data to Cloud using AWS Glue and PySpark. Migrated legacy systems to cloud platforms with DataVault modeling. Built business marts using dbt with necessary transformations. Learned Apache Iceberg and Kafka for real-time data processing. Used Airflow for pipeline orchestration. Consumed Supplier APIs using Python REST. Scraped data from HTML pages. Created documentation and training materials for knowledge transfer.

Participated in migration of data from Oracle to AWS Aurora. Mitigated performance bottlenecks post migration from Oracle PL/SQL to PostgreSQL PL/pgSQL. Resolved business bugs, new requirements, and enhancements.

Implemented OMS (Order Management System) and OB (Order Broker) for Intermix Brand. Developed REST APIs using ORDS (Oracle Rest Data Services) for web service integrations with vendors such as Cybersource, Clutch, DEG, UPS.

Built automated solutions for manual tasks and Guardium limitations. Developed solution to monitor STAP restarts and alert Operations Team. Analyzed Guardium data for business and team insights. Built solution to monitor and whitelist users in Guardium. Worked on SQL, Python, Oracle, SQL Server, Guardium, and Python multithreading.

Used PROC SQL, PROC SORT to resolve issues. Monitored weekly and monthly ETL jobs. Created and modified Linux scripts as per project and client requirements. Worked on SQL, Oracle, SAS, and UNIX Shell Scripting.