Supreeth Gowda

Led the production implementation of a Snowflake-to-Salesforce integration using Amazon AppFlow, defining the integration architecture, authentication strategy, and deployment approach while coordinating Salesforce, DevOps, QA, and Data Engineering teams to deliver a secure, production-ready solution. Designed and evolved a modern data platform using Snowflake, Airflow (MWAA), dbt, Fivetran, Power BI, and AWS, building reusable configuration-driven ingestion and orchestration frameworks supporting scalable batch and streaming data pipelines. Administered the Snowflake platform during its adoption across the engineering team, implementing RBAC-based access control, governance, and workload management to support secure and scalable data operations. Built Snowflake cost and utilization dashboards to identify expensive queries and warehouse usage patterns, enabling data-driven optimization that reduced critical pipeline runtimes by nearly 50% while lowering unnecessary compute costs. Automated infrastructure deployments using Terraform and developed centralized error logging, monitoring, and alerting frameworks, improving platform reliability and operational visibility. Developed custom Python utilities to parse and validate CSV and Excel files before ingestion into Snowflake, improving data quality and ingestion reliability.

Providing independent data engineering consulting for a healthcare provider, designing and maintaining a data aggregation solution that consolidates disparate datasets into standardized reporting formats for business stakeholders. Built Python-based data processing and reporting workflows to automate data consolidation, improve consistency, and support ongoing business reporting requirements.

Supported a centralized enterprise data warehouse managing 45-50 PB of data, providing production support and technical guidance for data integration across multiple engineering teams. Managed and optimized thousands of ETL workflows supporting the migration from Amazon Redshift to a modern Data Lake architecture while ensuring high platform reliability. Built Apache Spark on EMR pipelines to offload and transform Redshift data into Parquet, enabling Redshift Spectrum and a scalable Data Lakehouse architecture. Contributed to an enterprise AWS Glue Catalog sharing platform using AWS Glue, Lake Formation, Resource Access Manager (RAM), and Redshift Spectrum, reducing data duplication and improving cross-team data sharing.

Designed and built an automated CCPA data deletion framework that removed customer data across downstream systems within minutes, reducing a 45-day regulatory SLA to near real-time execution. Evaluated Apache Hudi, Delta Lake, Snowflake, and Athena CTAS through proof-of-concepts before selecting the optimal production implementation. Designed and implemented an enterprise AWS Data Lake using AWS Glue, Athena, EMR, Glue Data Catalog, and PySpark for large-scale data ingestion and transformation. Built and orchestrated scalable data processing workflows using Apache Airflow, ensuring reliable and timely data availability.

Built a PySpark validation framework performing byte-level data comparison across distributed systems, uncovering data inconsistencies that traditional sample-based testing failed to detect. Performed root-cause analysis through automated error log investigation, identifying defects in upstream processing logic before production deployment. Designed and implemented data pipelines using Python, PySpark, MySQL, AWS, Apache NiFi, Docker, Ansible, Ganglia, and GitHub. Developed and presented engineering innovations at internal hackathons involving automation, voice integration, testing tools, and PySpark-based solutions.

Research Assistant (Data Mining and Software Development) - co-advisor was the President of the International Educational Data Mining Society. Developed Java-based automation using RapidMiner and Weka APIs to build machine learning detectors and engineer complex features from educational datasets. Built feature extraction pipelines from log files and relational databases to identify student learning behaviors including boredom, confusion, frustration, and engagement within Intelligent Tutoring Systems (ASSISTments).

Participated in performance testing strategy, requirements analysis, and test plan development for enterprise web applications using HP LoadRunner. Developed automated performance test scripts using HTTP/HTML, Ajax, C, and Java, incorporating reusable components and robust error handling. Configured monitoring with SiteScope, executed load, stress, spike, and endurance testing, analyzed system bottlenecks, and collaborated with development teams to improve application performance.

Built automated validation frameworks using AWS Glue, EMR, Spark, and Python to verify large-scale ETL pipelines across data warehouses, databases, and Amazon S3.