profile-pic

Narayana Murthy Gopisetti

Experienced Senior Data Engineer with a strong track record of building and optimizing large-scale data pipelines across cloud platforms like GCP and Azure. Proficient in PySpark, Spark, SQL, Hive, Databricks, and Databricks SQL for developing robust ETL/ELT workflows that support marketing attribution, site traffic analysis, and business intelligence reporting.

Hands-on experience with Airflow for orchestration, Power BI and Tableau for visualization support, and PostgreSQL for downstream reporting needs. Skilled in using Hadoop, BigQuery, DataProc, and Scoop for data processing, with expertise in integrating source systems like Traffic360 into unified analytics layers.

Experienced in implementing DevOps practices using GitHub, Looper Pro, and Concord to enable automated CI/CD workflows and seamless deployments to Google Cloud Storage. Familiar with Azure Data Factory, Azure SQL, and Data Lake architecture, with a background in using Medallion architecture and ingestion frameworks for scalable data management.

Strong understanding of data quality, pipeline monitoring, and incident alerting via Slack and email integrations. Certified in Databricks (Associate, Professional, Spark Developer), Microsoft Azure Data Engineer (DP-203), and Snowflake (SnowPro Core), with a focus on reliability, performance tuning, and delivering business-impacting data solutions.

  • Role

    DATA & Databricks ENGINEER

  • Years of Experience

    4.42 years

Skillsets

  • Data Analysis
  • Data Processing
  • Big Data

Professional Summary

4.42Years
  • May, 2024 - Present1 yr 11 months

    Senior Software Engineer

    Tredence Inc.
  • Dec, 2022 - May, 20241 yr 5 months

    Associate Consultant

    Celebal Technologies
  • Nov, 2021 - Dec, 20221 yr 1 month

    Data Engineer

    Futurense Technologies

Applications & Tools Known

  • icon-tool

    Spark

  • icon-tool

    SQL

  • icon-tool

    Hive

  • icon-tool

    Databricks

  • icon-tool

    Python

  • icon-tool

    Power Bi

  • icon-tool

    Hadoop

  • icon-tool

    Azure

Work History

4.42Years

Senior Software Engineer

Tredence Inc.
May, 2024 - Present1 yr 11 months

Associate Consultant

Celebal Technologies
Dec, 2022 - May, 20241 yr 5 months

    Utilizing Databricks and Data Factory for ETL operations, handling data from diverse sources including Qlik files, SAP, Bizom, and SQL Servers.


    - Implemented Medallion architecture in Databricks, ensuring structured data processing from raw to gold layers, enhancing data reliability and accuracy.


    - Configured a monthly refreshed GST report using Power BI, providing stakeholders with insightful analytics.


    - Managed JSON data efficiently, employing techniques such as explode related queries to handle nested structures effectively.


    - Addressed data skewness using advanced techniques like salting, ensuring balanced data distribution and optimized query performance.


    - Employed broadcast joins to optimize performance and improve query execution efficiency.


    - Utilized the qualify method for efficient window function subqueries in SQL queries.


    - Applied repartitioning and coalescing techniques to optimize memory usage and mitigate out-of-memory issues.


    - Implemented Z-ordering and optimize commands to tackle the challenge of small files, optimizing data storage and query execution efficiency.


    - Overall, focused on delivering reliable, efficient, and scalable data solutions tailored to meet client needs.utilizing Databricks an

Data Engineer

Futurense Technologies
Nov, 2021 - Dec, 20221 yr 1 month


    Python (Programming Language)

    Microsoft Power BI

    DAX

    Data Processing

    Azure Databricks

    Hive

    Shell Scripting

    DWH

    Microsoft SQL Server

    Data Warehousing

    SQL

    Microsoft Azure

    Amazon Web Services (AWS)

    Query Writing

    Microsoft Excel

    Apache Spark

    GitHub

    Data Visualization

    Stored Procedures

    Distributed Computing

    Data Lineage

    Data Ingestion

    Sqoop

    Airflow

    Hadoop

Achievements

  • Streamlined Xendit's data analytics
  • Migrated Qlik and SAP BW data models to Azure platform
  • Managed migration of Starburst Presto DB to Databricks SQL
  • Modernize Teradata EDW to Azure Data services
  • Implemented SCD1 logic

Major Projects

5Projects

Xendit

    Streamlined Xendit's data analytics by seamlessly migrating a collection of Trino SQL queries embedded within LookML scripts to Databricks SQL, ensuring compatibility and enhanced performance.

Godrej

    Migrated Qlik and SAP BW data models to Azure platform for enhanced reporting and analytics capabilities.

Meesho

    Managed the successful migration of a clients Starburst Presto DB to Databricks SQL, overcoming technical complexities and optimizing query performance.

Siam Commercial Bank

    Modernized Teradata EDW to Azure Data services and optimized data processing.

SCD 1 Logic Implementation

    Implemented SCD1 logic to capture and track updated data, transferring files from LFS to MySQL and using Hive and Sqoop.

Education

  • BTech-Information Technology

    Andhra University (2022)

  • Intermediate-MPC

    Sri Chaitanya Junior College

  • SSC

    Nirmala High School

Certifications

  • Databricks certified data engineer professional

  • Databricks certified data engineer associate

  • Databricks certified spark developer

  • Microsoft certified azure data engineer associate