profile-pic

Sunando Bhattacharya

A collaborative engineering professional with substantial experience in designing and executing complex business problems involving large scale data. Overall experienced in building highly scalable, robust and fault tolerant systems.


Tech stack that I have worked in :

Big Data Ecosystem: HDFS, YARN, Map Reduce, Apache Spark 2.2, Hive, Oozie

• SQL Technologies: Teradata, DB2, SQL Server, Oracle 10g

• NoSQl Technologies: MongoDB

• Programming Languages: C, C++, Java, Python, C#

• ETL Tools: SAS Data Integration

• Real Time Data processing: Apache Spark Streaming

• Operating Systems: Redhat, Ubuntu, MS Windows

• Cloud Services Used: EC2 AWS Instances, Azure WebApps, kinesis, S3, EC2, Eventhub

  • Role

    Lead Data Engineer

  • Years of Experience

    8 years

Skillsets

  • Apache Kafka
  • Delta Lakehouse
  • Data Modelling
  • Kinesis
  • Impala
  • Databricks Unity Catalog
  • Data warehouse
  • Databricks
  • Apache Spark
  • Java
  • PySpark
  • Hive
  • Scala
  • Git
  • Jenkins
  • Hadoop
  • AWS

Professional Summary

8Years
  • Jan, 2021 - Present5 yr 9 months

    Lead Data Engineer

    Databricks
  • Jan, 2019 - Jan, 20212 yr

    Analyst Data Engineer

    JP Morgan Chase and Co
  • Jan, 2016 - Jan, 20193 yr

    Application Development Analyst

    Accenture

Applications & Tools Known

  • icon-tool

    Databricks

  • icon-tool

    Apache Spark

  • icon-tool

    Apache Kafka

  • icon-tool

    AWS

  • icon-tool

    JSON

  • icon-tool

    WebAPI

  • icon-tool

    Teradata

  • icon-tool

    Hadoop

  • icon-tool

    Spark SQL

  • icon-tool

    Hive

  • icon-tool

    Impala

  • icon-tool

    Scala

  • icon-tool

    Java

  • icon-tool

    JVM

  • icon-tool

    Git

  • icon-tool

    Jenkins

Work History

8Years

Lead Data Engineer

Databricks
Jan, 2021 - Present5 yr 9 months
    Experienced Databricks spark consultant orchestrating migration of legacy data infrastructure, leading to significant reductions in processing time, increased data availability, and enhanced team efficiency, while also contributing to the development and deployment of innovative customer features and solutions.

Analyst Data Engineer

JP Morgan Chase and Co
Jan, 2019 - Jan, 20212 yr
    Experienced Apache Spark Developer at J.P. Morgan Chase specializing in data analytics for strategic initiatives, orchestrating ELT processes from diverse sources, conducting rigorous code quality assessments, streamlining deployments, contributing to a cutting-edge Data Reconciliation Framework, and managing critical data modules for risk identification and mitigation.

Application Development Analyst

Accenture
Jan, 2016 - Jan, 20193 yr
    Experienced Big Data Engineer and Consultant at Nordea, specializing in migration projects, KYC application development, and customer analytics, adept at enhancing operational efficiency, delivering actionable insights, and pioneering transformative solutions leveraging Teradata, Java, Kafka, and Hadoop technologies for strategic decision-making and customer engagement.

Achievements

  • Orchestrated migration of legacy data infrastructure to Databricks
  • Managed seamless deployment of Databricks customer images
  • Developed custom Kafka DataSourceV2 connector
  • Implemented Kotlin UDF execution on Spark
  • Implemented ingestion framework for diverse data sources
  • Pioneered CDC framework for event-driven architecture
  • Migration projects and KYC application development at Nordea

Major Projects

3Projects

Migration to Databricks

    Successfully migrated clients from legacy Cloudera clusters or EMR to Databricks platform, resulting in a 30% reduction in data processing time and a 40% increase in data availability.

Unity Catalog Adoption

    Promoted adoption of Unity Catalog among clients, enhancing data governance practices.

Custom Kafka DataSourceV2 Connector

    Developed a custom Kafka DataSourceV2 connector for ingesting broken records, improving data integrity.

Education

  • B. Tech

    KIIT University (2016)

  • ISC

    Delhi Public School, Megacity, Kolkata (2012)

  • ICSE

    Delhi Public School, NewTown, Kolkata (2010)

Certifications

  • Databricks certified data engineering associate

  • Databricks certified data engineering professional

  • Databricks certified associate developer for apache spark 3.0

  • Certified Data Engineering Associate Databricks

  • Employee accreditation lakehouse with delta technical enablement