profile-pic

Ankit Kumar

I am an innovative and results-driven Data Engineer with a passion for designing and implementing robust data pipelines. With a strong background in real-time data processing, Kafka, PySpark, and Azure Databricks, I excel at integrating and analyzing massive volumes of semistructured data to derive valuable insights.

Key highlights of my career include:

Designed and implemented a cutting-edge real-time data pipeline that seamlessly integrated over 150 million raw records from more than 30 data sources. By utilizing Kafka and PySpark on Azure Databricks, I ensured efficient and reliable data processing. Leveraged Spark in Python to distribute data processing across large streaming datasets, resulting in a remarkable 67% improvement in ingestion and speed. This optimization enhanced overall system performance and accelerated data-driven decision-making. Created Airflow Dags to automate the triggering of Databricks notebooks based on scheduled intervals. This streamlined workflow automation not only saved time but also improved the efficiency of the data processing tasks.

  • Role

    Senior Python Engineer

  • Years of Experience

    3 years

Skillsets

  • Zookeeper
  • Probability and statistics
  • operating system
  • Object Oriented Programming
  • Database management system
  • Azure
  • AWS
  • Airflow
  • Software Engineering
  • Snowflake
  • Data Structures
  • computer networks
  • Azure DataBricks
  • Algorithms
  • MySQL
  • Kafka-connect
  • Kafka-streams
  • ETL
  • Postman
  • Data Warehousing
  • Docker
  • Apache Kafka
  • PySpark
  • Git
  • Apache Spark
  • SQL
  • Python
  • Java

Professional Summary

3Years
  • Jun, 2022 - Present3 yr 8 months

    Senior Software Engineer

    Optum Global Solutions
  • Dec, 2021 - Apr, 2022 4 months

    Python Developer

    IoTech Designs Pvt. Ltd.

Applications & Tools Known

  • icon-tool

    Python

  • icon-tool

    Java

  • icon-tool

    SQL

  • icon-tool

    MySQL

  • icon-tool

    Docker

  • icon-tool

    Postman

  • icon-tool

    Zookeeper

  • icon-tool

    Git

  • icon-tool

    Azure Databricks

  • icon-tool

    Snowflake

  • icon-tool

    Django

Work History

3Years

Senior Software Engineer

Optum Global Solutions
Jun, 2022 - Present3 yr 8 months
    Efficient transformation and filtration of data using Azure Databricks, Snowflake, and Kafka. Designed Airflow DAGs to streamline workflows and automate Databricks notebooks. Developed a Data Warehouse solution for multiple teams and created real-time streaming data applications.

Python Developer

IoTech Designs Pvt. Ltd.
Dec, 2021 - Apr, 2022 4 months
    Contributed to developing an ML model for vending machines and integrated Razorpay API into Django framework for payment systems.

Achievements

  • Solved 700+ problems on GeeksforGeeks/InterviewBit/Leetcode
  • Secured 304 ranks (AIR) in NIMCET

Major Projects

3Projects

ML Model Train for Vending Machines

    Developed an object recognition model to improve vending machine functionality and identify objects accurately.

Data Streaming and Analysis

    Streamed data from multiple sources, transformed it using Snowflake and Azure Databricks, and produced it back to Kafka topics.

Enterprise Data Warehouse for Delhivery

    Designed and executed a Data Warehouse solution to manage and analyze data from various teams using Snowflake and Databricks.

Education

  • Master of Computer Applications

    Maulana Azad National Institute of Technology (2022)

  • B.Sc (Computer Science)

    Mahatma Jyotiba Phule Rohilkhand University (2018)

Certifications

  • Databricks accredited lakehouse fundamentals

  • Az-900 microsoft azure fundamentals

  • Data structures in python

  • Core java programming