Rohith R Nidhiry
A dynamic professional with over 15 years of IT experience in banking and finance domain and 9+ years into Data Engineering space. 9+ years of experience in Data Engineering(Azure/AWS/Databricks/on-premise) and Analytics. Architected end to end data engineering pipelines that performs data ingestion, data quality checks, ELT/ETL.
Experience in handling Agile(Scrum) ,Kanban and Waterfalls projects
- Experience in ITIL framework including incident, problem, knowledge management and preventive maintenance
- Data Science Enthusiast
- Strong interpersonal and communication skills with an ability to lead a team and keep them motivated
- Steady performer, problem solver with deep knowledge in the software development process
- AWS/Azure, Databricks, Spark, Pyspark/Python, Delta Lake, Hive, Shell Scripting, Tableau
- Certifications : Databricks Certified Data Engineer, AWS Certified Cloud Practitioner, Azure Fundamentals, Databricks Lakehouse Fundamentals
- Role
Sr. Data Engineering & Hive developer
- Years of Experience
15 years
Skillsets
- Scala
- JobTrac
- Linux
- NDM
- Oracle
- Pig
- PySpark
- R
- REXX
- JCL
- Shell Scripting
- Snowflake
- Spark
- SQL
- Tableau
- UC4
- Visual Basic
- VSAM
- Databricks
- ADLS
- AutoSys
- AWS
- Azure Data Factory
- CICS
- COBOL
- Control-M
- Data Architecture
- Abend-AID
- DB2
- Delta Lake
- Git
- Hadoop
- Hive
- IBM Mainframe
- IMS
Professional Summary
15Years
- Jul, 2023 - Present2 yr 5 months
Senior Manager - Data Engineering
State Street Global Advisors - Oct, 2021 - Jul, 20231 yr 9 months
Senior Lead Data Engineer
Ness Digital Engineering - Jul, 2020 - Oct, 20211 yr 3 months
Lead Data Engineer
Tata Consultancy Services - AmEx - Feb, 2012 - Oct, 20164 yr 8 months
Technical Lead
Tata Consultancy Services - AmEx - Nov, 2016 - Jul, 20181 yr 8 months
Technical Lead
Tata Consultancy Services - ABN Amro - Aug, 2018 - Jun, 20201 yr 10 months
Data Engineer
Tata Consultancy Services - PayPal - Dec, 2010 - Jan, 20121 yr 1 month
Developer
Tata Consultancy Services - AmEx
Work History
15Years
Senior Manager - Data Engineering
State Street Global AdvisorsJul, 2023 - Present2 yr 5 months
Architected and developed scalable data solutions and data quality frameworks that ensure high-quality data is available for various investment research models. Worked closely with researchers and portfolio managers within ISG (Investment Solutions Group - multi-asset) and Fixed Income research area. Conducted regular data quality assessments, identifying and addressing data inaccuracies, thereby improving data reliability by 40%. Spearheaded the conversion of legacy R codebase to Python/PySpark, leveraging AI. Supervised and mentored a team of 3 data engineers, fostering a culture of continuous learning and technical innovation.
Senior Lead Data Engineer
Ness Digital EngineeringOct, 2021 - Jul, 20231 yr 9 months
Architected and developed scalable data engineering pipelines for a Canadian retail client. Developed scalable data pipelines using Azure Data Factory and Databricks. Data from various sources such as Hadoop, Netezza, Snowflake, Oracle in various formats were ingested into Data lake. Data processing layer in Databricks with Spark as the processing engine. Led and mentored a team of 2 data engineers.
Lead Data Engineer
Tata Consultancy Services - AmExJul, 2020 - Oct, 20211 yr 3 months
Worked closely with the clients in the US to gather requirements. Identified KPIs of the various sales teams such as dials & talk time, HICs, opportunities created/closed, charge volume growth. Designed and developed data pipelines that leverage Hadoop, Hive, and PySpark for data cleansing, processing, and transformation. Created data visualizations of KPIs using Tableau dashboards. Facilitated various scrum ceremonies.
Data Engineer
Tata Consultancy Services - PayPalAug, 2018 - Jun, 20201 yr 10 months
Engineered data pipelines that ingests data from centralised application logs, perform data cleansing, data quality checks, and required transformations. Handled massive datasets of the order of Terabytes using on-premise Spark 2 cluster. Evaluated and reported tag coverage across various enterprise applications by comparing the coverages between new and old logging systems. Published tag coverage statistics to Tableau dashboard.
Technical Lead
Tata Consultancy Services - ABN AmroNov, 2016 - Jul, 20181 yr 8 months
Developed a new interest calculation system that includes negative interest capability. Incorporated rewarding increase (bonus) functionalities. Developed an auto regression test suite that helps automate regression testing and reduces manual efforts by 75%. Worked on a project to send critical interest calculation result files to Data Lake maintained on Hadoop HDFS via NDM, loading the data to Hive tables.
Technical Lead
Tata Consultancy Services - AmExFeb, 2012 - Oct, 20164 yr 8 months
Team Lead of Accounts Receivable and Accounts Payable systems at American Express bank for Latin American markets for 3 years from offshore and 1 year from onsite UK. Supported the critical functionalities of AR/AP systems for Latin American markets at Amex. Worked on a proof of concept for Amex to migrate large datasets from Mainframe to Hadoop cluster. Re-engineered and fine-tuned code activities to meet client targets, focusing on-demand management and cost savings. Developed numerous tools and automation solutions to reduce redundant manual tasks. Implemented various process improvements. Carried out bi-annual disaster recovery exercises.
Developer
Tata Consultancy Services - AmExDec, 2010 - Jan, 20121 yr 1 month
Incident, problem, change, and knowledge management. Handled critical production issues. Identified issues, provided temporary workarounds, performed root cause analysis, and provided permanent resolutions based on ITIL processes. Deployed numerous process improvements. Developed tools and automation to reduce redundant manual tasks.
Education
B.Tech in Computer Science & Engineering
Rajagiri School of Engineering & Tech (2010)Higher Secondary
CBSE (2006)10th Board
ICSE (2004)