Varun Taneja

Led the end-to-end migration of all PostgreSQL databases from version 13 to 17, applying advanced performance tuning and automation scripts in Python and Bash, ensuring minimal downtime and zero product disruption. Decoupled the sourcing module from the main application and rearchitected and rebuilt it as an independent microservice using LLD and HLD concepts, which enabled moving this to a more cost-efficient AWS region and reducing overall infrastructure expenses by ~10%. Designed the architecture (LLD, HLD) and implemented partitioning on large database tables, significantly improving backend performance and enabling the product to scale more efficiently. Involved working on Python, Postgresql, Django, and FastAPI. Deployed and managed multiple FastAPI applications using Uvicorn and Nginx on AWS, integrating Redis and PostgreSQL backends and ensuring high availability, secure configurations, and scalable API delivery. Developed the Sourcing Manager feature from scratch, enabling end users to efficiently view, update, and add sources for tracking using Python and FastAPI. Automated sprint deployments to production using Ansible, Bash, and Git-based workflows, streamlining release cycles under tight deadlines and improving overall CI/CD reliability. Administered and maintained Ubuntu and Fedora environments on AWS EC2 and EKS, managing Python, Django, and FastAPI services while optimizing aarch64-based infrastructure for performance and cost efficiency. Managed PostgreSQL 17 across production and UAT environments, including routine backups, query optimization, Redis caching integration, and storage performance improvements.

Developed a high-performance bulk-update feature using Python and Django, enabling thousands of stories to be updated in one go, with Redis used for real-time status tracking. This made the update process 10x faster and significantly reduced manual effort. Enhanced the platform UI by integrating the Clearbit company logo service, backed by FastAPIs and Django views running on AWS infrastructure for faster, more reliable data delivery. Built a backend cleanup process in Python and Django to automatically delete redundant stories from both PostgreSQL and Elasticsearch, driven by dynamic client-specific configurations. This reduced Story DB size by 20% and made Newsfeed loads noticeably faster. Re-architected core sourcing tables (RSSFeed and SearchKeyword) using Django ORM and optimized PostgreSQL indexing strategies, improving system scalability and overall query performance. Collaborated cross-functionally with QA, Product, and Engineering teams, improving SDLC efficiency through Git-driven workflows, Ansible automation. Contributed to the hiring and onboarding pipeline, helping identify strong Python/Django engineers and training them on internal systems involving PostgreSQL, Redis, Elasticsearch, and Solr.

Enhanced data accuracy by 20% by implementing robust scraping rules in Python and Django, while resolving technical issues in the sourcing component running on Linux-based environments. Designed and developed RSS Integration System (RIS) using Django and FastAPIs, streamlining task assignment and real-time progress tracking, which improved team collaboration and increased productivity by 30%. Developed and led a large-scale data extraction project in Python to fetch millions of company and LLP records from the MCA website, ensuring high accuracy and reliability. Managed version control and delivery through Git and executed the workflows efficiently on Linux servers. Built a high-performance LinkedIn scraper using Python, optimizing request handling, parsing logic, and system-level tuning on Linux, resulting in a 50% reduction in data extraction time. Created a Django-based feature to integrate reports from leading consultancy firms like Omdia, ABI Research, and Strategy Analytics, exposing the data through FastAPIs and enriching the sourcing pipeline, which led to a 25% increase in user engagement.