profile-pic

Sanskar Nanegaonkar

AI engineer with 4+ years shipping production LLM and multimodal systems end-to-end. Currently at I'mbesideyou Inc. building agentic AI infrastructure — multi-tenant RAG pipelines for enterprise clients like NTT DATA, multimodal SOP-generation systems deployed across 3+ organizations with 100+ operators, voice AI agents on Vapi and LiveKit handling 1,000+ calls, and LLM evaluation frameworks with automated regression gates.

Previously built and productionized clinical AI: a multimodal depression-severity model (83% accuracy, 350+ patients) that contributed to an FDA Breakthrough Device Designation application, and an LLM-based SOAP note generation system that cut clinician documentation time by 70%.

Areas of expertise: Agentic AI · Multi-agent systems · RAG (Qdrant, FAISS, cross-encoder reranking) · LLM fine-tuning (LoRA/QLoRA, DPO, RLHF, preference optimization) · LLM evaluation & observability (Langfuse, LLM-as-judge, RAGAS) · Prompt engineering · Voice AI (Vapi, LiveKit) · Multimodal ML · Clinical AI · Production MLOps · LangGraph · Pydantic · FastAPI · PyTorch · AWS · Python

Published: "A Practical Approach to Predicting Depression" — Health Informatics Journal (2024)

Published: "Chatbot Interventions for Depression & Anxiety" — European Health Psychology Society (2025)

IIT Hyderabad, B.Tech Engineering Physics (CGPA: 8.48/10). Transferable H1B, immediately available for US relocation. Open to Senior AI Engineer, ML Engineer, and Applied AI roles.

  • Role

    Senior AI Engineer, Agentic AI & Production Systems

  • Years of Experience

    5 years

  • Professional Portfolio

    View here

Skillsets

  • Qdrant
  • LoRA
  • multimodal
  • OpenAI
  • OpenCV
  • PostgreSQL
  • Pydantic
  • pytest
  • PyTorch
  • Librosa
  • QLoRA
  • rag
  • Rlhf
  • Scikit-learn
  • Trl
  • vLLM
  • LLM
  • Docker
  • Python
  • SQL
  • FastAPI
  • Langfuse
  • LangGraph
  • MCP
  • Claude
  • Deepface
  • AWS
  • DPO
  • Evals
  • FAISS
  • Gemini
  • GPT
  • HuggingFace
  • Kubernetes

Professional Summary

5Years
  • Jan, 2025 - Present1 yr 6 months

    Senior AI Engineer, Agentic AI & Production Systems

    Imbesideyou
  • Aug, 2022 - Dec, 20242 yr 4 months

    Data Scientist, Clinical AI & Multimodal ML

    Imbesideyou
  • May, 2021 - Jul, 20221 yr 2 months

    Data Science Intern

    Imbesideyou

Work History

5Years

Senior AI Engineer, Agentic AI & Production Systems

Imbesideyou
Jan, 2025 - Present1 yr 6 months
    Shipped a multi-tenant HR-policy RAG assistant for NTT DATA and two enterprise customers, indexing 1,000+ HR policies: FastAPI service with Qdrant per-tenant retrieval, cross-encoder reranking, OpenAI GPT synthesis, safety-aware policy-conflict detection, source-priority rules, and Langfuse audit traces. Owned multimodal SOP-generation and gap-analysis modules that convert desktop activity into operating procedures: multi-provider LLM pipeline (Gemini vision, GPT synthesis, Claude critic), LangGraph agentic multi-stage orchestration, Pydantic structured outputs, and write-review-rewrite loops; deployed across 3+ organizations with 100+ operators. Reduced manual transcript review per agent from a week to a few automated hours using a multi-agent RAG optimization system: planner / extractor / synthesizer chains, RAG over source corpora, human-feedback interpreter, variant regeneration router, and regression scoring against correctness thresholds. Shipped voice AI agents on Vapi and LiveKit for lead qualification, interviewing, and structured data collection; LLM dialogue policy with Pydantic tool calls and transcript-grounded evals matched human-agent pickup and conversion rates across 1,000+ calls. Replaced random spot-check QSC audits with continuous automated VLM compliance coverage, piloted live across 2 restaurant locations; designed a VLM-assisted auto-labeling pipeline that bootstrapped a proprietary training dataset in a kitchen domain with no open-source data or model coverage.

Data Scientist, Clinical AI & Multimodal ML

Imbesideyou
Aug, 2022 - Dec, 20242 yr 4 months
    Built a multimodal depression-severity model using DeepFace facial emotions and librosa speech-prosody features (early fusion); reached 83% accuracy and extended to bipolar-vs-unipolar classification at 78% across 350+ patients. Productionized the clinical ML pipeline on AWS; collaborated with Keio on an FDA Breakthrough Device Designation application and Pre-Sub filings, and validated depression-severity predictions with Northwell Health and Hamamatsu across clinical cohorts. Cut therapy documentation time by 70% with an LLM system converting transcripts into Pydantic-validated SOAP notes; a clinician style-adaptation loop distilled edits into reusable per-clinician profiles.

Data Science Intern

Imbesideyou
May, 2021 - Jul, 20221 yr 2 months
    Built video analytics pipelines across classroom sessions, live-streaming, and sales call recordings; extracted behavioral and linguistic signals (Python, OpenCV, scikit-learn) to distinguish high from low performers across 10,000+ recordings and generated structured coaching recommendations.

Major Projects

2Projects

Preference Optimization & Alignment Stack

    Fine-tuned Llama 3 8B for medical Q&A (MedQA / PubMedQA): curated instruction and preference data, ran LoRA / QLoRA domain SFT followed by DPO preference optimization, and evaluated against the base model and a frontier API on task success, helpfulness, latency, and cost before quantized vLLM serving.

LLM Evaluation Harness

    Built a pytest-style evaluation harness for LLM-powered services: golden-set fixtures with versioned prompts, LLM-as-judge scoring with confidence-aware aggregation, per-test cost and latency budgets, and a GitHub Actions gate that blocks prompt or model changes regressing curated benchmarks. Plugs into Langfuse traces for failure inspection.

Education

  • B.Tech, Engineering Physics

    Indian Institute of Technology Hyderabad (2022)