Computer Vision Engineer
Nosh RoboticsAug, 2024 - Present2 yr 2 months
Leading the AI team in building culinary AI Systems to automate cooking. Ingredient Understanding - Built and fine-tuned ResNet, MobileNet, MobileViT and Swin Transformer models to infer shape, size, variants, and cooking state across many ingredients, with F1 > 0.95 on diverse test data for automated cooking decisions. Ingredient Segmentation - Fine-tuned SegFormer to segment different ingredients in the pan with mIoU of 0.94, enabling volume estimation to drive cooking logic. Universal Consistency Model - Developed a MobileViT model to classify dish consistency progression (liquid-to-dry) across 4 stages for adaptive cook-time and heat controls. Data Engine - Led data collection and human-in-the-loop labeling efforts to build high-quality culinary datasets. Vision Monitoring - Built a MobileViT model to detect camera degradation factors (steam blur, oil splatter, exposure shifts) and early sticking and burn detection, improving downstream model reliability and cooking outcomes. Edge AI Deployment - Built an optimized C++ inference system with dynamic NPU/GPU/CPU dispatch, supporting ONNX and LiteRT runtimes, deployed on Android. Recipe Intelligence - Built an LLM pipeline converting free-form online recipes into structured cooking programs, with modality adaptation (gas to induction, closed to open pot), achieving a 40% A/B preference rate compared to chef-curated machine tuned recipes.