Machine Learning Data Associate II(Prompt Engineer)
Amazon Web ServicesNov, 2024 - Present1 yr 2 months
Designed and implemented Python based pipelines to train, finetune, and evaluate large language models on AWS Bedrock. Evaluated and provided feedback on AI-generated responses, improving retrieval-augmented generation (RAG) workflows. Built Python scripts to automate model evaluation, data processing, and results analysis for large-scale AI workflows. Reviewed AI-generated content for potential biases, safety concerns, and ethical issues, ensuring compliance with Responsible AI (RAI) guidelines. Designed and executed experiments to test multi-modal LLM capabilities with text, code, and image-based inputs. Supported human-in-the-loop (HITL) processes, ensuring continuous improvements in AI learning and refinement. Documented AI system improvements and collaborated with teams for process automation and optimization. Developed custom Python scripts to automate dataset preparation, model training, and deployment workflows using Bedrock APIs. Contributed to multi-modal AI tasks, integrating different data types for improved model performance. Evaluated bot performance and API interactions using the QAction Benchmarking framework. Leveraged expertise in natural language processing (NLP) and machine learning to optimize prompts for multi-modal AI tasks, integrating various data types to improve model outputs. Created reusable Python utilities for API-driven LLM testing and integration with internal tooling. Utilized the Qaction Benchmarking framework to evaluate bot performance and API interactions, informing prompt refinement strategies. Contributed to the development of internal tools and frameworks for prompt testing, version control, and automated optimization. Deployed finetuned models to AWS environments ensuring scalability, low latency, and high accuracy in production. Collaborated with AI teams to refine architectures and deploy high performing models into production environments.