The Language AI Services Team in Amazon Web Services (AWS) is seeking a Language Data Scientist to join our data team focused on Health AI. We are looking for candidates who have a strong interest in applying their linguistic and data science expertise in collecting and analyzing natural language and human data to lead multiple data collection and data analysis efforts at a time in an exciting, fast-paced environment. In this role, you will curate, engineer, and analyze natural language datasets in the medical domain, as well as design experiments and surveys to elicit human-in-the-loop insights, which are critical for developing AI-powered language applications. You will partner closely with talented language engineers, program managers, clinical experts, applied scientists, engineers, and product managers to deliver data solutions that meet customer needs. Key job responsibilities - Translate business, modeling and ethical requirements in Health AI into executable data collection projects - Design human-in-the-loop evaluation tasks to measure the performance and usability of models in the medical domain - Develop the materials necessary to execute successful data collection efforts such as guidelines, annotation interfaces, quality assurance workflows - Support the sourcing and/or creation of high-quality language datasets and language artifacts for feature and language expansion - Analyze structured and unstructured data to provide actionable recommendations to improve data quality or model performance - Iterate and innovate on data collection methodologies to improve data turnaround time and reliability - Incorporate LLMs, prompt engineering, and ML techniques to automate repetitive annotation and data creation workflows - Stay up to date with developments in the world of AI, with a focus on model fine tuning and evaluation data needs and techniques A day in the life As a Language Data Scientist on the data team, you will take the lead on a couple of critical data projects related to AWS HealthScribe and Amazon Transcribe Medical, diving deep and developing materials to drive these projects forward. You will consult with stakeholders to understand the role data plays in developing and launching specific language services that meet customer needs. You will propose data collection and annotation strategy to curate and validate datasets, and partner with stakeholders to set the quality bar for these data projects. You will lead iterative data analysis and course correction efforts to address gaps and quality issues. You will also design experiments to elicit human-in-the-loop insights (including users who are domain experts) for model evaluation and usability testing, proposing the optimal business and evaluation metrics to use. You will gradually expand your scope by applying the principles of data-centric AI to conduct experiments to drive workflow and process improvements, with the goal to optimize the cost and quality of data. Leveraging your hands-on data analytics skills and up-to-date knowledge of machine learning (ML) and Generative AI techniques, you will collaborate with other language engineers and scientists to propose new strategies for sourcing ground truth data or generating reliable metrics, automating where appropriate. About the team The data team within AWS Language AI Services consists of language engineers, program managers, and clinical experts. We are part of a cross-functional team developing cutting-edge Language AI services within AWS. These services, ranging from subtitles and call analytics to clinical documentation and beyond, support numerous languages and enable customers across diverse industries to seamlessly integrate intelligence into their business operations and AI applications. We are customer obsessed and committed to delivering solutions with the highest quality and integrity. Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.