Feedback
Follow Us
Publication

Extended conversion: Capturing successful interactions in voice shopping

By Elad Haramaty, Zohar Karnin, Arnon Lazerson, Liane Lewin-Eytan, Yoelle Maarek
2023
Download Copy BibTeX
Share
Download
Copy BibTeX
Share
Being able to measure the success of online shopping interactions is crucial in order to evaluate and optimize the performance of e-commerce systems. It is especially challenging in the domain of voice shopping, typically supported by voice-based AI assistants. Unlike Web shopping, which offers a rich amount of behavioral signals such as clicks, in voice shopping a non-negligible amount of shopping interactions frequently ends without any immediate explicit or implicit user behavioral signal. Moreover, users may start their journey using a voice-enabled device, but complete it elsewhere, for example on their smartphone mobile app or a Web browser. We explore the challenge of measuring successful interactions in voice product search based on users’ behavior, and propose a medium-term reward metric named Extended ConVersion (ECVR). ECVR extends the notion of conversion beyond the usual purchase action, which serves as an undisputed measure of success in e-commerce. More specifically, it also captures purchase actions that occur at a later stage during a same shopping journey, and possibly on different channel than the one on which the interaction started. In this paper, we formally define the ECVR metric, describe multiple ways of evaluating the quality of a metric, and use these to explore different parameters for ECVR. After selecting the most appropriate parameters, we show that a ranking system optimized for ECVR, set up with these parameters, leads to improvements in long-term engagement and revenue, without compromising immediate conversion gains.
Research areas
Tags

Latest news

Work with us

See more jobs See more jobs
Data Scientist, AWS Generative AI Innovation Center
FR, Courbevoie
Amazon launched the Generative AI Innovation Center (GenAIIC) in June 2023 to help AWS customers accelerate the use of generative AI to solve business and operational problems and promote innovation in their organization. This is a team of strategists, data scientists, engineers, and solution architects working step-by-step with customers to build bespoke solutions that harness the power of generative AI.(https://press.aboutamazon.com/2023/6/aws-announces- generative-ai-innovation-center). We’re looking for Data Scientists capable of using generative AI and other techniques to design, evangelize, and implement state-of-the-art solutions for never-before-solved problems. You will work directly with customers and innovate in a fast-paced organization that contributes to game-changing projects and technologies. You will design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience. AWS Sales, Marketing, and Global Services (SMGS) is responsible for driving revenue, adoption, and growth from the largest and fastest growing small- and mid-market accounts to enterprise-level customers including public sector. The AWS Global Support team interacts with leading companies and believes that world-class support is critical to customer success. AWS Support also partners with a global list of customers that are building mission-critical applications on top of AWS services. Key job responsibilities As a Data Scientist, you will - Collaborate with AI/ML scientists, engineers, and architects to research, design, develop, and evaluate cutting-edge generative AI algorithms to address real-world challenges - Interact with customers directly to understand the business problem, help and aid them in implementation of generative AI solutions, deliver briefing and deep dive sessions to customers and guide customer on adoption patterns and paths to production - Create and deliver best practice recommendations, tutorials, blog posts, sample code, and presentations adapted to technical, business, and executive stakeholder - Provide customer and market feedback to Product and Engineering teams to help define product direction About the team The team helps customers imagine and scope the use cases that will create the greatest value for their businesses, select and train or fine tune the right models, define paths to navigate technical or business challenges, develop proof-of-concepts, and make plans for launching solutions at scale. The Generative AI Innovation Center team provides guidance on best practices for applying generative AI responsibly and cost efficiently. Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
Senior Applied Scientist - Multilingual LLM, Alexa International Tech
US, WA, Bellevue
Alexa International Tech (AIT) team is looking for a passionate, talented, and inventive Applied Scientist to help build industry-leading technology with Large Language Models (LLMs) and multimodal systems, requiring strong deep learning and generative models knowledge. Key job responsibilities As an Applied Scientist with the AIT team, you will work with talented peers to develop novel algorithms and modeling techniques to advance the state of the art with LLMs. Your work will directly impact our international customers in the form of products and services that make use of digital assistance technology. You will leverage Amazon’s heterogeneous data sources, unique but diverse international customer nuances and large-scale computing resources to accelerate advances in voice domain in multi-modal setup. The ideal candidate possesses a solid understanding of machine learning fundamentals and a passion for pushing boundaries in this vast and quickly evolving field. They thrive in fast-paced environments to will tackle complex challenges, and excel at swiftly delivering impactful solutions while iterating based on user feedback. A day in the life · Analyze, understand, and model customer behavior and the customer experience based on large scale data. Especially showing passion towards solving for international customer-centric challenges. · Build novel online & offline evaluation metrics an methodologies for personal digital assistants and customer scenarios, on multi-modal devices. · innovate and deliver deep learning based innovation across life-cycle such as policy-based learning, international customer specific model performance tuning. · Quickly experiment and setup experimentation framework for agile model and data analysis or A/B testing · Contribute through industry first research to drive the innovation forward.
Applied Scientist, International Machine Learning
IN, KA, Bengaluru
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to create state-of-the-art solutions for providing better value to Amazon’s customers? Do you want to build and deploy advanced ML systems that help optimize millions of transactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data to solve real-world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you like to innovate and simplify? If yes, then you may be a great fit to join the Machine Learning team for India Consumer Businesses. Machine Learning, Big Data and related quantitative sciences have been strategic to Amazon from the early years. Amazon has been a pioneer in areas such as recommendation engines, ecommerce fraud detection and large-scale optimization of fulfillment center operations. As Amazon has rapidly grown and diversified, the opportunity for applying machine learning has exploded. We have a very broad collection of practical problems where machine learning systems can dramatically improve the customer experience, reduce cost, and drive speed and automation. These include product bundle recommendations for millions of products, safeguarding financial transactions across by building the risk models, improving catalog quality via extracting product attribute values from structured/unstructured data for millions of products, enhancing address quality by powering customer suggestions We are developing state-of-the-art machine learning solutions to accelerate the Amazon India growth story. Amazon India is an exciting place to be at for a machine learning practitioner. We have the eagerness of a fresh startup to absorb machine learning solutions, and the scale of a mature firm to help support their development at the same time. As part of the India Machine Learning team, you will get to work alongside brilliant minds motivated to solve real-world machine learning problems that make a difference to millions of our customers. We encourage thought leadership and blue ocean thinking in ML. Key job responsibilities Use machine learning and analytical techniques to create scalable solutions for business problems Analyze and extract relevant information from large amounts of Amazon’s historical business data to help automate and optimize key processes Design, develop, evaluate and deploy, innovative and highly scalable ML models Work closely with software engineering teams to drive real-time model implementations Work closely with business partners to identify problems and propose machine learning solutions Establish scalable, efficient, automated processes for large scale data analyses, model development, model validation and model maintenance Work proactively with engineering teams and product managers to evangelize new algorithms and drive the implementation of large-scale complex ML models in production Leading projects and mentoring other scientists, engineers in the use of ML techniques About the team International Machine Learning Team is responsible for building novel ML solutions that attack India first (and other Emerging Markets across MENA and LatAm) problems and impact the bottom-line and top-line of India business. Learn more about our team from https://www.amazon.science/working-at-amazon/how-rajeev-rastogis-machine-learning-team-in-india-develops-innovations-for-customers-worldwide
Applied Scientist III, Alexa & Fire TV Security
US, WA, Bellevue
The Devices and Services Security team is looking for a passionate, talented, and inventive Senior Applied Scientist with a strong deep learning background, to secure the development of industry-leading Generative AI systems. As a Senior Applied Scientist with the Devices & Services Security team, you will lead the development of novel algorithms and modeling techniques to advance the state of the art with Generative AI systems. Your work will directly impact our customers in the form of products and services that make use of vision and language technology. You will leverage Amazon’s heterogeneous data sources and large-scale computing resources to accelerate development of security solutions with multimodal Large Language Models (LLMs) and Generative Artificial Intelligence (Gen AI). The Devices and Services (D&S) Security team works to ensure that Amazon devices and services are designed and implemented to the high standards required to maintain and enhance customer trust. The team develops security technologies for builder teams, performs penetration testing, and handles and tracks incident responses to resolution. The team is responsible for defining and executing on the security and privacy requirements for the entire organization. A day in the life Diverse Experiences Amazon Security values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why Amazon Security? At Amazon, security is central to maintaining customer trust and delivering delightful customer experiences. Our organization is responsible for creating and maintaining a high bar for security across all of Amazon’s products and services. We offer talented security professionals the chance to accelerate their careers with opportunities to build experience in a wide variety of areas including cloud, devices, retail, entertainment, healthcare, operations, and physical stores. Inclusive Team Culture In Amazon Security, it’s in our nature to learn and be curious. Ongoing DEI events and learning experiences inspire us to continue learning and to embrace our uniqueness. Addressing the toughest security challenges requires that we seek out and celebrate a diversity of ideas, perspectives, and voices. Training & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, training, and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve.
Applied Scientist, AI Security
US, CA, Santa Clara
We are seeking an Applied Scientist to join our AI Security team, which builds security tooling and paved path solutions to ensure Generative AI (GenAI) based experiences developed by Amazon uphold our high security standards, and uses AI to develop foundational services that make security mechanisms more effective and efficient. As an Applied Scientist, you’ll be responsible for designing and implementing state-of-the-art solutions, to build an AI-based foundational service for securing products and services at Amazon scale. You will collaborate with applied scientists and software engineers to develop innovative technologies to solve some of our hardest security problems, and AI-based security solutions that support builder teams across Amazon throughout their software development journey, enabling Amazon businesses to strengthen their security posture more efficiently and effectively. Key job responsibilities • design and implement accurate and scalable methods to solve our hardest AI security problems • Lead and partner with applied scientists and software development engineers to drive technical design and implementation for a foundational GenAI-based security service About the team The mission of the AI Security organization is to ensure Generative AI experiences delivered by Amazon to our customers uphold our high security standards and to harness AI to strengthen Amazon’s security posture more efficiently and effectively. A day in the life A day in the life involves meeting Vulnerability Management and Incident Responder teams to review data flows, prediction use cases, and automation gaps. From here you will research data sets, working with security/software engineers to retrieve data needed for your analysis and explorations. Once you have framed the problems, you will conduct experiments, regressions, and various analysis activities to find insights. You will develop and train models that will then be placed into a production environment with the help of software engineers. You will then work with your security team partners to understand the effectiveness of the models created. About the team The Defensive Security team is small, tight-knit, and located in Austin, Texas. It is primarily software engineers, but will be developed into a hybrid team of software engineers and security engineers. This team will have tenured Amazonian leadership, with a track record of mentoring, coaching, and career progression support. About Amazon Security Diverse Experiences — Amazon Security values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why Amazon Security? — At Amazon, security is central to maintaining customer trust and delivering delightful customer experiences. Our organization is responsible for creating and maintaining a high bar for security across all of Amazon’s products and services. We offer talented security professionals the chance to accelerate their careers with opportunities to build experience in a wide variety of areas including cloud, devices, retail, entertainment, healthcare, operations, and physical stores. Inclusive Team Culture — In Amazon Security, it’s in our nature to learn and be curious. Ongoing DEI events and learning experiences inspire us to continue learning and to embrace our uniqueness. Addressing the toughest security challenges requires that we seek out and celebrate a diversity of ideas, perspectives, and voices. Training & Career Growth — We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, training, and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance — We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve.
Applied Scientist - Computer Vision/ Multi-Modal Machine Learning, Last Mile Geospatial, LMDT- Geospatial
US, WA, Bellevue
Amazon’s maps play a crucial role in our vehicle navigation, routing, and planning problems to ensure fast and safe deliveries to our customers. As part of the Last Mile Geospatial Science organization, you’ll partner closely with other scientists and engineers in a collegial environment with a clear path to business impact. We have an exciting problem area to augment the maps and routing inputs from satellite/aerial imagery and street videos by leveraging the latest computer vision and deep learning techniques. Key job responsibilities Successful candidates should have a deep knowledge (both theoretical and practical) of various machine learning algorithms for large scale computer vision problems, the ability to map models into production-worthy code, the communication skills necessary to explain complex technical approaches to a variety of stakeholders and customers, and the excitement to take iterative approaches to tackle big, long term problems. The applied scientist should be proficient with image and video analysis using machine learning, including designing architecture from scratch, modify existing loss functions, full model training, fine-tuning, and evaluating the latest deep learning models. The Applied Scientist optimizes different models for specific platforms, including edge devices with restricted resources. Multi-modal models, e.g., Large Vision Language Models (LVLM), zero-shot, few-shot, and semi-supervised learning paradigms are used extensively. A day in the life If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you! At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply! Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include: 1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options 3. Paid Time Off (PTO) 4. 401(k) Plan
Data Scientist, GSSO
US, WA, Seattle
We are seeking a talented and analytical Data Scientist to join our team and drive data-driven insights and solutions. In this role, you will be responsible for performing exploratory data analysis, developing and deploying predictive models, and leveraging advanced analytics techniques to uncover valuable insights and support data-driven decision-making across the organization. Key job responsibilities • Collaborate with our applied and data scientists to build robust and scalable Generative AI solutions for business problems • Effectively use Foundation Models available on Amazon Bedrock and Amazon SageMaker to meet our customer's performance needs • Work hands on to build scalable cloud environment for our customers to label data, build, train, tune and deploy their models • Interact with customer directly to understand the business problem, help and aid them in implementation of their ML ecosystem • Analyze and extract relevant information from large amounts of historical data to help automate and optimize key processes • Work closely with partner teams to drive model implementations and new algorithms About the team Amazon Web Services (AWS) provides a scalable cloud computing platform to companies globally. AWS Global Services (GS), formed in 2022, delivers customer success throughout the cloud adoption lifecycle. Our 25K+ employees and integrated offerings enable us to combine technology and human expertise to maximize and accelerate customer outcomes. GS is comprised of four primary business units: 1) Global Services Security (GSS) provides security guidance and offerings, 2) Training & Certification (T&C) offers cloud skills training and certification, 3) Professional Services (ProServe) provides consulting and hands-on-keyboard services, and 4) Support and AWS Managed Services (Support) delivers 24/7 technical support and managed services. Together, these teams continuously improve our systems and processes to enable better results for both customers and employees, with the GS Strategy & Operations (GSS) teams supporting each. GSSO enables integrated business support, product management, planning, and deal strategy for GS. GSSO understands customer experiences and inspires bold ideas to deliver the best experiences and solutions to our customers. We embrace scientific thinking, pursue continuous improvement, and develop talent to provide world-class support across GS. About AWS Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Hybrid Work We value innovation and recognize this sometimes requires uninterrupted time to focus on a build. We also value in-person collaboration and time spent face-to-face. Our team affords employees options to work in the office every day or in a flexible, hybrid work model near one of our US Amazon offices.
Applied Scientist, Stores Security
RO, Iasi
In Stores Security, we develop authorization solutions using attribute-based access control (ABAC) to enable Amazon to enforce fine-grained access control (FGAC) on sensitive data. We create logging solutions that generate access events from thousands of services for evidence collection. Additionally, we implement rule-based and machine learning systems to monitor access to customer data, detect anomalous activities, and demonstrate compliance. We are seeking a talented, self-directed Applied Scientist to work on the cutting edge security technologies. You'll design and run experiments, research new algorithms, and find new ways of protecting Amazon's customer trust. Besides theoretical analysis and innovation, you will work closely with talented engineers to put your algorithms and models into practice. You should thrive in ambiguous environments that require to find solutions to problems that have not been solved before. You enjoy and succeed in fast paced environments where learning new concepts quickly is a must. You leverage your exceptional technical expertise, a sound understanding of the fundamentals of Computer Science, and practical experience in building large-scale distributed systems. Your strong communication skills enable you to work effectively with both business and technical partners. Key job responsibilities - Process and analyze large data sets using as many techniques as necessary - Deliver scalable models that can analyze large data sets efficiently - Build mathematical models to detect and classify specific data elements with high accuracy - Prototype these models by using high-level modeling languages such as Python. A software team will be working with you to transform prototypes into production. - Create, enhance, and maintain technical documentation, and present to other scientists and business leaders.
Research Scientist, Enterprise Engineering
US, WA, Seattle
Description How often have you had an opportunity to be an early member of a team that is tasked with solving a huge customer need through disruptive, innovative technology, reinventing an industry? Are you passionate about building cutting-edge AI solutions that leverage the power of Generative AI? Do you want to be at the forefront of transforming how Amazonians work and collaborate? If so, we want to hear from you! Our mission is to revolutionize the way Amazonians work and collaborate by transforming productivity through the power of advanced generative AI technologies. As a Research Scientist, you will play a critical role in driving the development of Generative AI solutions, in particular those based on large language models, information retrieval, AI Agents and knowledge graphs with goals being tailored to Amazonians needs. You will handle Amazon-scale use cases with significant impact. Key job responsibilities - Use deep learning, ML and NLP techniques to create scalable solutions for creation and development of language model centric solutions for building personalized agent systems based on a rich set of structured and unstructured contextual signals - Innovate new methods for knowledge extraction and information retrieval, using language models in combination with other learning techniques, that allows effective grounding in context providers when considering memory, compute, latency and quality - Collaborate with cross-functional teams of engineers, product managers, and scientists to identify and solve complex problems in knowledge aggregation, processing, modeling, and verification - Design and execute experiments to evaluate the performance of state-of-the-art algorithms and models, and iterate quickly to improve results - Think Big about the arc of development of Agents in Amazon over a multi-year horizon, and identify new opportunities to apply these technologies to solve real-world problems - Communicate results and insights to both technical and non-technical audiences, including through presentations and written reports - Mentor and guide junior scientists and engineers, and contribute to the overall growth and development of the team Basic Qualifications - 5+ years of building machine learning models for business application experience - PhD, or Master's degree and 3+ years of applied research experience - Experience programming in Java, C++, Python or related language - Experience with neural deep learning methods and machine learning Preferred Qualifications - Experience using spark and distributed computing, for big data engineering. - Experience with popular deep learning frameworks such as MxNet and Tensor Flow. - Experience with large scale machine learning systems such as profiling and debugging and understanding of system performance and scalability - Proficient in 2 of these areas: large language models, information retrieval and knowledge graphs
Applied Scientist, Sponsored Products, Amazon Ads
US, NY, New York
Amazon continues to invest heavily in building our world class advertising business. Our products are strategically important to our Retail and Marketplace businesses, driving long term growth. We deliver billions of ad impressions and millions of clicks daily, breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and strong bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.The Sponsored Products OSSR team is responsible for all non-search supply and associated experiences. As an Applied Scientist on our team, you will be responsible for defining the science and technical strategy for one of our most impactful strategic initiatives, creating lasting value for Amazon and our advertising customers. Key job responsibilities • Support business, science and engineering strategy and roadmap for Sponsored Products OSSR projects • Drive alignment across organizations for science, engineering and product strategy to achieve business goals • Lead/guide scientists and engineers across teams to develop, test, launch and improve of science models designed to optimize the shopper experience and deliver long term value for Amazon and advertisers • Develop state of the art experimental approaches and ML models. About the team Sponsored Products (SP) is Amazon's largest and fastest growing business. Over the last few years we grown to a multi-billion dollar business. SP ads are shown prominently throughout search and detail pages, allowing shoppers to seamlessly discover products sold on Amazon. Ad experience and market place is one of the highest impact decisions we make. This role has unparalleled opportunity to grow our marketplace and deliver value for advertisers and shoppers.
View from space of a connected network around planet Earth representing the Internet of Things.
Have feedback?
Let us know by taking this short survey
Get started

Amazon.com | Conditions of Use | Privacy | © 1996-2024 Amazon.com, Inc. or its affiliates

Follow Us