What’s the organizational model within Alexa Shopping?
Broadly speaking, the Alexa Shopping science organization includes two types of roles: applied scientists and research scientists. Our organization is managed independently from engineering and product. Our organizational model ensures that we remain focused on contribution to our customers and to the business. It also ensures that we maintain ownership over decision-making processes and are positioned to raise the bar for scientific innovation.
At Amazon, we develop products and processes around a core group of tenets. We had a robust discussion to arrive at a group of tenets as we developed our organizational model for Alexa Shopping. Some of our tenets include:
- Consider both customer/business needs and short/long term scientific challenges when defining and prioritizing our goals;
- Collaborate with product and engineering to make sure that we bring scientific value to each effort that requires advancing the research state of the art; and
- Validate that we are raising the science bar and innovating on behalf of our customers, by sharing our results with both the internal Amazon scientific community and the external research community.
How do you set goals to ensure that scientists can deliver a measurable business impact?
Our goals are set on a yearly basis, and updated on a quarterly basis if needed. Our “roofshot” goals (short-term goals that are attainable within a year time frame) are set in collaboration with engineering and product. Approximately 70%-80% of our resources are allocated to roofshots, which have well defined success metrics.
The remaining resources are devoted to moonshot goals that require scientific exploration. They can typically not justify an engineering investment before first establishing some evidence. Each moonshot effort is targeted towards a long-term contribution to addressing customers’ needs. When a prototype or proof of concept is developed, and its outcome is proven valuable, a discussion can then start with the engineering and product partners to verify how to take the effort to production.
What are some mechanisms to facilitate collaboration between scientists, engineers and product managers?
In each of our roofshot projects there are three main parties that work tightly together: engineering, product and science. As a science team, we step in when no textbook solution exists, and the development of a new state-of-art solution is required.
In the early phases of the project, science conducts an exploration phase during which an offline proof of concept is developed. In this phase, science is usually the sole player and owns the decision-making process. At every step along the way, science also owns decisions related to the algorithms and models we develop and ship
After a successful exploration phase, engineering and product come into play, together with science, in order to turn the new solution developed into a solution working in production. Product teams own decisions about the definition of the feature or experience that is supported by the new solution. These decisions also shape the final customer experience and business logic that will be integrated into production.
Finally, engineering owns all decisions that have to do with the integration and implementation of the new logic.
How do you validate your research results?
Yoelle Maarek, vice president of Alexa Shopping Research and Science, always says that research cannot be self-proclaimed. Publishing our contributions at top conferences is a validation by the research community that we have truly advanced the state of the art on behalf of our customers. A longer-term benefit of publishing is that our articles may inspire external researchers to investigate areas of interest to Amazon, and produce results that will be valuable to the entire ecosystem.
To give just one example, some work we presented at the SIGIR conference last year, attracted the attention of a team of academics in Italy, who will visit our office in February to encourage their PhD students to conduct research around the same topic. More recently, our paper “Why do customers buy seemingly irrelevant products?” was accepted at WSDM, the top conference in web search and data mining. We are hopeful that these types of publications will help us build ties with the larger community to build insights for moonshot goals.
