Feedback
Follow Us
ComputerVision.svg
Research Area

Computer vision

Helping devices see and understand our visual world.
Explore more

Recent publications

View All View All
  • Effectively fine-tune to improve large multimodal models for radiology report generation
    Yuzhe Lu, Sungmin Hong, Yash Shah, Panpan Xu
    NeurIPS 2023 Workshop on Deep Generative Models for Health
    2023
    Writing radiology reports from medical images requires a high level of domain expertise. It is time-consuming even for trained radiologists and can be error-prone for inexperienced radiologists. It would be appealing to automate this task by leveraging generative AI, which has shown drastic progress in vision and language understanding. In particular, Large Language Models (LLM) have demonstrated impressive
    Computer vision
  • Minuet: Accelerating 3D sparse convolutions on GPUs
    Jiacheng Yang, Christina Giannoula, Jun Wu, Mostafa Elhoushi, James Gleeson, Gennady Pekhimenko
    EuroSys 2024
    2023
    Sparse Convolution (SC) is widely used for processing 3D point clouds that are inherently sparse. Different from dense convolution, SC preserves the sparsity of the input point cloud by only allowing outputs to specific locations. To efficiently compute SC, prior SC engines first use hash tables to build a kernel map that stores the necessary General Matrix Multiplication (GEMM) operations to be executed
    Computer vision
  • RealFM: A realistic mechanism to incentivize data contribution and device participation
    Marco Bornstein, Amrit Singh Bedi, Anit Kumar Sahu, Furqan Khan, Furong Huang
    NeurIPS 2023 Workshop on Federated Learning in the Age of Foundation Models
    2023
    Edge device participation in federating learning (FL) has been typically studied under the lens of device-server communication (e.g., device dropout) and assumes an undying desire from edge devices to participate in FL. As a result, current FL frameworks are flawed when implemented in real-world settings, with many encountering the free-rider problem. In a step to push FL towards realistic settings, we
    Computer vision
  • Continual learning with low rank adaptation
    Martin Wistuba, Prabhu Teja Sivaprasad, Lukas Balles, Giovanni Zappella
    NeurIPS 2023 Workshop on Distribution Shifts (DistShifts)
    2023
    Recent work using pretrained transformers has shown impressive performance when fine-tuned with data from the downstream problem of interest. However, they struggle to retain that performance when the data characteristics changes. In this paper, we focus on continual learning, where a pre-trained transformer is updated to perform well on new data, while retaining its performance on data it was previously
    Computer vision
  • Detecting content segments from online sports streaming events: Challenges and solutions
    Zongyi (Joe) Liu, Yarong Feng, Shunyan Luo, Yuan Ling, Shujing Dong, Shuyi Wang
    WACV 2024
    2023
    Developing a client-side segmentation algorithm for online sports streaming holds significant importance. For instance, in order to assess the video quality from an end-user perspective such as artifact detection, it is important to initially segment the content within the streaming playback. The challenge lies in localizing the content due to the intricate scene changes between content and non-content
    Computer vision
Load more

Related content

Work with us

See more jobs See more jobs
Applied Scientist, Amazon
US, WA, Seattle
An information-rich and accurate product catalog is a strategic asset for Amazon. It powers unrivaled product discovery, informs customer buying decisions, offers a large selection, and positions Amazon as the first stop for shopping online. We use data analysis and statistical and machine learning techniques to proactively identify relationships between products within the Amazon product catalog. This problem is challenging due to sheer scale (billions of products in the catalog), diversity (products ranging from electronics to groceries to instant video across multiple languages) and multitude of input sources (millions of sellers contributing product data with different quality). Amazon’s Item and Relationship Identity Systems group is looking for an innovative and customer-focused applied scientist to help us make the world’s best product catalog even better. In this role, you will partner with technology and business leaders to build new state-of-the-art algorithms, models, and services to infer product-to-product relationships that matter to our customers. You will work in a collaborative environment where you can experiment with massive data from the world’s largest product catalog, work on challenging problems, quickly implement and deploy your algorithmic ideas at scale, understand whether they succeed via statistically relevant experiments across millions of customers. Key job responsibilities * Map business requirements and customer needs to a scientific problem. * Align the research direction to business requirements and make the right judgments on research/development schedule and prioritization. * Research, design and implement scalable machine learning (ML) techniques to solve problems that matter to our customers in an iterative fashion. * Design, experiment and evaluate highly innovative models for predictive, explainable learning * Partner with other scientists to build state-of-the-art ML systems powering Amazon * Work closely with software engineering teams to drive real-time model experiments, implementations and new feature creations * Stay informed on the latest machine learning, natural language and/or artificial intelligence trends and make presentations to the larger engineering and applied science communities. About the team The IRIS team owns programs and systems to ensure uniqueness and consistency of product identity and to infer relationships between products in Amazon Catalog. We focus on the following areas: 1) reducing customer perceived duplicates: eliminating all duplicate ASINs that are indistinguishable by customers and identifying broken and missing variations, 2) reducing product detail page inconsistency: preventing inconsistent item identities, and improving the customer experience by automatically detecting and creating factual relationships between ASINs: e.g. variation families, newer versions, 3) reducing selling partner listing friction: reducing GTIN defects in the catalog, and false conflicts in contributions, and 4) improving brand customer experience: providing a strong brand identity to contributions and ASINs, by matching them to Universal Brand Catalog brand entities.
Applied Scientist, Sponsored Products, Amazon Ads
US, NY, New York
Amazon continues to invest heavily in building our world class advertising business. Our products are strategically important to our Retail and Marketplace businesses, driving long term growth. We deliver billions of ad impressions and millions of clicks daily, breaking fresh ground to create world-class products. We are highly motivated, collaborative and fun-loving with an entrepreneurial spirit and strong bias for action. With a broad mandate to experiment and innovate, we are growing at an unprecedented rate with a seemingly endless range of new opportunities.The Sponsored Products OSSR team is responsible for all non-search supply and associated experiences. As an Applied Scientist on our team, you will be responsible for defining the science and technical strategy for one of our most impactful strategic initiatives, creating lasting value for Amazon and our advertising customers. Key job responsibilities • Support business, science and engineering strategy and roadmap for Sponsored Products OSSR projects • Drive alignment across organizations for science, engineering and product strategy to achieve business goals • Lead/guide scientists and engineers across teams to develop, test, launch and improve of science models designed to optimize the shopper experience and deliver long term value for Amazon and advertisers • Develop state of the art experimental approaches and ML models. About the team Sponsored Products (SP) is Amazon's largest and fastest growing business. Over the last few years we grown to a multi-billion dollar business. SP ads are shown prominently throughout search and detail pages, allowing shoppers to seamlessly discover products sold on Amazon. Ad experience and market place is one of the highest impact decisions we make. This role has unparalleled opportunity to grow our marketplace and deliver value for advertisers and shoppers.
Applied Scientist, Machine Learning Accelerator
US, WA, Seattle
Do you want to join an innovative team of scientists who use deep learning, natural language processing, large language models to help Amazon provide the best seller experience across the entire Seller life cycle, including recruitment, growth, support and provide the best customer and seller experience by automatically mitigating risk? Do you want to build advanced algorithmic systems that help manage the trust and safety of millions of customer interactions every day? Are you excited by the prospect of analyzing and modeling terabytes of data and creating state-of-the-art algorithms to solve real world problems? Are you excited by the opportunity to leverage GenAI and innovate on top of the state-of-the-art large language models to improve customer and seller experience? Do you like to build end-to-end business solutions and directly impact the profitability of the company? Do you like to innovate and simplify processes? If yes, then you may be a great fit to join the Machine Learning Accelerator team in the Amazon Selling Partner Services (SPS) group. Key job responsibilities The scope of an Applied Scientist II in the Selling Partner Services (SPS) Machine Learning Accelerator (MLA) team is to research and prototype Machine Learning applications that solve strategic business problems across SPS domains. Additionally, the scientist collaborates with engineers and business partners to design and implement solutions at scale when they are determined to be of broad benefit to SPS organizations. They develop large-scale solutions for high impact projects, introduce tools and other techniques that can be used to solve problems from various perspectives, and show depth and competence in more than one area. They influence the team’s technical strategy by making insightful contributions to the team’s priorities, approach and planning. They develop and introduce tools and practices that streamline the work of the team, and they mentor junior team members and participate in hiring.
Senior Applied Scientist, Amazon
US, NY, New York
We are seeking a motivated and experienced Senior Applied Scientist with expertise in Machine Learning (ML), Artificial Intelligence (AI), Big Data, and Service Oriented Architecture. You should have a deep understanding of the digital advertising business and scaled marketing across communication channels. In this role, you will collaborate with a cross-functional team of talented scientists and engineers to innovate, iterate, and solve real-world marketing problems with cutting-edge AWS technologies. You will lead in-depth analyses of the key problems faced by Amazon Ads customers and the challenges faced by marketing teams in meeting customer needs at scale. To address these problems, you will build innovative large-scale ML/AI solutions such as bespoke omni-channel recommender systems, and specialized LLM-powered assistants for customers and marketers. You will independently drive research and prototyping to deliver functional proofs of concept (POCs), and then partner with engineers to inform the technology roadmap and deploy successful POCs as scalable batch and real-time applications in production. Key job responsibilities • Define and execute a research and development plan that enables data-driven marketing decisions and delivers inspiring customer experiences • Evaluate, evolve, and invent scientific techniques to effectively address customer needs and business problems • Establish and drive science prototyping best practices to ensure coherence and integrity of data feeding into production ML/AI solutions • Collaborate with colleagues across science and engineering disciplines for rapid prototyping at scale • Partner with engineering teams to solve complex technical problems, define system-level requirements, develop implementation plans, and guide the adaptation of techniques to meet production needs • Partner with product managers and stakeholders to define forward-looking product visions and prospective business use-cases • Drive and lead of culture of data-driven innovation within and outside across Amazon Ads Marketing organization • Influence organizational vision across Ads Marketing organization About the team The Marketing Decisions Science team provides AI/ML products to enable Amazon Ads Marketing to deliver relevant and compelling guidance, education, and inspiration to prospective and active advertisers across marketing channels. We own the product, technology, and deployment roadmap for AI/ML products across Amazon Ads Marketing. We analyze the needs, experiences, and behaviors of Amazon advertisers at petabytes scale, to deliver the right marketing communications to the right advertiser at the right time. Our products enable applications and synergies across Ads organization, spanning marketing, product, and sales use cases.
Sr. Data Scientist, Generative AI Innovation Center
US, NY, New York
Are you looking to work at the forefront of Machine Learning and AI? Would you be excited to apply cutting edge Generative AI algorithms to solve real world problems with significant impact? The Generative AI Innovation Center at AWS is a new strategic team that helps AWS customers implement Generative AI solutions and realize transformational business opportunities. This is a team of strategists, data scientists, engineers, and solution architects working step-by-step with customers to build bespoke solutions that harness the power of generative AI. The team helps customers imagine and scope the use cases that will create the greatest value for their businesses, select and train and fine tune the right models, define paths to navigate technical or business challenges, develop proof-of-concepts, and make plans for launching solutions at scale. The GenAI Innovation Center team provides guidance on best practices for applying generative AI responsibly and cost efficiently. You will work directly with customers and innovate in a fast-paced organization that contributes to game-changing projects and technologies. You will design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience. We’re looking for Data Scientists capable of using GenAI and other techniques to design, evangelize, and implement state-of-the-art solutions for never-before-solved problems. This position requires that the candidate selected be a US Citizen. Key job responsibilities As an Data Scientist, you will - Collaborate with AI/ML scientists and architects to Research, design, develop, and evaluate cutting-edge generative AI algorithms to address real-world challenges - Interact with customers directly to understand the business problem, help and aid them in implementation of generative AI solutions, deliver briefing and deep dive sessions to customers and guide customer on adoption patterns and paths to production - Create and deliver best practice recommendations, tutorials, blog posts, sample code, and presentations adapted to technical, business, and executive stakeholder - Provide customer and market feedback to Product and Engineering teams to help define product direction A day in the life About AWS Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
Applied Scientist, Sponsored Products
US, NY, New York
Amazon Ads is one of Amazon's fastest growing and most profitable businesses. As a core product offering within our advertising portfolio, Sponsored Products (SP) helps merchants, retail vendors, and brand owners succeed via native advertising, which grows incremental sales of their products sold through Amazon. The SP team's primary goals are to help shoppers discover new products they love, be the most efficient way for advertisers to meet their business objectives, and build a sustainable business that continuously innovates on behalf of customers. Our products and solutions are strategically important to drive long-term growth. We deliver billions of ad impressions and millions of clicks and break fresh ground in product and technical innovations every day! The Bespoke Shopping Experience team within SP develops customer facing experiences and machine learning models to better understand and address the diverse needs and behaviors of various shopper cohorts. As an Applied Scientist on the team, you will: - Drive end-to-end Machine Learning projects that have a high degree of ambiguity, scale, complexity. - Perform hands-on analysis and modeling of enormous data sets to develop insights that increase traffic monetization and merchandise sales, without compromising the shopper experience. - Build machine learning models, perform proof-of-concept, experiment, optimize, and deploy your models into production; work closely with software engineers to assist in productionizing your ML models. - Run A/B experiments, gather data, and perform statistical analysis. - Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving. - Research new and innovative machine learning approaches.
Senior Research Scientist, AIGC Science and Analytics
US, NY, New York
The PXT (People Experience and Technology) Science and Analytics team for AIGC (Ads, IMDb and Grand Challenge) is seeking a highly skilled and motivated Senior Research Scientist to join our team. You will be leading Research Science space to support the AIGC PXT org initiatives. If you enjoy innovating, thinking big and want to contribute directly to the success of a growing team, you may be a prime candidate for this position. Key job responsibilities In this role you will: Design experiments, test hypotheses, and build actionable models Conduct quantitative analyses of talent management data and trends Conduct qualitative data collection and analysis Partner closely and drive effective collaborations across multi-disciplinary research and product teams Consult on appropriate analytic methodologies and scope research requests
Research Scientist, Prime Video
US, WA, Seattle
Prime Video is a first-stop entertainment destination offering customers a vast collection of premium programming in one app available across thousands of devices. On Prime Video, customers can find their favorite movies, series, documentaries, and live sports – including Amazon MGM Studios-produced series and movies Road House, The Lord of the Rings: The Rings of Power, Fallout, Reacher, The Boys, and The Idea of You; licensed fan favorites Dawson’s Creek and IF; Prime member exclusive access to coverage of live sports including Thursday Night Football, WNBA, and NWSL, and acclaimed sports documentaries including Bye Bye Barry and Federer; and programming from partners such as Apple TV+, Max, Crunchyroll, and MGM+ via Prime Video add-on subscriptions, as well as more than 500 free ad-supported (FAST) Channels. Prime members in the U.S. can share a variety of benefits, including Prime Video, by using Amazon Household. Prime Video is one benefit among many that provides savings, convenience, and entertainment as part of the Prime membership. All customers, regardless of whether they have a Prime membership or not, can rent or buy titles, including blockbusters such as Challengers and The Fall Guy, via the Prime Video Store, and can enjoy content such as Jury Duty and Bosch: Legacy free with ads on Freevee. Customers can also go behind the scenes of their favorite movies and series with exclusive X-Ray access. For more info visit www.amazon.com/primevideo. Are you interested in shaping the future of entertainment? Prime Video's technology teams are creating best-in-class digital video experience. As a Prime Video technologist, you’ll have end-to-end ownership of the product, user experience, design, and technology required to deliver state-of-the-art experiences for our customers. You’ll get to work on projects that are fast-paced, challenging, and varied. You’ll also be able to experiment with new possibilities, take risks, and collaborate with remarkable people. We’ll look for you to bring your diverse perspectives, ideas, and skill-sets to make Prime Video even better for our customers. With global opportunities for talented technologists, you can decide where a career Prime Video Tech takes you! Key job responsibilities As a Research Scientist at Prime Video, you will have end-to-end ownership of the product, related research and experimentation, applying advanced machine learning techniques in computer vision (CV), natural language processing (NLP), multimedia understanding and so on. You’ll work on diverse projects that enhance Prime Video’s recommendation systems, image/video understanding, and content personalization, driving impactful innovations for our global audience. Other responsibilities include: • Lead cutting-edge research in computer vision and natural language processing, applying it to video-centric media challenges. • Develop scalable machine learning models to enhance media asset generation, content discovery, and personalization. • Collaborate closely with engineering teams to integrate your models into production systems at scale, ensuring optimal performance and reliability. • Actively participate in publishing your research in leading conferences and journals. • Lead a team of skilled research scientists, you will shape the research strategy, create forward-looking roadmaps, and effectively communicate progress and insights to senior leadership • Stay up-to-date with the latest advancements in AI and machine learning to drive future research initiatives.
Senior Economist, Device Economics
US, WA, Seattle
Device Economics is looking for a senior economist experienced in causal inference, machine learning, empirical industrial organization, and scaled systems to work on business problems to advance critical resource allocation and pricing decisions in the Amazon Devices org. Senior roles lead vision setting, methods innovation, and act as thought leaders to Devices finance and business executives. Output will be included in scaled systems to automate existing processes and to maximize business and customer objectives. Amazon Devices designs and builds Amazon first-party consumer electronics products to delight and engage customers. Amazon Devices represents a highly complex space with 100+ products across several product categories (e-readers [Kindle], tablets [Fire Tablets], smart speakers and audio assistants [Echo], wifi routers [eero], and video doorbells and cameras [Ring and Blink]), for sale both online and in offline retailers in several regions. The space becomes more complex with dynamic product offering with new product launches and new marketplace launches. The Device Economics team leads in analyzing these complex marketplace dynamics to enable science-driven decision making in the Devices org. Device Economics achieves this through scientific applications that provide deep understanding of customer preferences. Our team’s outputs inform product development decisions, investments in future product categories, and product pricing and promotion. We have achieved substantial impact on the Devices business, and will achieve more. Device Economics seeks an experienced economist adept in measuring customer preferences and behaviors with proven capacity to innovate, scale measurement, drive rigor, and mentor talent. The candidate will work with Amazon Devices science leadership to refine science roadmaps, models, and priorities for innovation and simplification, and advance adoption of insights to influence important resource allocation and prioritization decisions. Effective communication skills (verbal and written) are required to ensure success of this collaboration. The candidate must be passionate about advancing science for business and customer impact.
Applied Scientist , Amazon
US, WA, Seattle
Amazon Advertising operates at the intersection of eCommerce and advertising, and is investing heavily in building a world-class advertising business. We are defining and delivering a collection of self-service performance advertising products that drive discovery and sales. Our products are strategically important to our Retail and Marketplace businesses driving long-term growth. We deliver billions of ad impressions and millions of clicks daily and are breaking fresh ground to create world-class products to improve both shopper and advertiser experience. With a broad mandate to experiment and innovate, we grow at an unprecedented rate with a seemingly endless range of new opportunities. The Ad Response Prediction team in Sponsored Products organization build advanced deep-learning models, large-scale machine-learning pipelines, and real-time serving infra to match shoppers’ intent to relevant ads on all devices, for all contexts and in all marketplaces. Through precise estimation of shoppers’ interaction with ads and their long-term value, we aim to drive optimal ads allocation and pricing, and help to deliver a relevant, engaging and delightful ads experience to Amazon shoppers. As the business and the complexity of various new initiatives we take continues to grow, we are looking for talented Applied Scientists to join the team. Key job responsibilities As a Applied Scientist II, you will: * Conduct hands-on data analysis, build large-scale machine-learning models and pipelines * Work closely with software engineers on detailed requirements, technical designs and implementation of end-to-end solutions in production * Run regular A/B experiments, gather data, perform statistical analysis, and communicate the impact to senior management * Establish scalable, efficient, automated processes for large-scale data analysis, machine-learning model development, model validation and serving * Provide technical leadership, research new machine learning approaches to drive continued scientific innovation * Be a member of the Amazon-wide Machine Learning Community, participating in internal and external MeetUps, Hackathons and Conferences
View from space of a connected network around planet Earth representing the Internet of Things.
Have feedback?
Let us know by taking this short survey
Get started
Amazon.com | Conditions of Use | Privacy | © 1996-2024 Amazon.com, Inc. or its affiliates
Follow Us