Feedback
Follow Us
Publication

PoCo: Point context cluster for RGBD indoor place recognition

By Jing Liang, Zhuo Deng, Zheming Zhou, Omid Ghasemalizadeh, Dinesh Manocha, Min Sun, Cheng-Hao Kuo, Arnie Sen
2024
Download Copy BibTeX
Share
Download
Copy BibTeX
Share
We present a novel end-to-end algorithm (PoCo) for the indoor RGB-D place recognition task, aimed at identifying the most likely match for a given query frame within a reference database. The task presents inherent challenges attributed to the constrained field of view and limited range of perception sensors. We propose a new network architecture, which generalizes the recent Context of Clusters (CoCs) to extract global descriptors directly from the noisy point clouds through end-to-end learning. Moreover, we develop the architecture by integrating both color and geometric modalities into the point features to enhance the global descriptor representation. We conducted evaluations on public datasets ScanNet-PR and ARKit with 807 and 5047 scenarios, respectively. PoCo achieves SOTA performance: on ScanNet-PR, we achieve R@1 of 64.63%, a 5.7% improvement from the best-published result CGis (61.12%); on Arkit, we achieve R@1 of 45.12%, a 13.3%improvement from the best-published result CGis (39.82%). In addition, PoCo shows higher efficiency than CGis in inference time (1.75X-faster), and we demonstrate the effectiveness of PoCo in recognizing places within a real-world laboratory environment.
Research areas
Tags

Latest news

Work with us

See more jobs See more jobs
Research Scientist, Shopping Conversation Foundation
US, WA, Seattle
Our team's mission is to improve Shopping experience for customers interacting with Amazon devices via voice. We work with Alexa and multiple other teams to research and develop advanced state-of-the-art speech technologies. Do you want to be part of the team developing the latest technology that impacts the customer experience of ground-breaking products? Then come join us and make history. Key job responsibilities We are looking for a passionate, talented, and inventive Research Scientist with a background in Machine Learning to help build industry-leading Speech and Language technology. As a Research Scientist at Amazon you will work with talented peers to develop novel algorithms and modelling techniques to drive the state of the art in speech synthesis. Position Responsibilities: * Participate in the design, development, evaluation, deployment and updating of data-driven models for Speech and Language applications. * Participate in research activities including the application and evaluation of Speech and Language techniques for novel applications. * Research and implement novel ML and statistical approaches to add value to the business.
Applied Scientist, People eXperience and Technology Central Science Team (PXTCS)
US, WA, Seattle
Are you motivated to explore research in ambiguous spaces? Are you interested in conducting research that will improve associate, employee and manager experiences at Amazon? Do you want to work on an interdisciplinary team of scientists that collaborate rather than compete? Join us at PXT Central Science! The People eXperience and Technology Central Science Team (PXTCS) uses economics, behavioral science, statistics, and machine learning to proactively identify mechanisms and process improvements which simultaneously improve Amazon and the lives, wellbeing, and the value of work to Amazonians. We are an interdisciplinary team that combines the talents of science and engineering to develop and deliver solutions that measurably achieve this goal. Key job responsibilities As an Applied Scientist for People Experience and Technology (PXT) Central Science, you will be working with our science and engineering teams, specifically on re-imagining Generative AI Applications and Generative AI Infrastructure for HR. Applying Generative AI to HR has unique challenges such as privacy, fairness, and seamlessly integrating Enterprise Knowledge and World Knowledge and knowing which to use when. In addition, the team works on some of Amazon’s most strategic technical investments in the people space and support Amazon’s efforts to be Earth’s Best Employer. In this role you will have a significant impact on 1.5 million Amazonians and the communities Amazon serves and ample scope to demonstrate scientific thought leadership and scientific impact in addition to business impact. You will also play a critical role in the organization's business planning, work closely with senior leaders to develop goals and resource requirements, influence our long-term technical and business strategy, and help hire and develop science and engineering talent. You will also provide support to business partners, helping them use the best scientific methods and science-driven tools to solve current and upcoming challenges and deliver efficiency gains in a changing marke About the team The AI/ML team in PXTCS is working on building Generative AI solutions to reimagine Corp employee and Ops associate experience. Examples of state-of-the-art solutions are Coaching for Amazon employees (available on AZA) and reinventing Employee Recruiting and Employee Listening.
Applied Scientist, Alexa Daily Essentials
CA, BC, Vancouver
Alexa Daily Essentials is hiring an Applied Scientist to research and implement large language model innovations to enhance Alexa's language understanding, knowledge representation, reasoning and generation capabilities. The Alexa Daily Essentials team delivers experiences critical to how customers interact with Alexa as part of daily life. We drive over 40 billion+ actions annually across 60 million+ monthly customers, who engage with our products across experiences connected to Timers, Alarms, Calendars, Food, and News. Our experiences include critical time saving techniques, ad-supported news audio and video, and in-depth kitchen guidance aimed at serving the needs of the family from sunset to sundown. Our upcoming launches are at the forefront of innovation, delivering step-function improvements in experiences that stretch across the customer journey, and new AI technologies that will enable customers to send Alexa information for future recall and conversation. We collaborate closely with partners such as Amazon.com, Whole Foods, Spotify, CNN, Fox, NPR, BBC, Discovery, and Food Network to deliver our vision. If you are passionate about redefining the personal assistant experience and leveraging innovative technology to improve daily life, we’d love to hear from you. This is an opportunity to make a tangible impact at the heart of the Alexa ecosystem. As an applied scientist, you will advance state of the art techniques in ML and LLM, and work closely with product and engineering teams to build the next generation of the Alexa smart assistant. Key job responsibilities - Rapidly prototype ML/LLM solutions, evaluate feasibility, and drive projects to production deployment - Continuously monitor and improve model performance through retraining, parameter tuning, and architecture refinements - Develop new training and inference techniques to improve model performance - Work cross-functionally across engineering, product, and business teams to understand customer needs, scope science work, and drive science solutions from conception to customer delivery - Research and develop LLM innovations, and lead paper publications. - Code proficiently in Python (required) and Java (preferred); optimize systems for high performance at scale; contribute code directly into production services - Innovate and develop science and engineering solutions that optimize team operations and increase team effectiveness. - Clearly communicate complex technical concepts to non-technical stakeholders and leadership
Amazon Robotics - Data Scientist (New Grad)
US, MA, North Reading
Are you inspired by invention? Is problem solving through teamwork in your DNA? Do you like the idea of seeing how your work impacts the bigger picture? Answer yes to any of these and you’ll fit right in here at Amazon Robotics. We are a smart team of doers who work passionately to apply cutting edge advances in robotics and software to solve real-world challenges that will transform our customers’ experiences. We invent new improvements every day. We are Amazon Robotics and we will give you the tools and support you need to invent with us in ways that are rewarding, fulfilling, and fun. Amazon Robotics empowers a smarter, faster, more consistent customer experience through automation. Amazon Robotics automates fulfillment center operations using various methods of robotic technology including autonomous mobile robots, sophisticated control software, language perception, power management, computer vision, depth sensing, machine learning, object recognition, and semantic understanding of commands. Amazon Robotics has a dedicated focus on research and development to continuously explore new opportunities to extend its product lines into new areas. Key job responsibilities We are seeking an enthusiastic Data Scientist to: - Design and implement state-of-the-art solutions for never-before-solved problems. - Collaborate closely with other research and robotics experts to design and run experiments, research new algorithms, and find new ways to improve Amazon Robotics analytics to optimize the Customer experience. - Partner with technology and product leaders to solve business problems using scientific approaches. - Build new tools and invent business insights that surprise and delight our customers. - Work to quantify system performance at scale, and to expand the breadth and depth of our analysis to increase the ability of software components and warehouse processes. - Work to evolve our library of key performance indicators and construct experiments that efficiently root cause emergent behaviors. - Engage with software development teams and warehouse design engineers to drive the evolution of the Amazon Robotics system, as well as the simulation engine that supports our work. A day in the life As a member of the Software Research and Data Science (SWRDS) team, there are multiple different paths you can take. For example, you may conduct deep dive analysis of both structured and unstructured robotics field analyses. Some of our team members focus on modeling, pipelines and tooling work, while others focus more on analytic work. We also have team members working on early-concept solution analysis (leveraging simulations, ML-based models, experimenting with new solutions). About the team The Amazon Robotics Software Research and Data Science (SWRDS) team builds and runs simulation experiments and delivers analyses that are central to understanding the performance of the entire Amazon Robotics system. This includes operational and software scaling characteristics, bottlenecks, and robustness to “chaos monkey” stresses -- we inform critical engineering and business decisions about Amazon’s approach to robotic fulfillment.
Sr. Applied Scientist, Amazon
US, WA, Seattle
We are building GenAI based shopping assistant for Amazon. We reimage Amazon Search with an interactive conversational experience that helps you find answers to product questions, perform product comparisons, receive personalized product suggestions, and so much more, to easily find the perfect product for your needs. We’re looking for the best and brightest across Amazon to help us realize and deliver this vision to our customers right away. This will be a once in a generation transformation for Search, just like the Mosaic browser made the Internet easier to engage with three decades ago. If you missed the 90s—WWW, Mosaic, and the founding of Amazon and Google—you don’t want to miss this opportunity.
Sr. Applied Scientist, Private Brands Intelligence - Sourcing Guidance
US, WA, Seattle
The Private Brands team is looking for an Applied Scientist to join the team in building science solutions at scale. Our team applies Optimization, Machine Learning, Statistics, Causal Inference, and Econometrics/Economics to derive actionable insights. We are an interdisciplinary team of Scientists, Engineers, and Economists and primary focus on building optimization and machine learning solutions in supply chain domain with specific focus on Amazon private brand products. Key job responsibilities You will work with business leaders, scientists, and economists to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable optimization solutions and ML models. This is a unique, high visibility opportunity for someone who wants to have business impact, dive deep into large-scale problems, enable measurable actions on the consumer economy, and work closely with scientists and economists. As a senior scientist, you bring business and industry context to science and technology decisions. You set the standard for scientific excellence and make decisions that affect the way we build and integrate algorithms. Your solutions are exemplary in terms of algorithm design, clarity, model structure, efficiency, and extensibility. You tackle intrinsically hard problems, acquiring expertise as needed. You decompose complex problems into straightforward solutions. We are particularly interested in candidates with academic and/or practical background in Operations Research and Machine Learning. Experience in applying Operations Research and/or ML to supply chain problems is a plus.
Applied Scientist, Private Brands Intelligence - Sourcing Guidance
US, CA, San Diego
The Private Brands team is looking for an Applied Scientist to join the team in building science solutions at scale. Our team applies Optimization, Machine Learning, Statistics, Causal Inference, and Econometrics/Economics to derive actionable insights. We are an interdisciplinary team of Scientists, Engineers, and Economists and primary focus on building optimization and machine learning solutions in supply chain domain with specific focus on Amazon private brand products. Key job responsibilities You will work with business leaders, scientists, and economists to translate business and functional requirements into concrete deliverables, including the design, development, testing, and deployment of highly scalable optimization solutions and ML models. This is a unique, high visibility opportunity for someone who wants to have business impact, dive deep into large-scale problems, enable measurable actions on the consumer economy, and work closely with scientists and economists. As a scientist, you bring business and industry context to science and technology decisions. You set the standard for scientific excellence and make decisions that affect the way we build and integrate algorithms. Your solutions are exemplary in terms of algorithm design, clarity, model structure, efficiency, and extensibility. You tackle intrinsically hard problems, acquiring expertise as needed. You decompose complex problems into straightforward solutions. We are particularly interested in candidates with experience in predictive and machine learning models and working with distributed systems. Academic and/or practical background in Machine Learning are particularly relevant for this position. Familiarity and experience in applying Operations Research techniques to supply chain problems is a plus.
Data Scientist, Amazon Private Brands Intelligence - Sourcing Guidance
US, CA, San Diego
Amazon Private Brands is looking for a Data Scientist to join our Private Brand Intelligence (PBI) Sourcing Guidance team. PBI applies Machine Learning, Statistics, and Econometrics/Economics to derive actionable insights about the complex economy of Amazon’s retail business and develop statistical models and algorithms to drive strategic business decisions and improve operations. About the team We are an interdisciplinary team of Scientists, Economists, and Engineers, incubating and building day one solutions using cutting-edge technology, to solve some of the toughest business problems at Amazon. About the role You will work with business leaders, PMs, scientists, and economists to deep dive existing business problems, translate them into business and functional requirements and design concrete deliverables. These deliverables can include the design, development, testing of new in-house statistical models/ML models/Optimization engines, etc. and/or partnering with our sister teams to develop an improved version of an existing model/system. This is a unique, high visibility opportunity for someone who wants to have business impact, dive deep into large-scale economic problems and enable measurable actions on the consumer economy. We are particularly interested in candidates with experience applying stat, ML and OR concepts to business problems. To learn more about Amazon Science, please visit https://www.amazon.science (https://www.amazon.science/).
Amazon Robotics - Human Factors and Ergonomics Research Scientist
US, MA, North Reading
Are you inspired by invention? Is problem solving through teamwork in your DNA? Do you like the idea of seeing how your work impacts the bigger picture? Answer yes to any of these and you’ll fit right in here at Amazon Robotics. We are a smart team of doers that work passionately to apply cutting edge advances in robotics and software to solve real-world challenges that will transform our customers’ experiences in ways we can’t even imagine yet. We invent new improvements every day. We are Amazon Robotics and we will give you the tools and support you need to invent with us in ways that are rewarding, fulfilling and fun. Amazon Robotics is looking for a bright and motivated research scientist who is excited to work on complex, real-world challenges. We are interested in someone who has experience applying human factors and ergonomics standards, method, and analysis tools. The ideal candidate would be familiar with the ISO 11228 standard (Ergonomics - Manual Handling) and have experience conducting ergonomic evaluations with common tools, such as the NIOSH lifting equation, ACGIH threshold limit values, Recommended Cumulative Rest Allowance (RCRA) method, and the Liberty Mutual Manual Material Handling tables. We are also seeking candidates who have experience with digital human modeling software, such as Siemens Jack or Process Simulate, wearable/optical motion capture systems, and EMG analysis. Additionally, we are seeking candidates who have experience conducting ergonomics/biomechanics research and publishing peer reviewed articles in reputable journals and/or conference proceedings. The ideal candidate would have experience coding in MATLAB and/or Python, and be able to utilize data science and machine learning to develop novel ergonomic assessment methodologies. Key job responsibilities The role will entail conducting ergonomics research to ensure the safety of Amazon associates, operators, and maintainers. This research will help shape the HFE team’s methodology, allowing us to better guide product design decisions that limit injury risk. You will help to identify fundamental questions pertaining to human capabilities and tolerances in a variety of work environments and provide human factors/ergonomics input to teams across Amazon Robotics, such as Hardware Engineering, Product and Program Management, Safety, Systems Engineering, and Operations. You will work closely with our ergonomics research colleagues in the Workplace Health and Safety (WHS) organization to develop and refine a unified toolkit of ergonomic analysis methodologies. Your responsibilities include: * Conducting novel HFE research to improve the speed and accuracy of our assessment methodologies. * Staying up-to-date on the latest in ergonomics industry research, and recommending new approaches/technologies that the team should consider adopting. * Contributing to comprehensive assessments of workstations and processes covering biomechanical, physiological, and psychophysical demands. * Continuously striving to gain in-depth knowledge of your profession, as well as branch out to learn about intersecting fields, such as user experience, industrial design, robotics and mechatronics. * Supporting the development of internal guidelines, process documents, and standards to broaden our impact and improve the clarity of our methods for the hardware design organization. * Occasionally traveling to various sites to conduct research or support product assessments (e.g., collect biomechanical/physiological data and collect associate feedback), likely 3-4 trips per year (3-5 days per trip). About the team At Amazon Robotics, we create high tech products that help associates to work faster, safer, and with greater satisfaction. In order to best serve our associate customers, we need to incorporate ergonomic considerations into the design cycle for associate facing hardware development projects. The Human Factors and Ergonomics (HFE) team conducts iterative evaluations of every new Amazon Robotics product, in order to minimize injury potential and enhance the overall associate experience. Utilizing advanced technologies and cutting-edge research, we develop and institutionalize state-of-the-art ergonomic assessment tools that accurately and efficiently predict impacts to associates working in AR fulfillment centers.
Data Scientist, Amazon Stores Finance Science
US, WA, Seattle
WW Amazon Stores Finance Science (ASFS) works to leverage science and economics to drive improved financial results, foster data backed decisions, and embed science within Finance. ASFS is focused on developing products that empower controllership, improve business decisions and financial planning by understanding financial drivers, and innovate science capabilities for efficiency and scale. We are looking for an outstanding data scientist to lead high visibility initiatives for forecasting Amazon Stores' financials. You will develop new science-based forecasting methodologies and build scalable models to improve financial decision making and planning for senior leadership up to VP and SVP level. You will build new ML and statistical models from the ground up that aim to transform financial planning for Amazon Stores. We prize creative problem solvers with the ability to draw on an expansive methodological toolkit to transform financial decision-making with science. The ideal candidate combines data-science acumen with strong business judgment. You have versatile modeling skills and are comfortable owning and extracting insights from data. You are excited to learn from and alongside seasoned scientists, engineers, and business leaders. You are an excellent communicator and effectively translate technical findings into business action. Key job responsibilities - Demonstrating thorough technical knowledge on feature engineering with large datasets, effective exploratory data analysis, and model building using industry standard ML models - Working with technical and non-technical stakeholders across every step of science project life cycle - Collaborating with finance, product, data engineering, and software engineering teams to create production implementations for large-scale ML models - Innovating by adapting new modeling techniques and procedures - Presenting research results to our internal research community
View from space of a connected network around planet Earth representing the Internet of Things.
Have feedback?
Let us know by taking this short survey
Get started
Amazon.com | Conditions of Use | Privacy | © 1996-2024 Amazon.com, Inc. or its affiliates
Follow Us