Feedback
Follow Us
Publication

Amazon Redshift and the case for simpler data warehouses

By Anurag Gupta, Deepak Agarwal, Derek Tan, Jakub Kulesza, Rahul Pathak, Stefano Stefani, Vidhya Srinivasan
2015
Download Copy BibTeX
Share
Download
Copy BibTeX
Share
Amazon Redshift is a fast, fully managed, petabyte-scale data warehouse solution that makes it simple and cost-effective to efficiently analyze large volumes of data using existing business intelligence tools. Since launching in February 2013, it has been Amazon Web Service’s (AWS) fastest growing service, with many thousands of customers and many petabytes of data under management.

Amazon Redshift’s pace of adoption has been a surprise to many participants in the data warehousing community. While Amazon Redshift was priced disruptively at launch, available for as little as $1000/TB/year, there are many open-source data warehousing technologies and many commercial data warehousing engines that provide free editions for development or under some usage limit. While Amazon Redshift provides a modern MPP, columnar, scale-out architecture, so too do many other data warehousing engines. And, while Amazon Redshift is available in the AWS cloud, one can build data warehouses using EC2 instances and the database engine of one’s choice with either local or networkattached storage.

In this paper, we discuss an oft-overlooked differentiating characteristic of Amazon Redshift – simplicity. Our goal with Amazon Redshift was not to compete with other data warehousing engines, but to compete with non-consumption. We believe the vast majority of data is collected but not analyzed. We believe, while most database vendors target larger enterprises, there is little correlation in today’s economy between data set size and company size. And, we believe the models used to procure and consume analytics technology need to support experimentation and evaluation. Amazon Redshift was designed to bring data warehousing to a mass market by making it easy to buy, easy to tune and easy to manage while also being fast and cost-effective.
Research areas
Tags

Latest news

Work with us

See more jobs See more jobs
Reseach Scientist, AWS Q for Business
US, CA, Santa Clara
AWS AI is looking for passionate, talented, and inventive Research Scientists with a strong machine learning background to help build industry-leading Conversational AI Systems. Our mission is to provide a delightful experience to Amazon’s customers by pushing the envelope in Natural Language Understanding (NLU), Dialog Systems including Generative AI with Large Language Models (LLMs) and Applied Machine Learning (ML). As part of our AI team in Amazon AWS, you will work alongside internationally recognized experts to develop novel algorithms and modeling techniques to advance the state-of-the-art in human language technology. Your work will directly impact millions of our customers in the form of products and services that make use language technology. You will gain hands on experience with Amazon’s heterogeneous text, structured data sources, and large-scale computing resources to accelerate advances in language understanding. We are hiring in all areas of human language technology: NLU, Dialog Management, Conversational AI, LLMs and Generative AI. About the team Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Utility Computing (UC) AWS Utility Computing (UC) provides product innovations — from foundational services such as Amazon’s Simple Storage Service (S3) and Amazon Elastic Compute Cloud (EC2), to consistently released new product innovations that continue to set AWS’s services and features apart in the industry. As a member of the UC organization, you’ll support the development and management of Compute, Database, Storage, Internet of Things (IoT), Platform, and Productivity Apps services in AWS, including support for customers who require specialized security solutions for their cloud services. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud.
Applied Scientist I, Amazon
US, WA, Seattle
Our team's mission is to improve Shopping experience for customers interacting with Amazon devices via voice. We research and develop advanced state-of-the-art speech and language modeling technologies. Do you want to be part of the team developing the latest technology that impacts the customer experience of ground-breaking products? Then come join us and make history. Key job responsibilities We are looking for a passionate, talented, and inventive Applied Scientist with a background in Machine Learning to help build industry-leading Speech and Language technology. As an Applied Scientist at Amazon you will work with talented peers to develop novel algorithms and modelling techniques to drive the state of the art in speech synthesis. Position Responsibilities: * Participate in the design, development, evaluation, deployment and updating of data-driven models for Speech and Language applications. * Participate in research activities including the application and evaluation of Speech and Language techniques for novel applications. * Research and implement novel ML and statistical approaches to add value to the business. * Mentor junior engineers and scientists.
Applied Scientist, Shanghai AI Lab
CN, 31, Shanghai
The AWS Shanghai AI Lab is looking for a passionate, talented, and inventive staff in all AI domains with a strong machine learning background as an Applied Scientist. Founded in 2018, the Shanghai Lab has been an innovation center of for long-term research projects across domains as machine learning, computer vision, natural language processing, and open-source AI system. Meanwhile, these incubated projects power products across various AWS services. As part of the lablet, you will take a leadership role and join a vibrant team with a diverse set of expertise in both machine learning and applicational domains. You will work on state-of-the-art solutions on fundamental research problems with other world-class scientists and engineers in AWS around the globe and across the boarders. You will have the responsibility to design and innovate solutions to our customers. You will build models to tame large amount of data, achieve industry-level scalability and efficiency, and along the way rapidly grow and build the team.
Applied Scientist II, Generative AI Innovation Center
US, VA, Herndon
Machine learning (ML) has been strategic to Amazon from the early years. We are pioneers in areas such as recommendation engines, product search, eCommerce fraud detection, and large-scale optimization of fulfillment center operations. The Generative AI team helps AWS customers accelerate the use of Generative AI to solve business and operational challenges and promote innovation in their organization. As an applied scientist, you are proficient in designing and developing advanced ML models to solve diverse challenges and opportunities. You will be working with terabytes of text, images, and other types of data to solve real-world problems. You'll design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience. We’re looking for talented scientists capable of applying ML algorithms and cutting-edge deep learning (DL) and reinforcement learning approaches to areas such as drug discovery, customer segmentation, fraud prevention, capacity planning, predictive maintenance, pricing optimization, call center analytics, player pose estimation, event detection, and virtual assistant among others. Key job responsibilities The primary responsibilities of this role are to: • Design, develop, and evaluate innovative ML models to solve diverse challenges and opportunities across industries • Interact with customer directly to understand their business problems, and help them with defining and implementing scalable Generative AI solutions to solve them • Work closely with account teams, research scientist teams, and product engineering teams to drive model implementations and new solution About the team ABOUT AWS: Diverse Experiences Amazon values diverse experiences. Even if you do not meet all of the preferred qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why we strive for flexibility as part of our working culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship and Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional.
Senior Economist, Specialized Selection
US, WA, Bellevue
Amazon is looking for an outstanding Senior Economist to help build next generation selection/assortment systems. On the Specialized Selection team within the Supply Chain Optimization Technologies (SCOT) organization, we own the selection to determine which products Amazon offers in our fastest delivery programs. We build tools and systems that enable our partners and business owners to scale themselves by leveraging our problem domain expertise, focusing instead on introspecting our outputs and iteratively helping us improve our ML models rather than hand-managing their assortment. We partner closely with our business stakeholders as we work to develop state-of-the-art, scalable, automated selection. Our team is highly cross-functional and employs a wide array of scientific tools and techniques to solve key challenges, including supervised and unsupervised machine learning, non-convex optimization, causal inference, natural language processing, linear programming, reinforcement learning, and other forecast algorithms. Some critical research areas in our space include modeling substitutability between similar products, incorporating basket awareness and complementarity-aware logic, measuring speed sensitivity of products, modeling network capacity constraints, and supply and demand forecasting. We're looking for a candidate with a background in experiment design and causal analysis to lead studies related to selection and speed. Potential projects include understanding the short-term and long-term customer impact of assortment changes across different speed. As an Senior Economist, you'll build econometric models using our world-class data systems and apply economic theory to solve business problems in a fast-moving environment. You will work with software engineers, product managers, and business teams to understand the business problems and requirements, distill that understanding to crisply define the problem, and design and develop innovative solutions to address them. To be successful in this role, you'll need to communicate effectively with product and tech teams, and translate data-driven findings into actionable insights. You'll thrive if you enjoy tackling ambiguous challenges using the economics toolkit and identifying and solving problems at scale. We have a supportive, fast-paced team culture, and we prioritize learning, growth, and helping each other continuously raise the bar. Key job responsibilities - Lead data-driven econometric studies to create future business opportunities - Consult with stakeholders in Selection and other teams to help solve existing business challenges - Independently identify and pursue new opportunities to leverage economic insights - Advise senior leaders and collaborate with other scientists to drive innovation - Support innovative delivery program growth worldwide - Write business and technical documents communicating business context, methods, and results to business leadership and other scientists - Serve as a technical lead and mentor for junior scientists, ensuring a high science bar - Serve as a technical reviewer for our team and related teams, including document and code reviews
Microwave Design Engineer, AWS MAS2 (WJV)
US, CA, Pasadena
The Amazon Web Services (AWS) Center for Quantum Computing in Pasadena, CA, is looking to hire a Research Scientist specializing the design of microwave components for cryogenic environments. Working alongside other scientists and engineers, you will design and validate hardware performing microwave signal conditioning at cryogenic temperatures for AWS quantum processors. Candidates must have a background in both microwave theory and implementation. Working effectively within a cross-functional team environment is critical. The ideal candidate will have a proven track record of hardware development from requirements development to validation. Key job responsibilities Our scientists and engineers collaborate across diverse teams and projects to offer state of the art, cost effective solutions for the signal conditioning of AWS quantum processor systems at cryogenic temperatures. You’ll bring a passion for innovation, collaboration, and mentoring to: Solve layered technical problems across our cryogenic signal chain. Develop requirements with key system stakeholders, including quantum device, test and measurement, cryogenic hardware, and theory teams. Design, implement, test, deploy, and maintain innovative solutions that meet both performance and cost metrics. Research enabling technologies necessary for AWS to produce commercially viable quantum computers. A day in the life As you design and implement cryogenic microwave signal conditioning solutions, from requirements definition to deployment, you will also: Participate in requirements, design, and test reviews and communicate with internal stakeholders. Work cross-functionally to help drive decisions using your unique technical background and skill set. Refine and define standards and processes for operational excellence. Work in a high-paced, startup-like environment where you are provided the resources to innovate quickly. About the team AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying.
Aerodynamics & Vehicle Design, Prime Air Flight Sciences Vehicle Design & Test
US, WA, Seattle
Here at Amazon, we embrace our differences. We are committed to furthering our culture of diversity and inclusion of our teams within the organization. How do you get items to customers quickly, cost-effectively, and—most importantly—safely, in less than an hour? And how do you do it in a way that can scale? Our teams of hundreds of scientists, engineers, aerospace professionals, and futurists have been working hard to do just that! We are delivering to customers, and are excited for what’s to come. Check out more information about Prime Air on the About Amazon blog (https://www.aboutamazon.com/news/transportation/amazon-prime-air-delivery-drone-reveal-photos). If you are seeking an iterative environment where you can drive innovation, apply state-of-the-art technologies to solve real world delivery challenges, and provide benefits to customers, Prime Air is the place for you. Come work on the Amazon Prime Air Team! Our Prime Air Drone Vehicle Design and Test team within Flight Sciences is looking for an outstanding engineer to help us rapidly configure, design, analyze, prototype, and test innovative drone vehicles. You’ll be responsible for assessing the Aerodynamics, Performance, and Stability & Control characteristics of vehicle designs. You’ll help build and utilize our suite of Multi-disciplinary Optimization (MDO) tools. You’ll explore new and novel drone vehicle conceptual designs in both focused and wide open design spaces, with the ultimate goal of meeting our customer requirements. You’ll have the opportunity to prototype vehicle designs and support wind tunnel and other testing of vehicle designs. You will directly support the Office of the Chief Program Engineer, and work closely across all vehicle subsystem teams to ensure integrated designs that meet performance, reliability, operability, manufacturing, and cost requirements. About the team Our Flight Sciences Vehicle Design & Test organization includes teams that span the following disciplines: Aerodynamics, Performance, Stability & Control, Configuration & Spatial Integration, Loads, Structures, Mass Properties, Multi-disciplinary Optimization (MDO), Wind Tunnel Testing, Noise Testing, Flight Test Instrumentation, and Rapid Prototyping.
Applied Scientist, Core Shopping
US, WA, Seattle
This is a unique opportunity to build technology and science that millions of people will use every day. Are you excited about working on large scale Natural Language Processing (NLP), Machine Learning (ML), and Large Language Models (LLM)? We are embarking on a multi-year journey to improve the shopping experience for customers using Alexa globally. In 2024, we started building all Shopping experiences leveraging LLMs in the US. We create customer-focused solutions and technologies that makes shopping delightful and effortless for our customers. Our goal is to understand what customers are looking for in whatever language happens to be their choice at the moment and help them find what they need in Amazon's vast catalog of billions of products. We are seeking an Applied Scientist to lead a new, greenfield initiative that shapes the arc of invention with Machine Learning and Large Language Models. Your deliverables will directly impact executive leadership team goals and shape the future of shopping experiences with Alexa. We’re working to improve shopping on Amazon using the conversational capabilities of LLMs, and are searching for pioneers who are passionate about technology, innovation, and customer experience, and are ready to make a lasting impact on the industry. You'll be working with talented scientists, engineers, across the breadth of Amazon Shopping and AGI to innovate on behalf of our customers. If you're fired up about being part of a dynamic, driven team, then this is your moment to join us on this exciting journey!
Data Scientist II, Alexa Communications Experience Analytics, Alexa Communications
US, WA, Seattle
The vision for Alexa is to be the world’s best personal assistant. Such an assistant will play a vital role in managing the communication lives of customers, from drafting communications to coordinating with people on behalf of customers. At Alexa Communications, we’re leveraging Generative AI to bring this vision to life. If you’re passionate about building magical experiences for customers, while solving hard, complex technical problems, then this role is for you. You will operate at the intersection of large language models, real time communications, voice and graphical user interfaces, and mixed reality to deliver cutting-edge features for end users. Come join us to invent the future of how millions of customers will communicate with and through their virtual AI assistants. Key job responsibilities The Comms Experience Insights (CXI) team is looking for an experienced, self-driven, analytical, and strategic Data Scientist II. We are looking for an individual who is passionate about tying together huge amounts of data to answer complex stakeholder questions. You should have deep expertise in translating data into meaningful insights through collaboration with Data Engineers and Business Analysts. You should also have extensive experience in model fitting and explaining how the insights derived from those models impact a business. In this role, you will take data curated by a dedicated team of Data Engineers to conduct deep statistical analysis on usage trends. The right candidate will possess excellent business and communication skills, be able to work with business owners to develop and define key business questions, and be able to collaborate with Data Engineers and Business Analysts to analyze data that will answer those questions. The right candidate should have a solid understanding of how to curate the right datasets that can be used to train data models, and the desire to learn and implement new technologies and services to further a scalable, self-service model.
Data Scientist, AWS Gen AI Innovation Center
US, VA, Arlington
Are you looking to work at the forefront of Machine Learning and AI? Would you be excited to apply cutting edge Generative AI algorithms to solve real world problems with significant impact? The Generative AI Innovation Center at AWS is a new strategic team that helps AWS customers implement Generative AI solutions and realize transformational business opportunities. This is a team of strategists, data scientists, engineers, and solution architects working step-by-step with customers to build bespoke solutions that harness the power of generative AI. The team helps customers imagine and scope the use cases that will create the greatest value for their businesses, select and train and fine tune the right models, define paths to navigate technical or business challenges, develop proof-of-concepts, and make plans for launching solutions at scale. The GenAI Innovation Center team provides guidance on best practices for applying generative AI responsibly and cost efficiently. You will work directly with customers and innovate in a fast-paced organization that contributes to game-changing projects and technologies. You will design and run experiments, research new algorithms, and find new ways of optimizing risk, profitability, and customer experience. We’re looking for Data Scientists capable of using GenAI and other techniques to design, evangelize, and implement state-of-the-art solutions for never-before-solved problems. Key job responsibilities As an Data Scientist, you will * Collaborate with AI/ML scientists and architects to Research, design, develop, and evaluate cutting-edge generative AI algorithms to address real-world challenges * Interact with customers directly to understand the business problem, help and aid them in implementation of generative AI solutions, deliver briefing and deep dive sessions to customers and guide customer on adoption patterns and paths to production * Create and deliver best practice recommendations, tutorials, blog posts, sample code, and presentations adapted to technical, business, and executive stakeholder * Provide customer and market feedback to Product and Engineering teams to help define product direction About the team About AWS Diverse Experiences AWS values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why AWS? Amazon Web Services (AWS) is the world’s most comprehensive and broadly adopted cloud platform. We pioneered cloud computing and never stopped innovating — that’s why customers from the most successful startups to Global 500 companies trust our robust suite of products and services to power their businesses. Inclusive Team Culture Here at AWS, it’s in our nature to learn and be curious. Our employee-led affinity groups foster a culture of inclusion that empower us to be proud of our differences. Ongoing events and learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences, inspire us to never stop embracing our uniqueness. Mentorship & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, mentorship and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve in the cloud
View from space of a connected network around planet Earth representing the Internet of Things.
Have feedback?
Let us know by taking this short survey
Get started
Amazon.com | Conditions of Use | Privacy | © 1996-2024 Amazon.com, Inc. or its affiliates
Follow Us