'Dive into Deep Learning' book cover
Dive into Deep Learning, an open source, interactive book provided in a unique form factor that integrates text, mathematics and code, recently added a new chapter on attention mechanisms.

Amazon team adds key programming frameworks to Dive into Deep Learning book

With PyTorch and TensorFlow incorporated, the authors hope to gain a wider audience.

Over the past few years, a team of Amazon scientists has been developing a book that is gaining popularity with students and developers attracted to the booming field of deep learning, a subset of machine learning focused on large-scale artificial neural networks. Called Dive into Deep Learning, the book arrives in a unique form factor, integrating text, mathematics, and runnable code. Drafted entirely through Jupyter notebooks, the book is a fully open source living document, with each update triggering updates to the PDF, HTML, and notebook versions.

Its authors are Aston Zhang, an AWS senior applied scientist; Zachary Lipton, an AWS scientist and assistant professor of Operations Research and Machine Learning at Carnegie Mellon University; Mu Li, AWS principal scientist; and Alex Smola, AWS vice president and distinguished scientist.

Recently the authors added two programming frameworks to their book: PyTorch and TensorFlow. That gives the book—originally written for MXNet—even broader appeal within the open-source machine-learning community of students, developers, and scientists. The book also is incorporated into Amazon Machine Learning University courseware.

Amazon Science spoke to the authors previously about their book, and we recently reconnected with them to learn about the significance of the new frameworks they’ve added to their book.

Q. What’s the significance of adding PyTorch and TensorFlow implementations to Dive into Deep Learning?

Mu Li: The book is designed to teach people different algorithms used in machine learning. A big asset of the book is the fact we provide all the coding information. Originally, we used MXNet because it’s a major interface and easy to learn. But then we started getting a lot of requests for PyTorch and TensorFlow implementations. So, we decided to add them to the book.

Aston Zhang: Another factor is that for machine learning practitioners, it’s not enough to know just one framework. That’s because a researcher may propose a new model or algorithm and provide implementation in only one framework. So, if you don’t know that framework, you can’t work with the model. Dive into Deep Learning now provides a way to address those different implementations. It fixes a pain point for our readers.

Zachary Lipton: Like any good product, you have to pay attention to what people are doing. And the audience available for a book that's only available in one framework is somewhat limited. Already, a great team from IIT Roorkee asked us if they could translate the code portions of our book, yielding a PyTorch version, and we gave it our blessing. We knew that a massive audience of students and practitioners would be excited for the PyTorch and TensorFlow versions.

Q. How does the change make the book better?

Alex Smola: The book is basically a collection of Jupyter notebooks – you can read the book in your web browser and run every code example in real time. Because we support multiple frameworks, we can have multiple code paths within each notebook, so you can compare both the implementations, and the results that they give side by side. That’s very powerful as a teaching tool.

Mu Li: We feel that by adding PyTorch and TensorFlow to Dive into Deep Learning, we’ve made it the best textbook to learn about and execute machine learning and deep learning. It’s a textbook, but also teaches you how to implement the code. Another thing is that some people already using PyTorch want to systematically learn deep learning. Now they can run different algorithms from scratch and learn how to do that in different frameworks.

Zachary Lipton: Nobody can survive as a professional in machine learning without having the skills to work with multiple frameworks. You might learn in MXNet or TensorFlow, but then switch jobs, and need to rapidly port those skills over to a place that uses PyTorch when you’re not familiar with it. In general, it’s important for people to learn several languages.

Q. Is any one of the frameworks superior to the others?

Alex Smola: Each of them has some advantages over the other and given the state of the open-source landscape, those advantages constantly shift. They’re all competing with each other for which is the fastest, most usable, has the best data loaders and so on. At one point, people argued that philosophy was best written in German, and music best written in Italian. If you want to have the widest audience, you don’t want to limit yourself to one approach to doing things.

Aston Zhang: We’re not asking our readers to use just one framework. We provide three implementations. Readers can click on each framework, learn how it works, and decide what works best for them. If you’re a new user, you can see the subtle differences between the three and can compare their speed. Also, we separate text and code—the text is framework-neutral, but in the code book we ask people to contribute material. We’ve had people from Google, Alibaba, IIT students and others add material. For the first few chapters, Anirudh Dagar and Yuan Tang have contributed most of the PyTorch and TensorFlow adaptations.. Many others have also helped with the adaptations to these frameworks.

Zachary Lipton: The book is starting to be useful as a Rosetta stone of sorts to allow practitioners to see what the best strategy is to solve the same problem in multiple frameworks— MXNet, PyTorch, TensorFlow—without having to chase down incompatible and idiosyncratic variants on GitHub.

Q. Was it challenging to add the different frameworks to the book?

Mu Li: Yes! PyTorch and MXNet are similar, but TensorFlow is pretty different. Fortunately, TensorFlow 2.0 is very different from TensorFlow 1.0, and closer to MXNet.

Alex Smola: The proper tuning and refinement of the models took quite a while to ensure the implementations for TensorFlow on modern convolutional neural networks were just as good as the ones in PyTorch and MXNet. That’s due to the different ways in how the frameworks implement things. And we still need to back-port the content into Mandarin. This isn’t a trivial endeavor, because there currently isn’t great tooling available to synch the text with the source code.

Q. What has been the response to the additions to Dive into Deep Learning?

Mu Li: Very good. In the past three months, compared with the prior three, we’ve seen about a 40 percent increase in users.

Q. What motivates you to continue improving Dive into Deep Learning?

Alex Smola: We write books because we want to teach and share content. It’s also our way of saying “thanks” to the machine-learning community. The book is a key enabler for spreading knowledge about machine learning and its applications much more widely. We want to make it easy for people to come in, learn about machine learning, and then surprise us with their additions to the book.

Zach Lipton: I don't think anyone involved in the project thinks of it as a book that will someday be finished, in the traditional sense. Having everything online, we can update and add material much, much more quickly than if it were made from dead trees.

Aston Zhang: Every day we get feedback from users around the world. Their comments, suggestions, encouragement, and endorsement motivates me to continue improving our book.

Research areas

Related content

US, NY, New York
Fauna Robotics is building capable, safe, and delightful robots for everyday life, and voice is one of the most natural ways people will interact with them. Cloud speech and language models are good and getting better, but they can only work with the audio they receive, and a robot is a hard place to listen. Its microphones sit beside motors, fans, and moving joints. It speaks through its own loudspeaker while people talk over it. It moves, turns, and shares a room with several people at once. We are hiring a Principal Audio Scientist to be Fauna's technical authority on how our robots hear. You will design, prototype, and ship the hardest algorithms in the robot's audio system. You will set the audio architecture that other engineers build on, shape hardware decisions across robot generations, mentor the engineers and scientists working on audio, and be the person teams come to when the robot can't hear. Key job responsibilities - Set the long-range science roadmap and technical architecture for the robot's audio system, and serve as Fauna's primary technical authority on robot hearing - Design and implement suppression of the robot's own noise from motors, fans, and moving joints - Design and implement echo cancellation for the robot's own voice, so people can interrupt it naturally - Develop multi-microphone processing that holds up as the robot and the people around it move, including locating who is speaking so the robot can turn toward them - Make on-robot listening decisions robust to internal and external noise sources: wake word, voice activity, and whether speech is directed at the robot - Drive microphone and speaker placement, enclosure acoustics, and vibration isolation decisions with mechanical, electrical, and industrial design, backed by your own measurements - Design the robot-specific data collection and evaluation methods to validate the performance of our audio design - Present audio science and its tradeoffs to senior leadership and partner teams - Mentor scientists and engineers, raising the scientific bar for audio across the organization through design reviews, code reviews, and hiring
IN, KA, Bengaluru
Interested to build the next generation Financial systems that can handle billions of dollars in transactions? Interested to build highly scalable next generation systems that could utilize Amazon Cloud? Massive data volume + complex business rules in a highly distributed and service oriented architecture, a world class information collection and delivery challenge. Our challenge is to deliver the software systems which accurately capture, process, and report on the huge volume of financial transactions that are generated each day as millions of customers make purchases, as thousands of Vendors and Partners are paid, as inventory moves in and out of warehouses, as commissions are calculated, and as taxes are collected in hundreds of jurisdictions worldwide. Key job responsibilities • Understand the business and discover actionable insights from large volumes of data through application of machine learning, statistics or causal inference. • Analyse and extract relevant information from large amounts of Amazon’s historical transactions data to help automate and optimize key processes • Research, develop and implement novel machine learning and statistical approaches for anomaly, theft, fraud, abusive and wasteful transactions detection. • Use machine learning and analytical techniques to create scalable solutions for business problems. • Identify new areas where machine learning can be applied for solving business problems. • Partner with developers and business teams to put your models in production. • Mentor other scientists and engineers in the use of ML techniques. A day in the life • Understand the business and discover actionable insights from large volumes of data through application of machine learning, statistics or causal inference. • Analyse and extract relevant information from large amounts of Amazon’s historical transactions data to help automate and optimize key processes • Research, develop and implement novel machine learning and statistical approaches for anomaly, theft, fraud, abusive and wasteful transactions detection. • Use machine learning and analytical techniques to create scalable solutions for business problems. • Identify new areas where machine learning can be applied for solving business problems. • Partner with developers and business teams to put your models in production. • Mentor other scientists and engineers in the use of ML techniques. About the team The FinAuto TFAW(theft, fraud, abuse, waste) team is part of FGBS Org and focuses on building applications utilizing machine learning models to identify and prevent theft, fraud, abusive and wasteful(TFAW) financial transactions across Amazon. Our mission is to prevent every single TFAW transaction. As a Machine Learning Scientist in the team, you will be driving the TFAW Sciences roadmap, conduct research to develop state-of-the-art solutions through a combination of data mining, statistical and machine learning techniques, and coordinate with Engineering team to put these models into production. You will need to collaborate effectively with internal stakeholders, cross-functional teams to solve problems, create operational efficiencies, and deliver successfully against high organizational standards.
US, NY, New York
We are seeking a Robotics/AI Motor Control Scientist to develop cutting-edge machine learning algorithms for motor control systems in robots. In this role, you will focus on creating and optimizing intelligent motor control strategies to enable robots to perform complex, whole-body tasks. Your contributions will be essential in advancing robotics by enabling fluid, reliable, and safe interactions between robots and their environments. Key job responsibilities - Develop controllers that leverage reinforcement learning, imitation learning, or other advanced AI techniques to achieve natural, robust, and adaptive motor behaviors - Collaborate with multi-disciplinary teams to integrate motor control systems with robotic hardware, ensuring alignment with real-world constraints such as actuator dynamics and energy efficiency - Use simulation and real-world testing to refine and validate control algorithms - Stay updated on advancements in robotics, AI, and control systems to apply advanced techniques to robotic motion challenges - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers - Bridge research initiatives with practical engineering implementation About the team Fauna Robotics, an Amazon company, is building capable, safe, and genuinely delightful robots for everyday life. Our goal is simple: make robots people actually want to live and interact with in everyday human spaces. We believe that future won’t arrive until building for robotics becomes far more accessible. Today, too much effort is spent reinventing the fundamentals. We’re changing that by developing tightly integrated hardware and software systems that make it faster, safer, and more intuitive to create real-world robotic products. Our work spans the full stack: mechanical design, control systems, dynamic modeling, and intelligent software. The focus is not just functionality, but experience. We’re building robots that feel responsive, expressive, and genuinely useful. At Fauna, you’ll work at the frontier of this space, helping define how robots move, manipulate, and interact with people in natural environments. It’s an opportunity to solve hard problems across hardware and software with a team focused on making robotics accessible and joyful to build. If you care about making robotics real for everyone and building systems that are as delightful as they are capable, we’re interested in hearing from you. an opportunity to solve hard problems across hardware and software with a team focused on making robotics accessible and joyful to build. If you care about making robotics real for everyone and building systems that are as delightful as they are capable, we’re interested in hearing from you.
US, CA, Sunnyvale
Amazon's Artificial General Intelligence (AGI) organization is seeking an Applied Scientist III to advance the science of Responsible AI evaluation for large language models and generative AI. In this role, you will lead the design and development of rigorous evaluation methods, benchmarks, and metrics that measure the safety, fairness, robustness, and trustworthiness of frontier models. You will work with large-scale datasets, modern deep learning frameworks, and world-class scientists and engineers to turn research into evaluation systems that shape model launch decisions at Amazon scale. Key job responsibilities - Lead the design and implementation of evaluation frameworks, benchmarks, and metrics for responsible AI, including safety, fairness, robustness, and harmful content. - Build scalable automated evaluation pipelines for large language models, including model-based and human-in-the-loop evaluation. - Partner with pretraining, post-training, and product teams to translate evaluation results into model improvements and launch decisions. - Conduct rigorous experimentation and statistical analysis, and publish research at top venues. - Mentor junior scientists and help raise the scientific bar of the team. - Champion responsible AI practices across the model development lifecycle. About the team The AGI Responsible AI (RAI) team builds the science and systems that make Amazon's large language models safe, fair, and trustworthy. We work on problems spanning safety evaluation, content moderation, watermarking, bias mitigation, and alignment. Our team values scientific rigor, customer obsession, and rapid iteration, and we collaborate closely with pretraining, post-training, and product teams across AGI.
CH, Zurich
RIVR, an Amazon company, is building Physical AI by deploying autonomous robots for real-world doorstep delivery. Operating daily in diverse urban environments, RIVR's robots continuously learn from and navigate the millions of scenarios encountered during deliveries. By owning the full stack from software to hardware, RIVR is purpose-built for safety, reliability, and the customer from day one. Reinforcement learning is transforming our robotic intelligence, enabling autonomous behavior without human guidance. We are seeking a Senior AI Engineer with deep expertise in reinforcement learning and deep learning, including supervised and self-supervised learning with a focus on dexterous manipulation. Your role will involve leveraging both simulated and real-world data to address practical challenges in dynamic grasping, contact-rich manipulation, and object interaction. If you are passionate about advancing AI and developing innovative solutions, join us in shaping the future of intelligent robotics. Key job responsibilities Develop cutting-edge reinforcement learning algorithms to enable robust, contact-rich dexterous manipulation, translating vision, depth, tactile, and proprioceptive sensor input into precise end-effector and joint-level motor commands. Design, test, and refine algorithms to solve complex real-world manipulation challenges, such as handling diverse package form factors, dynamic hand-offs, and operating door handles or latches. Collaborate with the foundation model team to innovate methods that leverage both simulated and real-world data.
US, WA, Seattle
Do you want to join an innovative team of scientists who use machine learning and statistical techniques to help Amazon provide the best customer experience by preventing eCommerce fraud? Are you excited by the prospect of analyzing and modeling terabytes of data and creating state-of-the-art algorithms to solve real world problems? Do you like to own end-to-end business problems/metrics and directly impact the profitability of the company? Do you enjoy collaborating in a diverse team environment? If yes, then you may be a great fit to join the Amazon Selling Partner Trust & Store Integrity Science Team. We are looking for a talented scientist who is passionate to build advanced machine learning systems that help manage the safety of millions of transactions every day and scale up our operation with automation. Key job responsibilities Innovate with the latest GenAI/LLM/VLM technology to build highly automated solutions for efficient fraud detection, risk evaluation and automated operations Design, develop and deploy end-to-end advance machine learning solutions with vision anf GenAi technologies in the Amazon production environment to create impactful business value Learn, explore and experiment with the latest machine learning advancements to create the best customer experience A day in the life You will be working within a dynamic, diverse, and supportive group of scientists who share your passion for innovation and excellence. You'll be working closely with business partners and engineering teams to create end-to-end scalable machine learning solutions that address real-world problems. You will build scalable, efficient, and automated processes for large-scale data analyses, model development, model validation, and model implementation. You will also be providing clear and compelling reports for your solutions and contributing to the ongoing innovation and knowledge-sharing that are central to the team's success.
US, NY, New York
We are seeking an Applied Scientist to contribute to research and development of novel security validation and monitoring techniques for AI systems at scale. You will own and contribute to four critical work-streams: 1. Real-Time Agent Monitoring Design and implement scientific approaches for continuous behavioral analysis of AI agents in production—detecting anomalous actions, prompt injection exploitation, and policy violations in real time. 2. Protection & Automated Remediation Invent and deliver novel protection technologies and automated remediation techniques building on research in security, cryptography, privacy, automated reasoning, and others domains, to enable safe and secure agentic AI models and AI applications. 3. AI Application and Capabilities Validation Invent and deliver scalable methodologies for security testing of AI applications and AI capabilities (e.g. MCP, skills), including adversarial robustness evaluation, safety guardrail bypass detection, tool-use authorization boundaries, and trust boundary verification. 4. AI Asset Discovery & Inventory Research and build scalable techniques to automatically discover, identify, and catalog all AI-enabled applications, services, and capabilities across the company—maintaining a comprehensive, continuously updated database of AI assets. Key job responsibilities Invent • Identify and frame new research challenges in AI security where problems are ill-defined and require novel scientific paradigms at the product level. • Contribute to the team's scientific agenda for agent monitoring, protection, remediation, validation research, and AI asset discovery. • Publish research results at peer-reviewed internal and external venues (e.g., USENIX Security, ACM CCS, IEEE S&P, NeurIPS, ICML security workshops, ICSE, PETS) when appropriate. • Articulate key scientific challenges of current and future AI security threats and deliver novel research to address them. • Design, implementation, and successful delivery of scientifically complex security solutions into production—both brand new systems and evolutions of existing ones. • Write significant portions of critical-path code for detection models, validation engines, protection technologies, and asset discovery / classification systems. • Assess and select appropriate technologies (e.g., data protection, private inference, graph-based anomaly detection, NLP-based service classification, code/traffic analysis for AI fingerprinting) for production systems. • Use best practices in scientific methodology and software engineering across the team; provide insightful peer reviews of code, design, and architecture artifacts. • Deliver solutions that are inventive, maintainable, scalable, and extensible. Influence • Autonomously drive discussions with security engineers, software engineers, product managers, and scientist peers across multiple teams. • Build consensus on larger cross-team security initiatives and factor complex efforts into independent workstreams. • Identify and resolve endemic problems, including areas where current security tooling limits innovation of partner teams. • Contribute to the broader internal and external scientific communities as a subject matter expert in AI security. About the team Diverse Experiences Amazon Security values diverse experiences. Even if you do not meet all of the qualifications and skills listed in the job description, we encourage candidates to apply. If your career is just starting, hasn’t followed a traditional path, or includes alternative experiences, don’t let it stop you from applying. Why Amazon Security? At Amazon, security is central to maintaining customer trust and delivering delightful customer experiences. Our organization is responsible for creating and maintaining a high bar for security across all of Amazon’s products and services. We offer talented security professionals the chance to accelerate their careers with opportunities to build experience in a wide variety of areas including cloud, devices, retail, entertainment, healthcare, operations, and physical stores. Inclusive Team Culture In Amazon Security, it’s in our nature to learn and be curious. Ongoing DEI events and learning experiences inspire us to continue learning and to embrace our uniqueness. Addressing the toughest security challenges requires that we seek out and celebrate a diversity of ideas, perspectives, and voices. Training & Career Growth We’re continuously raising our performance bar as we strive to become Earth’s Best Employer. That’s why you’ll find endless knowledge-sharing, training, and other career-advancing resources here to help you develop into a better-rounded professional. Work/Life Balance We value work-life harmony. Achieving success at work should never come at the expense of sacrifices at home, which is why flexible work hours and arrangements are part of our culture. When we feel supported in the workplace and at home, there’s nothing we can’t achieve.
IN, KA, Bangalore
Amazon’s Last Mile Team is looking for a passionate individual with strong optimization and analytical skills to join its Last Mile Science team in the endeavor of designing and improving the most complex planning of delivery network in the world. Last Mile builds global solutions that enable Amazon to attract an elastic supply of drivers, companies, and assets needed to deliver Amazon's and other shippers' volumes at the lowest cost and with the best customer delivery experience. Last Mile Science team owns the core decision models in the space of jurisdiction planning, delivery channel and modes network design, capacity planning for on the road and at delivery stations, routing inputs estimation and optimization. Our research has direct impact on customer experience, driver and station associate experience, Delivery Service Partner (DSP)’s success and the sustainable growth of Amazon. Optimizing the last mile delivery requires deep understanding of transportation, supply chain management, pricing strategies and forecasting. Only through innovative and strategic thinking, we will make the right capital investments in technology, assets and infrastructures that allows for long-term success. Our team members have an opportunity to be on the forefront of supply chain thought leadership by working on some of the most difficult problems in the industry with some of the best product managers, scientists, and software engineers in the industry. Key job responsibilities Candidates will be responsible for developing solutions to better manage and optimize delivery capacity in the last mile network. The successful candidate should have solid research experience in one or more technical areas of Operations Research or Machine Learning. These positions will focus on identifying and analyzing opportunities to improve existing algorithms and also on optimizing the system policies across the management of external delivery service providers and internal planning strategies. They require superior logical thinkers who are able to quickly approach large ambiguous problems, turn high-level business requirements into mathematical models, identify the right solution approach, and contribute to the software development for production systems. To support their proposals, candidates should be able to independently mine and analyze data, and be able to use any necessary programming and statistical analysis software to do so. Successful candidates must thrive in fast-paced environments, which encourage collaborative and creative problem solving, be able to measure and estimate risks, constructively critique peer research, and align research focuses with the Amazon's strategic needs. As a senior scientist, you will also help coach/mentor junior scientists in the team.
US, NY, New York
We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks. Your work will play a critical role in shaping how policies are validated and how high-quality datasets are generated to accelerate system performance. You will operate at the intersection of robotics, machine learning, and human-in-the-loop systems, building the infrastructure and methodologies that connect teleoperation, evaluation, and learning. This includes developing evaluation policies, defining task structures, and contributing to operator-facing interfaces that enable scalable and reliable data collection. The ideal candidate is highly experimental, systems-oriented, and comfortable working across software, robotics, and data pipelines, with a strong focus on turning ambiguous capability goals into measurable and actionable evaluation systems. Key job responsibilities - Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios - Develop task definitions, success criteria, and benchmarking methodologies that enable consistent and reproducible evaluation of policies - Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs - Build and iterate on teleoperation workflows and operator interfaces to support efficient, reliable, and scalable data collection - Analyze evaluation results and collected data to identify performance gaps, failure modes, and opportunities for targeted data collection - Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the broader robotics stack - Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to continuously improve internal approaches - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers
US, NY, New York
We are seeking an Applied Scientist to lead the development of evaluation frameworks and data collection protocols for robotic capabilities. In this role, you will focus on designing how we measure, stress-test, and improve robot behavior across a wide range of real-world tasks. Your work will play a critical role in shaping how policies are validated and how high-quality datasets are generated to accelerate system performance. You will operate at the intersection of robotics, machine learning, and human-in-the-loop systems, building the infrastructure and methodologies that connect teleoperation, evaluation, and learning. This includes developing evaluation policies, defining task structures, and contributing to operator-facing interfaces that enable scalable and reliable data collection. The ideal candidate is highly experimental, systems-oriented, and comfortable working across software, robotics, and data pipelines, with a strong focus on turning ambiguous capability goals into measurable and actionable evaluation systems. Key job responsibilities - Design and implement evaluation frameworks to measure robot capabilities across structured tasks, edge cases, and real-world scenarios - Develop task definitions, success criteria, and benchmarking methodologies that enable consistent and reproducible evaluation of policies - Create and refine data collection protocols that generate high-quality, task-relevant datasets aligned with model development needs - Build and iterate on teleoperation workflows and operator interfaces to support efficient, reliable, and scalable data collection - Analyze evaluation results and collected data to identify performance gaps, failure modes, and opportunities for targeted data collection - Collaborate with engineering teams to integrate evaluation tooling, logging systems, and data pipelines into the broader robotics stack - Stay current with advances in robotics, evaluation methodologies, and human-in-the-loop learning to continuously improve internal approaches - Lead technical projects from conception through production deployment - Mentor junior scientists and engineers