End-to-end question generation to assist formative assessment design for conceptual knowledge learning

Formative assessment can be used by learning designers to evaluate a learners' comprehension, learning needs, and learning progress during a lesson, unit, or course. The general goal of a formative assessment is to collect detailed information that can be used to improve instruction and learning while learning is happening. Designing effective formative assessments for complex or technical knowledge can be difficult for a learning designer who does not have sufficient breadth or depth of expertise in the subject. The goal of this work is to provide assistance to designers in understanding the technical details of a subject and in constructing meaningful formative assessments. We propose an end-to-end solution that leverages text summarization, question generation, and context fine-tuning techniques to provide such assistance. In our solution, text summarization is applied to a text block to derive the main concept of the assessment. Question generation is then applied to both the text block and the main concept to generate a question. Human intervention is applied after the text summarization module to improve question quality. We use quantitative and qualitative measures to test various techniques in both the text summarization and question generation steps. The techniques include transformer-based solutions, sequence-to-sequence text generation, and contextualization of an NLP task. We demonstrate the solution with a use case from workforce learning. We also report findings on the effectiveness of the different approaches.