Amirhossein Razavi

Amirhossein Razavi

Second-year MASc Student at Toronto Metropolitan University

Dataset Generation and Evaluation for Natural Question Generation

Supervisor: Prof. An
Type: Capstone Project, Natural Language Processing (NLP), Natural Question Generation (NQG)

Overview

This capstone project focuses on developing and evaluating a novel dataset for natural question generation (NQG). Aimed at bridging the gap in sentence-based natural question answering datasets, this project utilizes state-of-the-art NLP models, including BART and T5, to generate and validate a dataset conducive to practical NQG applications.

Methodology

  1. Dataset Creation: Utilizing the Google Natural Questions (NQ) corpus, refining it to construct a more targeted dataset.
  2. Data Parsing and Coreference Resolution: Implementing sophisticated data parsing techniques and coreference resolution using AllenNLP’s library.
  3. Question Generation: Employing sequence-to-sequence models, focusing on generating questions using BART and T5 algorithms.

Evaluation

  • Automatic Evaluation: Using metrics like BLEU, METEOR, and ROUGE.
  • Test Runs: Conducting tests on different datasets, like the Car Manual Dataset.
  • Human Evaluation: Engaging human evaluators for assessing the naturalness and relevance of questions.

Key Findings

  • Development of the Coreference Resolved Sentence-based Natural Question Dataset (CRSNQD).
  • BART and T5 models showed promising results in both automatic and human evaluations.
  • Highlighted the need for improved automatic evaluation methods for NQG.

Personal Contributions

  • Developed Coref Resolved SNQD.
  • Comprehensive evaluations of BART and T5 models.
  • Insights into text processing and model training in NLP.

Conclusion

The project represents a significant step in NQG, offering a robust dataset and insights into model evaluations, paving the way for future research in NLP.

Source Code

The complete implementation is available on GitHub: Dataset-Generation-and-Evaluation-for-Natural-Question-Generation

Project Report

View the detailed project report below (if the file is not showing up, please refresh the page or download the report here):

Project Slides

View the slides created for this project below (if the file is not showing up, please refresh the page or download the slides here):