2023 EACL Main - Benchmark Results for Question Answering Datasets

Sep 2, 2026

Benchmark results compare multiple question answering models across several datasets, reporting retrieval metrics such as R-1, R-2, R-L, and aggregated R-Lsum. Training, validation, and test splits are listed for WikiCQA, GooAQ-S, and ELI5, showing differing dataset scales. Model variants including QGb, QGours, and enhanced combinations are evaluated with fluency and correctness-related measures, and separate tables summarize performance for different prompt or answer types (e.g., QAb vs QAs).

LanguageEnglish

CategoryResearch & Report

FormatPDF

Pages13

Reading Time33 min

AccessFree to download

Datasets

WikiCQA

GooAQ-S

ELI5

Model Evaluation Metrics

Fluency and Relevance

Correctness Score

Results by Dataset

R-1 R-2 R-L R-Lsum

| Datasets | Train | Val | Test | | --- | --- | --- | --- | | WikiCQA | 16,162 | 2,020 | 2,020 | | GooAQ-S | 200,000 | 500 | 498 | | ELI5 | 272,634 | 9,812 | 24,512 | | Title: How to Prepare a Healthy Meal for Your Pet Dog. Summary: To prepare a healthy meal for your dog, choose lean meat with the bones and fat removed, like chicken or beef | | --- | | Q: What can I feed my dog if I have run out of dog food?
A: In the short term, any bland human food such as chicken or white fish with rice or pasta is just fine . . .
Q: How much homemade dog food do you feed your dog?
A: Great question because it highlights one of the problems of feeding home prepared foods . . .
Q: What should I not feed my dog?
A: There are many human foods that are toxic to dogs. Top of the list of foods NOT to give are . . . | | Dataset Model | | R-1 | R-2 | R-L | R-Lsum | | --- | --- | --- | --- | --- | --- | | WikiCQA | QGb QGours | 48.39 | 26.84 | 46.08 | 46.16 | | | | 49.22 | 27.79 | 46.98 | 47.08 | | GooAQ-S | QGb QGours | 44.26 | 19.73 | 41.11 | 41.08 | | | | 45.26 | 20.69 | 42.20 | 42.06 | | ELI5 | QGb QGours | 28.62 | 10.10 | 25.93 | 26.23 | | | | 29.15 | 10.36 | 26.40 | 26.69 | | Model | fluency
score | % | relevance | | correctness
score % | | --- | --- | --- | --- | --- | --- | | | | | score | % | | | QGb | 4.83 | 8.3 | 3.84 | 19.0 | 3.49 19.3 | | QGours | 4.83 | 7.3 | 3.89 | 24.7 | 3.58 23.3 | | Dataset | Model | R-1 | R-2 | R-L | R-Lsum | | --- | --- | --- | --- | --- | --- | | WikiCQA | QAb | 18.41 | 5.53 | 14.72 | 15.98 | | | QAs | 24.10 | 7.07 | 18.02 | 20.31 | | GooAQ-S | QAb | 18.58 | 5.80 | 14.77 | 15.66 | | | QAs | 21.93 | 6.12 | 16.62 | 18.35 | | ELI5 | QAb | 12.38 | 2.19 | 8.98 | 10.63 | | | QAs | 13.97 | 2.29 | 10.00 | 11.77 | | Dataset | Model | R-1 | R-2 | R-L | R-Lsum | | --- | --- | --- | --- | --- | --- | | WikiCQA | QAb+f | 27.16 | 7.32 | 19.85 | 23.49 | | | QAs+f | 28.44 | 8.14 | 20.56 | 24.77 | | GooAQ-S | QAb+f | 27.72 | 7.68 | 20.08 | 23.64 | | | QAs+f | 28.44 | 7.73 | 20.17 | 24.12 | | ELI5 | QAb+f | 22.06 | 3.93 | 13.93 | 19.72 | | | QAs+f | 23.37 | 4.27 | 14.53 | 20.88 | | Models | R-1 | R-2 | R-L | R-LSum | | --- | --- | --- | --- | --- | | QGb | 48.39 | 26.84 | 46.08 | 46.16 | | QGb +CLs | 48.67 | 27.34 | 46.43 | 46.46 | | QGb +CLt | 48.72 | 27.26 | 46.33 | 46.41 | | QGb +CLs +AR | 49.26 | 27.77 | 46.82 | 46.88 | | QGb +CLt +AR | 49.22 | 27.79 | 46.98 | 47.08 | | A The most recent statistics show that around 2 .5% of small businesses are audited by the irs.
QB Are small businesses audited by the irs?
QG How many small businesses are audited by theirs?
QR What percentage of small businesses are audited? | | --- | | A Try using poster putty to secure the images in your collage. if that doesn’t work, you may have to use nails.
QB How do i put pictures in a collage?
QG How do i make a collage without nails?
QR How can i make a collage on a wall with a textured surface? | | (a) Good examples. | | A Foods such as popsicles, hard candy, and gelatin can be eaten on a clear liquid diet.
QB what can you eat on a clear liquid diet?
QG What can I eat to lose weight?
QR What food can be included on a clear liquid diet? | | A You’ll have to remove the door and sand, prime, and repaint it.
QB What do i have to do to make my bedroom look nice?
QG How do i fix a broken door?
QR What do i do if my door is sticking to the weather strip? |

FAQs

Which datasets are used for the question answering benchmark and how are they split?

The document reports WikiCQA, GooAQ-S, and ELI5 with Train, Val, and Test counts. Each dataset uses its own scale across these splits.

What evaluation metrics are reported for model performance?

Results include R-1, R-2, R-L, and R-Lsum. Additional tables also report fluency-related and correctness-related scores.

Which model variants are compared in the final results table?

The document compares QGb and several enhanced variants such as QGb +CLs, QGb +CLt, and versions adding AR. These variants are listed with their corresponding R-1, R-2, R-L, and R-Lsum scores.

Sep 2, 2026

Upload by Nguyễn Văn Học