|
ClueWeb 09 Labeled Near-Duplicate News Articles |
This data release is a companion to the paper Duplicate News Story Detection Revisited by Omar Alonso, Dennis Fetterly, and Mark Manasse published at The Ninth Asia Information Retrieval Societies Conference (AIRS 2013) in December 2013. Last published: August 28, 2013.
This data release is a companion to the paper Duplicate News Story Detection Revisited by Omar Alonso, Dennis Fetterly, and Mark Manasse published at The Ninth Asia Information Retrieval Societies Conference (AIRS 2013) in December 2013. The package provides approximately 5.5 million document identifiers of a subset of the ClueWeb’09 “Category A English” documents that are likely to be from news sources. The package also contains two sets of human generated labels. The first set of labels is the assessment of 456 pairs of documents by the authors as near-duplicate, non-duplicate, containment, near-duplicate irrelevant, or non-duplicate irrelevant. The second set of labels is 710 labels obtained via a crowdsourcing system where the pairs of articles are labeled as near-duplicate or non-duplicate articles. Finally, the data release contains the experimental design templates used for the crowdsourced assessments.
Files
|
Status: LiveThis download is still available on microsoft.com. The downloads below will come directly from the Microsoft Download Center. |
| Files |
|---|
news-dedupe-data.zip
|
System Requirements
Operating Systems: Windows 10, Windows 7, Windows 8
- Windows 7, Windows 8, or Windows 10
Installation Instructions
- Click Download and follow the instructions.