Learning-based Fusion for Data Deduplication

View Description

Download

small (250x250 max)

medium (500x500 max)

large ( > 500x500)

Full Resolution

PDF
Text

you wish to report:

Your comment:

Your Name:

Submit

Cancel

...

Rating
Permanent URL	http://hdl.lib.byu.edu/1877/2402
Title	Learning-based Fusion for Data Deduplication
BYU Creator	Dinerstein, Sabra Egbert, Parris K.
Creator	Clyde, Stephen W. Dinerstein, Jared
Keywords	data deduplication; learning-based
Description/Abstract	Rule-based deduplication utilizes expert domain knowledge to identify and remove duplicate data records. Achieving high accuracy in a rule-based system requires the creation of rules containing a good combination of discriminatory clues. Unfortunately, accurate rule-based deduplication often requires significant manual tuning of both the rules and the corresponding thresholds. This need for manual tuning reduces the efficacy of rule-based deduplication and its applicability to real-world data sets. No adequate solution exists for this problem. We propose a novel technique for rule-based deduplication. We apply individual deduplication rules, and combine the resultant match scores via learning-based information fusion. We show empirically that our fused deduplication technique achieves higher average accuracy than traditional rule-based deduplication. Further, our technique alleviates the need for manual tuning of the deduplication rules and corresponding thresholds.
Source	Jared Dinerstein, Sabra Dinerstein, Parris K. Egbert, Stephen W. Clyde. "Learning-based Fusion for Data Deduplication", In Proceedings of The Seventh International Conference on Machine Learning and Applications, pp. 66 - 71, 2008. IEEE Computer Society.;
Publisher	IEEE
Date Original	2008
Date Digital	2010-09-02
Language	English eng en
Publication Type	Articles Conference Proceedings
Type	Text
Format	application/pdf
Owning Institution	Brigham Young University
BYU College	College of Physical and Mathematical Sciences
BYU Department	Computer Science
Copyright Status	(c) 2008 Institute of Electrical and Electronics Engineers;
Copyright Use Information	http://www.lib.byu.edu/generic_copyright.html

Find results with:

Searching collections:

Home Learning-based Fusion for Data Deduplication

Learning-based Fusion for Data Deduplication

Description

Tags

Comments