A New Filtering Algorithm for Duplicate Document Based on Concept Analysis
Ahmad M. Hasnah
DOI : 10.3844/jcssp.2006.434.440
Journal of Computer Science
Volume 2, Issue 5
Data bases and web pages contain currently a huge number of duplicate document. It is then fundamental to have a filter which can be embedded, for instance, within an information retrieval system like a search engine in order to prohibit the redundant documents references to appear on the screen as a reply to the user's query. This filter can save the user time and increases his satisfaction. In this study, we propose a new algorithm based on concept analysis principle, which can act as a filter for duplicate document. It can be applied on a collection of documents or databases and reduce their storage spaces by eliminating redundant documents without loosing knowledge. Our experiments show that this algorithm increases the precision of the information retrieval system and improves its performance.
© 2006 Ahmad M. Hasnah. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.