Big Data Quality Assessment Model for Unstructured Data

Ikbal Taleb, Mohamed Adel Serhani, Rachida Dssouli

Research output: Chapter in Book/Report/Conference proceedingConference contribution

17 Citations (Scopus)

Abstract

Big Data has gained an enormous momentum the past few years because of the tremendous volume of generated and processed Data from diverse application domains. Nowadays, it is estimated that 80% of all the generated data is unstructured. Evaluating the quality of Big data has been identified to be essential to guarantee data quality dimensions including for example completeness, and accuracy. Current initiatives for unstructured data quality evaluation are still under investigations. In this paper, we propose a quality evaluation model to handle quality of Unstructured Big Data (UBD). The later captures and discover first key properties of unstructured big data and its characteristics, provides some comprehensive mechanisms to sample, profile the UBD dataset and extract features and characteristics from heterogeneous data types in different formats. A Data Quality repository manage relationships between Data quality dimensions, quality Metrics, features extraction methods, mining methodologies, data types and data domains. An analysis of the samples provides a data profile of UBD. This profile is extended to a quality profile that contains the quality mapping with selected features for quality assessment. We developed an UBD quality assessment model that handles all the processes from the UBD profiling exploration to the Quality report. The model provides an initial blueprint for quality estimation of unstructured Big data. It also, states a set of quality characteristics and indicators that can be used to outline an initial data quality schema of UBD.

Original languageEnglish
Title of host publicationProceedings of the 2018 13th International Conference on Innovations in Information Technology, IIT 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages69-74
Number of pages6
ISBN (Electronic)9781538666739
DOIs
Publication statusPublished - Jan 8 2019
Event13th International Conference on Innovations in Information Technology, IIT 2018 - Al Ain, United Arab Emirates
Duration: Nov 18 2018Nov 19 2018

Publication series

NameProceedings of the 2018 13th International Conference on Innovations in Information Technology, IIT 2018

Conference

Conference13th International Conference on Innovations in Information Technology, IIT 2018
Country/TerritoryUnited Arab Emirates
CityAl Ain
Period11/18/1811/19/18

Keywords

  • Big Data
  • Data Quality
  • Quality of Unstructured Big Data
  • Unstructured Data

ASJC Scopus subject areas

  • Hardware and Architecture
  • Information Systems and Management
  • Information Systems

Fingerprint

Dive into the research topics of 'Big Data Quality Assessment Model for Unstructured Data'. Together they form a unique fingerprint.

Cite this