MY ONE BLOGby Manishdai
Data & Analytics

What is Data Cleansing?

Quick answer

Finding and correcting errors, duplicates, and inconsistencies in data so it is accurate and usable for analysis.

Data Cleansing, explained

Data cleansing, also called data cleaning, deals with common problems such as missing values, duplicate records, misspellings, inconsistent formats, and impossible values like negative ages. It can be done manually in spreadsheets, with scripts, or with dedicated data quality tools, and it is often a regular step in a data pipeline.

For example, a gym preparing a mailing list might find the same member listed three times with slightly different spellings, phone numbers in several formats, and some email addresses missing the at sign. Cleansing would merge the duplicates, standardize the formats, and flag invalid entries. Analysts often spend a large share of their time on this work. Fixing problems at the source, such as adding validation to sign-up forms, prevents many issues from recurring.

Related terms

Data Pipeline

An automated series of steps that moves data from one or more sources, transforms it, and delivers it to a destination for use.

ETL

Extract, Transform, Load: a process that pulls data from sources, cleans and reshapes it, and loads it into a destination such as a data warehouse.

Regular Expression

A compact pattern language used to search, match, and manipulate text, such as finding email addresses or validating phone numbers.

Guides that mention Data Cleansing