In Research, the emphasis is on the report writing because a good report comprehensively explains all the stages with the relevant outcome and valuable way forwards.
Reports are the product of data gathered either from secondary or primary sources and it is, therefore, very important for the data to be authentic, reliable, and up to date. In order to ensure the reliability of data, it has to be processed for omitting any error or mistake. And before we can work with our data, we need to make sure it’s valid, accurate, and reliable.
In the age of Big Data, companies may spend just as much or more on maintaining the health and cleaning their data as they spend on collecting it in the first place. Consider the issues that can stem from missing or wrong values, duplicates, and typos. The validity, accuracy, and reliability of your calculations depend on your ability to keep your data up-to-date, this is also evident from Ace Research’s projects.
To prepare data for later analysis, it is important to have a clean data table. Depending on the origin of the data, you may need to do some of the following steps to ensure that the data are as complete and consistent as possible.
- Assign unique code to your fields
Unique codes are very useful while sorting and cleaning data because at any stage if trouble arises you can sort out the data from your database with the help of unique codes already assigned to the data set.
- Maintain separate sheets if you are working on a huge data set
Often the data is very large and you cannot work on the whole data set at the same time, so it is preferred to maintain separate files for each change you make. This helps when you refer back in case you missed anything at any step.
- Get rid of extra spaces
Extra spaces are painfully difficult to spot. While you may somehow spot the extra spaces between words or numbers, trailing spaces are not even visible. Here is a neat way to get rid of these extra spaces
Excel TRIM function takes the cell reference (or text) as the input. It removes leading and trailing spaces as well as the additional spaces between words (except single spaces).
- Select and treat all blank cells
Blank cells can create havoc if not treated beforehand. We often face issues with blank cells in a data set that is used to create reports.
You may want to fill all blank cells with ‘0’ or ‘Not Available’, or may simply want to highlight it. If there is a huge data set, doing this manually could take hours. Thankfully, there is a way you can select all the blank cells at once.
- Select the entire data set
- Press F5 (this opens the Go to dialogue box)
- Click on the Special button (at the bottom left).
- This opens the Go To Special dialogue box
- Select Blank and Click OK
This selects all the blank cells in your data set. If you want to enter 0 or Not Available in all these cells, just type it and press Control + Enter (remember if you press only enter, the value is inserted only in the active cell).
- Remove duplicates
There can be 2 things you can do with duplicate data – Highlight It or Delete It.
Highlight Duplicate Data:
Select the data and Go to Home – Conditional Formatting – Highlight Cells Rules – Duplicate Values.
Specify the formatting and all the duplicate values get highlighted.
Delete Duplicates in Data:
- Select the data and Go to Data – Remove Duplicates.
- If your data has headers, ensure that the checkbox at the top right is checked.
- Select the Column(s) from which you want to remove duplicates and click OK.
This removes duplicate values from the list.
If you want the original list intact, copy-paste the data at some other location and then do this.
- Highlight errors
There are 2 ways you can highlight Errors in Data in Excel:
Using Conditional Formatting
- Select the entire data set
- Go to Home –Conditional Formatting – New Rule
- In New Formatting Rule Dialogue Box select ‘Format Only Cells that Contain’
- In the Rule Description, select Errors from the drop-down
- Set the format and click OK. This highlights any error value in the selected dataset
Using Go To Special
- Select the entire data set
- Press F5 (this opens the Go To Dialogue box)
- Click on Special Button at the bottom left
- Select Formulas and uncheck all options except Errors
This selects all the cells that have an error in it. Now you can manually highlight these, delete it, or type anything into it.
- Change text to lower/upper/proper case
When you import data from text files, often the names or titles are not consistent. Sometimes all the text could be in lower/upper case or it could be a mix of both. You can easily make it all consistent by using these three functions:
- LOWER () – Converts all text into Lower Case.
- UPPER () – Converts all text into Upper Case.
- PROPER () – Converts all Text into Proper Case.
- Parse data using text to column
When you get data from a database or import it from a text file, it may happen that all the text is cramped in one cell. You can parse this text into multiple cells by using Text to Column functionality in Excel.
- Select the data/text you want to parse
- Go To Data –Text to Column (This opens the Text to Columns Wizard)
Step 1: Select the data type (select Delimited if your data is not equally spaced, and is separated by characters such as comma, a hyphen, dot.). Click Next
Step 2: Select Delimiter (the character that separates your data). You can select pre-defined delimiter or anything else using the other option
Step 3: Select the data format. Also, select the destination cell. If the destination cell is not selected, the current cell is overwritten.
- Spell check
Nothing lowers the credibility of your work than a spelling mistake.
Use the keyboard shortcut F7 to run a spell check for your data set in Excel.
- Delete all formatting
In my job, I used multiple databases to get the data in excel. Every database had its own data formatting. When you have all the data in place, here is how you can delete all the formatting at one go:
- Select the data set
- Go to Home – Clear –Clear Formats
Similarly, you can also clear only the comments, hyperlinks, or content.
- Use find and replace to clean data in excel
Find and replace is indispensable when it comes to data cleansing. For example, you can select and remove all zeros, change references in formulas, find and change formatting, and so on.
