Tutorials › Introduction to Statistics › Preserving Raw Data Before Cleaning

Ethics, Data Quality and Reproducible Work · Tutorial 86 of 996

Preserving Raw Data Before Cleaning

Learn to distinguish raw data from working data and protect the original values while preparing a separate copy for cleaning.

Beginner 9 min read

What You'll Learn

  • Define raw data and a working copy in a data-cleaning workflow
  • Set up separate files while keeping their relationship clear
  • Explain why overwriting raw values can make information unrecoverable
  • Work through examples of corrections on a copy, not the original
  • Identify safeguards for protecting an unchanged source dataset

Why Preserve the Original?

A dataset may need changes before it is ready for analysis. Some values may need review, formatting may need adjustment, or a documented correction may be appropriate. But changing the only copy creates a problem: if a change is mistaken, the original value may be impossible to recover. Keeping an unchanged source and making a separate working copy protects the evidence while allowing careful preparation.

Preserving raw data is not a claim that every recorded value is correct. It is a way to retain exactly what was received so that later decisions can be checked against it. The goal is to make changes to a separate copy, not to silently replace the source.

Definition: Raw data are the original data as received or recorded, retained without cleaning changes. A working copy is a separate copy on which preparation or cleaning changes may be made. The raw data preserve the source values; the working copy is used to prepare data for later analysis.

The distinction is about the role of each file, not its format. A raw dataset might be a spreadsheet, a text file, or another data file. A working copy might use the same format or a different one. What matters is that the raw source stays unchanged and the working file is clearly identified as a copy.

Set Up Two Distinct Data Files

A practical workflow starts by retaining the file as received, then making a separate copy for work. Give the files names that make their roles apparent, such as survey_source and survey_working. Avoid names that make the changed file look like the original, such as saving over the source using the same filename.

The dataset's provenance—where it came from and how it was obtained—helps explain what the raw file represents. Record enough information to identify the source and distinguish it from the copy. Keep the files in an appropriate secure location, especially if the data are sensitive. Protecting the raw copy may include limiting who can edit it or storing it where accidental changes are less likely.

FeatureRaw dataWorking copy
PurposePreserve the values as receivedPrepare a separate dataset for subsequent work
ChangesDo not make cleaning changesMay be changed as needed
Role in checkingReference for comparison with the sourceFile to inspect and prepare
File identityClearly labeled as the source or raw fileClearly labeled as a working copy

The raw file should remain available even after a working copy has been changed. A copy is useful only if the relationship is clear: someone reviewing the work should be able to tell which file is the source and which one has been prepared.

Worked Example: Making a Working Copy Before Editing

A student receives a hypothetical spreadsheet containing a participant code and a recorded height in centimeters. The first four rows include heights of 164, 171, 158, and 169. The student notices that the file needs preparation before analysis. What should happen before any edits?

Step 1: Retain the source. Save the received file as height_raw and do not edit it. Its height values remain 164, 171, 158, and 169 cm.

Step 2: Create a separate copy. Copy the file and name the new file height_working. Check that both files exist and are distinct. At this stage, the values in the working copy should also be 164, 171, 158, and 169 cm.

Step 3: Make any later preparation changes only in the working copy. If the student edits a value during subsequent cleaning, the raw file still retains the original recorded value for comparison. No conclusion about whether a value needs changing follows simply from making the copy; that decision requires evidence and a reason.

This procedure preserves a direct reference to what was received. It does not itself establish that the measurements are accurate or that the working copy is ready for analysis.

Why Separation Matters: Changes Can Lose Information

Some transformations are reversible: if a value is converted from centimeters to millimeters using a known exact rule, the original can be recovered by applying the inverse rule. But other changes discard information. Rounding, replacing a value, or removing detail can make distinct original values look identical. Once the only copy has been changed this way, the exact original may no longer be knowable from the changed file.

A simple mathematical example shows why. Suppose a cleaning step rounds a measurement to the nearest whole number. Two different recorded values, 12.2 and 12.4, both become 12. The operation has mapped distinct inputs to the same output:

$$ 12.2 \longrightarrow 12,\qquad 12.4 \longrightarrow 12 $$

Knowing that the rounded result is 12 does not reveal whether the original was 12.2, 12.4, or another value that rounds to 12. The transformation is therefore not one-to-one, so it cannot be reversed uniquely. Retaining the raw value is what makes it possible to inspect the original rather than guess it.

This justification applies broadly: if a change can discard distinctions between possible source values, keeping an unchanged source prevents that loss from becoming permanent. A working copy allows the analyst to try a change while retaining the ability to compare it with the record received.

Worked Example: Rounding Without Losing the Source

A hypothetical data file records three times in seconds as 8.2, 8.4, and 8.7. For a particular display, a student creates rounded values to the nearest whole second. Show why the raw values should remain available.

The rounded values are 8, 8, and 9 seconds, respectively: 8.2 rounds to 8, 8.4 rounds to 8, and 8.7 rounds to 9. The first two different source values have become the same displayed value.

If the original file is preserved: the raw values remain 8.2, 8.4, and 8.7 seconds, and the working copy can contain 8, 8, and 9 for the intended display.

If the only file is overwritten: the value 8 in the changed file does not identify whether the original was 8.2 or 8.4 seconds. The specific source value cannot be recovered from the rounded value alone. Keeping both files preserves the exact records and the prepared display values.

Changes Need a Clear Basis

Preserving raw data is one part of responsible preparation, not a substitute for careful judgment. A working copy can still be changed incorrectly. Before changing a value, identify what the change is meant to do and what evidence supports it. If a value is uncertain, changing it to what seems typical may conceal uncertainty rather than resolve it. The raw value remains useful because it lets a reviewer see what was actually recorded.

A useful mental model is that each version answers a different question:

  • Raw file: What values were received or recorded?
  • Working copy: What values are being prepared for the intended analysis?

Keeping those questions separate helps prevent a preparation decision from being mistaken for an original observation. It also allows work to be retraced from the unchanged source, rather than relying on memory about what a value used to be.

Worked Example: A Possible Correction

A hypothetical attendance dataset records a session length of 480 minutes. The analyst suspects that the entry may have been intended to be 48.0 minutes, but has not yet found evidence confirming that interpretation. How should the analyst handle the raw and working data?

Raw file: retain the recorded value, 480 minutes, unchanged. This file documents what the source contained.

Working copy: do not silently replace 480 with 48.0 as though the correction were established. The analyst can flag the value for review in the working process and seek an appropriate source of evidence. If reliable evidence later supports a correction, any change belongs in the working copy, while the raw file still shows 480.

Conclusion: suspicion alone is not enough to establish the intended value. Preserving the source keeps the recorded value visible while the analyst determines what, if anything, should be changed for analysis.

A Short Procedure for Preserving Raw Data

The following sequence is a practical safeguard before cleaning begins:

1
Identify the source file.
Confirm which file was received or recorded and retain it in an appropriate location.
2
Keep the source unchanged.
Avoid editing or saving changes over the original raw file.
3
Create a separate working copy.
Make a distinct copy and give it a name that clearly identifies its role.
4
Check the separation.
Verify that both files exist and that the working copy begins with the same recorded values as the raw file.
5
Make preparation changes only in the copy.
Keep the source available for comparison whenever a value or transformation needs review.

This procedure is deliberately simple. It does not prescribe every later cleaning decision; it establishes a reliable starting point before those decisions are made.

Common Mistake: Treating “Raw” as “Correct”

A common mistake is to think, “The raw file is the original, so every value in it must be true,” or, in the opposite direction, “I know a value is wrong, so I can overwrite it.” Neither conclusion follows. Raw data describe what was received or recorded; they are not automatically a guarantee of accuracy. At the same time, an unverified suspicion does not establish the correct replacement.

The correction is to preserve the recorded value in the raw file and evaluate possible changes separately in the working copy. The raw data serve as a stable reference, not as proof that all measurements or entries are correct. Keeping that distinction protects both the original record and the quality of later analysis.

Key takeaway: Keep an unchanged raw dataset and do preparation work in a clearly labeled, separate working copy. The raw file preserves what was recorded; the working copy allows changes without erasing that evidence.

Check Your Understanding

Try these questions without looking back.

  1. In your own words, distinguish raw data from a working copy.
  2. Why can rounding a value make it impossible to recover the exact original from the rounded result?
  3. A colleague saves edits over the only copy of a dataset. What safeguard should have been used first?
  4. Does an unchanged raw value necessarily mean that the value is accurate? Explain.
  5. An analyst suspects a recorded value is wrong but has no confirming evidence. What should remain in the raw file, and where should any later change be made?