Convert Data Types

You’ve just loaded your dataset, run df.head() and everything looks perfect — until you try to compute the average of a column and Python silently returns NaN or a crazy number like 1234.0 when you expected 12.34. The culprit is almost always the same: your data types are wrong. Strings posing as numbers, dates stored as text, booleans hiding as 1 and 0 — these sneaky type mismatches are the #1 reason your analysis quietly goes off the rails. In this lesson, you’ll learn how to convert data types for accurate analysis using pandas and NumPy, so your sums, means, and plots actually mean something.

The problem this lesson solves

When you load data from CSV files, APIs, or databases, pandas does its best to infer the correct dtype for each column — but it often guesses wrong. Here are the most common headaches you’ll face if you ignore data types:

• Numeric columns stored as strings (e.g., '123.45' instead of 123.45). Performing arithmetic on strings will either throw a TypeError or concatenate them, producing results like '12' + '34' = '1234'.
• Dates stored as text (e.g., '2024-01-15'). You can’t calculate time differences or resample by month if your dates are strings.
• Boolean fields stored as integers (0/1) — they work for some calculations but confuse visualizations and make your code less readable.
• Mixed-type columns where a few stray values like 'N/A' force the entire column to object dtype, making numeric operations fail.
• Memory bloat — incorrectly typed columns (like using int64 when float32 would do) waste RAM, especially on large datasets.

These problems don’t just cause errors; they produce results that are wrong but look plausible. That’s far more dangerous — you might trust a corrupted mean or a misaligned date range and base decisions on it. Converting data types isn’t a chore; it’s a sanity check that ensures every column is the right kind of value for the operation you want to run.

Core concept / mental model

Think of a pandas DataFrame as a spreadsheet where each column has a strict type — like a mailbox with a label. If you try to put a letter into a parcel slot, things get messy. Data type conversion is the act of moving each letter into the proper slot.

The mental model to adopt: Every column in pandas has a dtype, and every operation you run depends on that dtype being correct. If you want to add, compare, sort, or plot a column, the dtype determines how pandas interprets the values. A column of '10', '20', '30' (strings) is a list of words, not a list of numbers.

Key definitions to keep handy:

• dtype: Short for “data type” — the kind of values a column holds (e.g., int64, float64, object, datetime64, bool).
• cast / convert: Changing a column from one dtype to another, e.g., pd.to_numeric() or astype('%d').
• implicit coercion: When pandas automatically changes dtypes during operations — which is often the source of bugs.
• explicit conversion: When you deliberately specify the desired dtype — this is what we’ll practice.

Pro tip: Never assume the dtype of a column is what a data dictionary says it is. Always verify with df.dtypes before any calculation.

How it works step by step

Here’s a reliable workflow for converting data types for accurate analysis — follow it every time you load a new dataset:

1. Inspect the current dtypes with df.dtypes (or df.info() for a quick overview). Look for columns listed as object that you expect to be numeric or datetime.
2. Diagnose the problem — print a few unique values from suspicious columns to spot strings like '$100', 'N/A', or '1,200'.
3. Clean the values if needed — remove currency symbols, commas, or replace placeholders like 'unknown' with NaN.
4. Convert — use the right pandas function for the target type: - pd.to_numeric() for integers/floats - pd.to_datetime() for dates/times - astype() for booleans or when you need a simple cast
5. Validate the result — re-check df.dtypes and run a quick sanity check (e.g., compute a mean) to confirm the conversion worked.

Each step is a cause-and-effect loop: wrong data in → wrong result out. When you convert correctly, downstream operations behave as expected.

Hands-on walkthrough

Let’s put the steps into practice with a realistic scenario. Imagine you have a CSV file of sales records with columns for item, price, quantity, and date.

Step 1: Load and inspect

import pandas as pd

# Sample data — notice the price column uses '$' and a comma
sales_df = pd.DataFrame({
    'item': ['widget', 'gadget', 'gizmo'],
    'price': ['$10.50', '$25.00', '$15.75'],
    'quantity': ['3', '5', '2'],
    'date': ['2024-01-01', '2024-01-02', '2024-01-03']
})

print(sales_df.dtypes)
"""
item       object
price      object   # string with $ and comma
quantity   object   # numbers but stored as text
date       object   # dates as strings, not datetime64
"""

Step 2: Clean and convert

# Remove the currency symbol and convert price to float
sales_df['price_cleaned'] = sales_df['price'].str.replace('$', '').astype(float)

# Convert quantity to integer
sales_df['quantity'] = pd.to_numeric(sales_df['quantity'])

# Convert date to datetime64
sales_df['date'] = pd.to_datetime(sales_df['date'])

print(sales_df.dtypes)
"""
item            object
price           object
quantity         int64
date    datetime64[ns]
price_cleaned   float64
"""

Step 3: Validate with a calculation

# Now we can compute total revenue safely
sales_df['revenue'] = sales_df['price_cleaned'] * sales_df['quantity']
print(sales_df[['item', 'revenue']])
"""
     item  revenue
0  widget    31.50
1  gadget   125.00
2   gizmo    31.50
"""

If we hadn’t converted, '3' * '5' would have thrown a TypeError, or worse — '$10.50' + '$25.00' would have given a single concatenated string instead of a sum. The conversion is the difference between garbage and insight.

Handling messy data with errors parameter

# Column with non-numeric values
messy = pd.Series(['1.2', 'abc', '3.4'])

# Use errors='coerce' to turn invalid entries into NaN
converted = pd.to_numeric(messy, errors='coerce')
print(converted)
"""
0    1.2
1    NaN
2    3.4
"""

When you have a column with mixed types, errors='coerce' is your best friend — it converts what it can and sets the rest to NaN so you can decide how to handle them later.

Compare options / when to choose what

When to choose what: If you’re working with user-input data or anything that might have dirty values, always use pd.to_numeric() or pd.to_datetime() with error handling. Only use astype() on data you trust to be already clean. For memory efficiency with categorical text, consider category dtype.

Pro tip: astype() will raise an error if it encounters a value it can’t convert — that’s a feature, not a bug. Use it as a validation tool in your pipelines.

Troubleshooting & edge cases

Even with the right approach, you’ll hit snags. Here are common errors and how to fix them:

• ValueError: could not convert string to float — This means your string has characters like '$', ',', or spaces. Clean the string first with .str.replace() or use pd.to_numeric(errors='coerce') to see which values break.
• TypeError: unsupported operand type(s) for +: 'int' and 'str' — You tried to add a number to a string. Convert the string column first with pd.to_numeric().
• pd.to_datetime() throws a ParserError — Your date string doesn’t match the expected format. Pass format='%Y-%m-%d' explicitly or use errors='coerce' to avoid crashing.
• After conversion, your column is object even though you used pd.to_numeric() — This happens when some values can’t be converted and you didn’t use errors='coerce'. With coerce, the column becomes float64 (with NaNs), not object.
• astype('int') fails on a column that looks like integers — Look for NaNs — integers can’t hold NaN. Convert to float first, then handle the missing values, or use pd.Int64Dtype() (nullable integer) in pandas 1.0+.
• Time zone issues — pd.to_datetime() may produce timezone-naive datetimes. Convert to a consistent timezone with .dt.tz_localize('UTC') if needed.

Edge-case tip: Always check df.isna().sum() after a coercion — you might have created NaNs you weren’t aware of.

What you learned & what's next

You now understand the core idea behind convert data types for accurate analysis: verify your dtypes, clean the values, and explicitly cast each column to the right type before any operation. You practiced the workflow with a realistic sales dataset and learned when to use pd.to_numeric(), pd.to_datetime(), and astype().

These are the key takeaways from this lesson:

• Check dtypes first — df.dtypes and df.info() are your best friends.
• pd.to_numeric() and pd.to_datetime() are the go-to converters for messy data, with the errors parameter to handle bad values.
• astype() is for clean data — it’s strict and will raise errors if you’re wrong.
• Clean before you convert — strip symbols, handle missing values, and then cast.
• Always validate after conversion with a quick calculation or .dtypes check.

You’ve just built a solid foundation for reliable analysis. In the next lesson, you’ll learn how to filter and subset DataFrames to focus on the rows and columns that matter — something that will rely on the correctly typed data you now know how to produce.