Exploratory Data Analysis
Exploratory data analysis is the practice of summarizing and visualizing a dataset to understand its structure, spot patterns, detect anomalies, and form hypotheses before formal modeling or testing. It answers the question: what is in this data?
itData engineering and analytics | OpenSkills.info
Course pathWalk it in order
Look it upDip in anytime
Go furtherLeaves this page
Intro
Exploratory Data Analysis
Exploratory data analysis (EDA) is the first thing you do with data. Before fitting models, testing hypotheses, or building dashboards, you look at what you have. EDA reveals the shape of the data, the relationships between variables, the problems that need cleaning, and the questions worth pursuing.
Why EDA matters
Jumping to modeling without exploration leads to predictable failures:
- You fit a linear model to a non-linear relationship.
- You miss a column that is 40% null.
- You divide by a variable that contains zeros.
- You treat a categorical code as a continuous number.
- You build on an outlier that is a data entry error.
EDA catches these problems cheaply. A histogram takes seconds; debugging a broken model takes hours.
The EDA mindset
Continue the course
This section is part of the paid course.
See pricing to subscribe, or log in if you already have access.
Where this skill leads
Relevant careers
See how this topic contributes to broader role-level skill maps.
Sources
- https://www.itl.nist.gov/div898/handbook/eda/eda.htm
Supports
- EDA philosophy and Tukey's foundational approach
- Graphical techniques for distribution analysis
- Quantitative summaries (location, spread, shape)
- Outlier detection methods (IQR, visual)
- Distribution shape identification
- Multicollinearity detection through correlation
- https://r4ds.hadley.nz/eda
Supports
- Question-driven EDA workflow
- Variation and covariation as organizing concepts
- Non-linear relationship discovery
- Bimodal distribution investigation
- https://pandas.pydata.org/docs/getting_started/intro_tutorials/06_calculate_statistics.html
Supports
- Computing summary statistics with pandas
- GroupBy aggregation for comparative analysis
- Programmatic data inspection methods
- https://seaborn.pydata.org/tutorial.html
Supports
- Distribution plots (histplot, kdeplot, boxplot)
- Relational plots (scatterplot, lineplot)
- Categorical plots (countplot, boxplot, violinplot)
- Pair plots and faceted grids
- https://www.khanacademy.org/math/statistics-probability
Supports
- Descriptive statistics (mean, median, standard deviation)
- Distribution shapes (normal, skewed, bimodal)
- Correlation and scatter plot interpretation
- Quartiles and interquartile range