Scatterplots · B05 · Sunflower Intelligence

Before you start

You will be able to

Describe a scatterplot by its direction, form, strength and unusual points, in the words of the business question.
Choose which variable to put on each axis, and explain why the correlation is the same either way.
Recognise linear, curved and clustered forms, and points that are extreme but on the pattern versus points that break it.
Handle overplotting in a large dataset with transparency, jitter and one group at a time, and reveal time order that a scatterplot hides.
Explain with Anscombe’s quartet why a correlation must always be read alongside its plot.

Review first

Charts and conditional formatting — Making a scatter chart in Excel, with its two value axes.
Descriptive statistics and regression in Excel — CORREL and the correlation coefficient, which this course takes apart and extends.
Describing distributions — Shape, centre, spread and outliers for one variable; a scatterplot does the same for two.

Fictional case A fictional coffee business founded by Ana Lim. It starts as a catering and barista-training service, opens its own cafés and an online shop for beans, and plans a canned cold brew.

The idea

Fictional case

Lantern’s two cafés, Harbourside on the waterfront and Millbrook in the office district, opened on 1 March 2027. By 31 December their managers had logged 306 days each: 612 rows, one per café per day, with the day’s high temperature and rain, the door counter’s footfall, transactions and sales at the till, hot and iced drinks made, the barista hours rostered for the 7–11 a.m. peak, the average wait at the peak, and the loyalty app’s members. Ana wants to know what goes with what: which measures rise and fall together, and which of those links she can plan around.

Plot first

Every question in this course is about two variables measured on the same units, here the same café on the same day. The first tool is a picture. A scatterplot draws one point per unit, placed by its value of one variable across the bottom (the x-axis) and its value of the other up the side (the y-axis). B03 built one in Excel as a Scatter chart; this lesson is about reading one and making a large one readable.

A correlation coefficient, which B03 calculated with CORREL and the next lesson takes apart, squeezes a whole cloud of points into one number. It is a good summary of some clouds and a misleading one of others, and only the plot tells you which kind you have. Plotting takes seconds; skipping it is how analysts end up reporting a strong relationship that is really one wrong digit, or a weak one that is really a strong curve.

Four things to read

Read every scatterplot in the same order, and say what you see in the units of the question.

Direction. Do the points rise from lower left to upper right, so that high values of one variable go with high values of the other (a positive association), or fall, so that high goes with low (a negative association)? Or neither?
Form. Do the points follow a straight band (a linear relationship), a curve, or separate clusters?
Strength. How tightly do the points hug that form? A thin band is strong; a fat cloud is weak.
Unusual points. Are there points far from the rest, and do they lie on the pattern or break it?

The log has an example of each. Plot the share of drinks that are iced against the day’s high temperature and the points rise steeply: a positive association. Plot hot drinks against the high and they fall: a negative one. Footfall and transactions form a thin straight band, with a correlation of 0.98; rain and footfall form a loose cloud with a correlation of −0.14. Five days at Harbourside sit far to the right of everything else, and in the log as the manager typed it, one day at Millbrook sits off on its own: the In depth section explains both.

Which variable goes where

Put the variable that you think helps explain or predict the other on the x-axis: the explanatory variable. Put the one you want to understand on the y-axis: the response variable. Temperature goes on x and iced share on y, because the weather is known before the day starts and the drinks follow; footfall goes on x and transactions on y, because people must come in before they buy. When neither variable comes first, as with hot and iced drinks made on the same day, either layout is fine.

The choice changes how the plot reads, not the correlation. Pearson’s r of transactions with footfall is 0.981, exactly the same as footfall with transactions, because the formula treats the two variables alike. That symmetry is worth remembering: correlation describes how two variables move together and has no idea which one you think comes first. Regression, in B06, is not symmetric, and there the choice of axis changes the answer.

In depth

Direction and strength

Direction is about the trend, not every point. In a positive association there are days with a high temperature and a modest iced share, but on average the share rises with the temperature. Strength is about scatter around the trend, not its steepness. A gentle slope with the points packed close to it is a strong relationship; a steep slope with points spread widely around it is a weaker one. The correlation coefficient follows strength, not steepness: footfall and transactions at Harbourside correlate at 0.985, and so do footfall and transactions at Millbrook, though the two cafés’ bands rise at different rates.

Words such as “strong” and “weak” have no agreed cut-offs. Textbooks attach different labels to the same value of r, and a correlation of 0.3 can be a useful finding in one field and noise in another. The next lesson gives the number its meaning; in a description, say what the plot shows (“a loose downward cloud”) and give r with it.

Form: lines, curves and clusters

A linear form is the one a correlation coefficient is built for. Many business relationships are not straight. The iced share is an S-shaped curve: on days below 50°F about 6–8% of drinks are iced whatever the exact temperature, between 60°F and 80°F each degree adds roughly two to three percentage points, and above 80°F the share levels off near 80%. The correlation, 0.94, is high because the curve is steady, but a straight line through it misses in a pattern: it predicts a negative share on the coldest days, too high a share in the 50s and 60s and on the hottest days, and too low a share in the 70s and 80s. The nonlinear-and-restricted lesson deals with curves, including a hockey-stick shape in the peak waits.

Clusters are separate clouds of points, and they usually mean the data mix two kinds of unit. Plot the day’s sales against its transactions for both cafés together and you see two bands, not one: at the same number of transactions Harbourside takes much more money, because its average ticket is about $5.34 against Millbrook’s $3.27. Inside each café the relationship is close to a perfect line (r = 0.99 at Harbourside and 0.98 at Millbrook), but pooled the correlation is only 0.83. When you see clusters, colour the points by group or plot one group at a time; the groups-and-simpsons-paradox lesson shows how pooling groups can even reverse a correlation.

Watch the spread too. If the cloud fans out as x grows, the relationship is tighter for small values than for large ones, and a single number can’t describe both ends. The NIST/SEMATECH handbook lists changing spread among the questions a scatterplot should answer, with outliers and curvature.

Unusual points

A point can be unusual in x, in y, or in both, and what matters is whether it breaks the pattern. On Harbourside’s five harbour-festival days footfall ran from 502 to 620, far above any other day, and transactions were far higher too. Those points are extreme in both variables but lie on the same band as the rest: they are real days that stretch the range, and they strengthen the straight-line pattern rather than distort it.

Millbrook’s footfall on 14 September is recorded as 2,566, more than six times the busiest day it ever had, while its transactions, 130, were ordinary. The point sits far to the right with an ordinary height: it breaks the pattern. The true count was 256; someone typed an extra digit. That one point drops Millbrook’s correlation between footfall and transactions from 0.985 to 0.519, and the two cafés’ pooled correlation from 0.98 to 0.68. A scatterplot shows a slip like this at a glance; a correlation coefficient hides it inside a number that merely looks disappointing. Checking, correcting and reporting such points is the outliers-and-influence lesson’s subject; this course’s lessons use the corrected log unless they say otherwise.

Too many points: overplotting

With 612 points, many land on top of one another and the plot hides how many there are: this is overplotting. It is worst when values are recorded in steps. The day’s high is a whole number of degrees, and both cafés share it, so only 60 different temperatures appear across 612 rows. Peak barista hours are rostered in half-hours, 20 different values in all. Plot transactions against peak barista hours and 345 of the 612 points sit exactly on top of at least one other point: a column of dots can stand for one day or for nine.

Transparency. Draw each point partly see-through, so that places where many points overlap look darker. In Excel, the marker fill of a data series has a transparency setting.
Jitter. Jittering adds a small random nudge to each point’s position, less than half the step between recorded values either way (under a quarter of an hour either way for half-hourly barista hours), so stacked points spread into a visible clump. Jitter is for drawing only: calculate every statistic from the recorded values.
One group at a time. Plot each café on its own panel, with the same axes, or colour the points by café. Side-by-side panels on common axes are called small multiples.
Smaller or hollow markers. They overlap less, and the eye can still count them.
Bins. With many thousands of points, count the points in small squares or hexagons of the plot and shade each by its count. With 612 points this is rarely needed.

Time order hides in a scatterplot

Each point in the log is a day, but a scatterplot throws the order of the days away. When both variables drift through the year, the plot can show a shape that only the calendar explains. Harbourside’s loyalty members grow steadily from opening day, while the high temperature rises to July and falls after it. Plotted against each other, the 306 days form a hairpin: members rise with temperature through the spring, then keep rising as temperatures fall in the autumn. The correlation over the whole year is −0.13, which describes neither half.

To see time order, colour the points by month, or join them with a line in date order (a connected scatterplot). Whenever both variables are Time-series data, ask whether each simply trends with the calendar. Two series that both trend can correlate strongly without having anything to do with each other; the correlation-and-causation lesson returns to Harbourside’s members. Days that follow one another also tend to be alike, so 306 days carry less independent information than 306 unrelated observations would; the testing-correlation lesson says when that matters.

Anscombe’s quartet

In 1973 the statistician Francis Anscombe published four small datasets to make the case for plotting. Each has 11 pairs of x and y. In all four the mean of x is 9, the variance of x is 11, the mean of y is 7.50, the variance of y is about 4.12–4.13, the correlation is 0.82 (0.8162 to 0.8165), and the least-squares line is $\hat{y} = 3.00 + 0.500x$ . To two or three decimal places, every number in a standard summary is the same.

Anscombe’s quartet (values from Anscombe, 1973). Each panel has, to two or three decimal places, the same means, variances, correlation (0.82) and least-squares line, the dashed line ŷ = 3.00 + 0.500x, but tells a different story.

Anscombe’s quartet: the same summary, four different relationships
Set	What the plot shows	What r = 0.82 hides
1	A loose straight band	Nothing: here r is a fair summary
2	A smooth curve that rises and falls	y is an exact function of x, just not a straight one
3	A perfect line with one stray point	Without the stray point r is 1.000
4	One column of points and one far-off point	x varies only because of one point; without it there is no relationship to measure

The quartet is the strongest argument for the habit this lesson teaches: compute the number, but never report it without looking at the picture it summarises.

Formulas

Correlation is symmetric

r_{xy} = r_{yx}

$r_{xy}$: Pearson’s correlation of x with y
$r_{yx}$: Pearson’s correlation of y with x

Swapping the axes of a scatterplot never changes r. CORREL(A2:A31,B2:B31) equals CORREL(B2:B31,A2:A31). A regression slope is not symmetric (B06).

Jitter for drawing

x_i^{\text{plot}} = x_i + u_i, \qquad u_i \sim \text{Uniform}(-h, h)

$x_i$: The recorded value of point iunits of x
$u_i$: A random nudge drawn afresh for each pointunits of x
$h$: The largest nudge: less than half the step between recorded valuesunits of x
$x_i^{\text{plot}}$: Where the point is drawnunits of x

Assumes. Used only to draw the plot; every statistic uses the recorded values.

Jitter y the same way when it is recorded in steps too.

Worked examples

A first example: Reading a plot: advance bookings and admissions at a cinema

Fictional figures

Pemberton Cinema (fictional) records, for ten evenings, the tickets booked online by noon and the evening’s total admissions. The manager wants to know whether noon bookings are a useful early guide to how busy the evening will be. Describe the scatterplot.

Evening	Booked by noon	Admissions
1	42	118
2	55	152
3	31	93
4	68	174
5	47	124
6	74	188
7	25	84
8	60	155
9	38	112
10	20	210

Axes: bookings by noon are known first and are the candidate guide, so they go on x (explanatory) and admissions on y (response).
Direction: on nine of the ten evenings, more bookings go with more admissions: a positive association.
Form and strength: those nine points lie close to a straight band rising by roughly two to three admissions per extra booking, a strong linear pattern.
Unusual points: evening 10 had the fewest bookings (20) and the most admissions (210). It sits far above the band at the left: unusual in y for its x, and it breaks the pattern. The manager checks the diary: it was a private hire, sold to a club by phone, so it never appeared in online bookings.
Number, read with the plot: with evening 10, =CORREL(B2:B11,C2:C11) is 0.41; without it, 0.996. The one point changes r from about 0.4 to almost 1.

Answer. A strong, positive, linear relationship on ordinary evenings (r = 0.996 without evening 10), with one point that breaks the pattern: a private hire. With it included, r is only 0.41.

What it means. Noon bookings are a good early guide for public screenings, and private hires need their own line in the plan. Neither number alone tells that story: the plot shows why they differ. Leaving the point out is justified here only because there is a documented reason it belongs to a different kind of evening; the outliers-and-influence lesson sets out when removing a point is honest.

A harder example: Anscombe’s quartet: one summary, four stories

Anscombe’s four datasets each have 11 pairs and, to two or three decimal places, the same means (x̄ = 9, ȳ = 7.50), the same variances, r = 0.82 and the same least-squares line. Using the figure in In depth, say for each set whether r = 0.82 is a fair summary, and what you would report instead when it is not.

Set 1: the points scatter evenly around a straight line with no unusual points. r = 0.82 is a fair summary of a moderately strong linear relationship.
Set 2: the points lie on a smooth curve that rises then falls. The relationship is perfect, but not linear; r measures how close the points are to a straight line, so it understates the link. Report the curve (and in B06 fit it), not r.
Set 3: ten points lie exactly on one line and a single point at x = 13 lies far above it. Without that point r is 1.000. Check the point: if it is an error, correct it; if it is real, report the relationship with and without it.
Set 4: ten points share x = 8, so x varies only because of the single point at x = 19. That one point creates the whole correlation. Report that x barely varies and that one observation creates the number; collect data across a range of x before claiming anything.
Common lesson: the summary statistics agree to two or three decimal places, so only the plot separates a fair summary (set 1) from a curve (2), an outlier (3) and a single influential point (4).

Answer. r = 0.82 is a fair summary only for set 1. Set 2 is a perfect curve, set 3 a perfect line spoiled by one point, and set 4 has no relationship apart from one point.

What it means. Anscombe built the quartet so that the numbers agree to two or three decimal places while the pictures disagree completely. In business data the same thing happens less tidily: a data-entry slip (set 3), a curve such as the iced-drink share (set 2), or one big event that stretches the range (set 4). Plot first, and let the plot decide which number to report.

Applied: a business decision: Planning iced drinks from the weather forecast

Fictional figures

Ana wants her baristas to prepare cold-brew concentrate and ice in line with the day’s forecast high. She plots the share of drinks served iced against the high for all 612 café-days and groups the days into temperature bands. What should the plan say?

High (°F)	Café-days	Average iced share
25–39	32	6%
40–49	92	8%
50–59	116	13%
60–69	128	32%
70–79	122	60%
80–89	110	77%
90–99	12	81%

Axes: the forecast high is known in the morning and the drinks follow, so temperature goes on x and iced share on y.
Direction and strength: a strong positive association; r = 0.94 for the 612 café-days, and each café’s own plot looks the same (0.94 at both).
Form: the band averages show an S-shape rather than a line. Below 50°F the share hardly moves (6% to 8%); from 50°F to 80°F it climbs steeply (13%, then 32%, then 60%); above 80°F it flattens (77% to 81%).
Unusual points and structure: both cafés share each day’s temperature, so each temperature appears about ten times; drawing the points partly transparent, or the two cafés on separate panels, shows the band clearly. No point breaks the pattern.
Plan: base prep on the band, not on a straight line. Cold days need a small fixed amount of iced stock; between 50°F and 80°F each extra 10°F roughly doubles the iced share; above 80°F stock for about four drinks in five being iced.

Answer. Iced share rises with the day’s high in an S-shape: about 6–8% below 50°F, climbing steeply through the 60s and 70s, and levelling at about 80% above 80°F. Plan prep by temperature band.

What it means. The correlation, 0.94, says the link is strong and steady; the plot says how it works, which is what a prep plan needs. A straight-line rule would have Ana overstock iced drinks on cool days in the 50s and 60s and understock on warm days in the 70s and 80s. In the case, warmer weather is built into the data as a reason people choose iced drinks; in real data the plot alone would show only that the two go together, and the prep plan would still work as long as the pattern holds.

Explore

Common mistakes

“r is 0.94, so the relationship is a straight line.”
A high r says the points are close to some rising pattern that a line approximates well; it doesn’t say the pattern is straight. The iced share is an S-curve with r = 0.94. Look at the plot, and at whether the points sit on one side of a straight line in some ranges and on the other side elsewhere.
“r is close to 0, so there is no relationship.”
r measures only straight-line association. Anscombe’s set 2 is a perfect curve; a U-shape can give r = 0 with a perfect link. A correlation near 0 means no linear trend, which the plot must confirm.
Reporting the correlation without plotting
One slipped digit at Millbrook turns r = 0.985 into 0.519, and nothing in the number says why. The plot shows the stray point immediately. Plot every pair you report, and keep the plot with the number.
Treating every extreme point as an error
The festival days are far from the rest but real, and they lie on the pattern. Only a point that is impossible or contradicts other records (2,566 people and an ordinary 130 transactions) is a candidate error. Unusual is a reason to check, not to delete.
Computing statistics on jittered values
Jitter is a drawing aid. Calculate r and everything else from the recorded values; the jittered ones contain made-up noise.
Ignoring which café, or which month, a point belongs to
Pooled data can show clusters (sales against transactions) or hairpins (members against temperature) that come from mixing groups or periods. Colour by group and by time before trusting the overall pattern.

Guided practice

Work through these with hints when you need them. Every hint and every worked solution is there to be used.

Loading four questions…

On your own

No hints this time. Answer, check, and read the feedback; the solution appears once you have answered or if you ask for it. Your first answers count toward mastery.

Loading six questions…

In practice

Fictional case

Ana’s first pass through the log is a page of scatterplots. For each pair of measures she cares about, she plots one café at a time on common axes, with points drawn partly transparent, and writes one sentence under each plot before looking at any correlation. The page catches the footfall slip on 14 September (corrected to 256 from the counter’s own record), shows that sales against transactions form two separate bands, one per café, and turns the iced-drink question into a prep plan by temperature band.

This is how analysts use scatterplots in practice: as the first look at every relationship, before any model, and as the evidence that goes in the report next to any correlation. A plot of each pair is cheap; a decision based on a number nobody looked at can be expensive. A scatterplot matrix, a grid with one small scatterplot for every pair of variables, puts a whole dataset’s pairs on one page, and the correlation-matrices lesson pairs it with a table of correlations.

Write the description in the reader’s terms. “On warmer days a larger share of drinks is iced: almost none below 50°F, about four in five above 80°F, rising steeply in between” tells a café manager what to do. “r = 0.94” does not, and nor does it say that the rise is not straight.

Keep the language careful. A scatterplot shows what goes with what in the data you have. It does not show that one variable causes the other: peak barista hours go with transactions because the manager rosters more baristas on days she expects to be busy, not because baristas bring customers in. The NIST/SEMATECH handbook puts it plainly in its entry on scatterplots: association does not prove causality. This course separates the two in its last lesson.

Finally, remember what the plot can’t show. It has no time order unless you add it, no groups unless you colour them, and no record of how the data were collected. A scatterplot of the days Ana chose to look at is not a scatterplot of all days; if she had logged only weekdays, or only summer, the picture and its correlation would both change, as the nonlinear-and-restricted lesson shows.

Knowledge check

Six questions, one attempt each, no hints. Score 80% or more to complete the lesson; you can take it again at any time.