Why Overall Range Is Not Enough
A fitted curve may be evaluated at predictor values between the smallest and largest observed values, yet some portions of that interval may contain very few observations. A broad range can hide a large gap, a thinly sampled stretch, or dense clusters separated by sparse regions. In those places, the curve is still within the observed range, but the data provide less direct local information about its shape.
The key question is therefore not only whether a predictor value lies between the observed minimum and maximum. It is also how many observations lie near that value, how widely spaced those observations are, and whether the conclusion changes when “near” is defined using a different interval width. This assessment concerns support in the predictor values; it does not by itself establish whether the fitted relationship is correct.
Counting Observations Near a Predictor Value
For observed pairs \((x_i,y_i)\), where \(i=1,\ldots,n\), choose a window width \(w\) in the units of the predictor. A centered window around \(x\) includes observations whose predictor values are within \(w/2\) of \(x\). The local count records how many observations fall inside it. Dividing by the count of all observations gives the fraction of the sample in that neighborhood.
Here \(I(\cdot)\) equals 1 when the condition inside it is true and 0 otherwise. A count is easy to explain, but it depends on the chosen window width: a wider window will usually include more observations. If widths differ, a rate per predictor unit can help make counts comparable. For a fixed interval \(A\) of width \(|A|\), the sample density within that interval is the number of observed predictor values in \(A\), divided by \(|A|\).
Density is a descriptive summary, not a universal pass-or-fail threshold. It is meaningful only alongside the interval width, predictor units, sample size, and comparison regions. For a window centered near an endpoint, one can use the intersection of the window with the observed range and divide by the length of that intersection; this avoids treating a shorter, clipped interval as if it had the full window width. Another option is to use fixed bins defined across the range and compare their counts directly.
Worked Example: Finding a Sparse Stretch With Fixed Bins
Worked Example: Finding a Sparse Stretch With Fixed Bins
Suppose a hypothetical data set records the weekly operating time \(x\), in hours, of 40 machines. The observed values range from 0 to 100 hours. Before examining the outcomes, an analyst divides this range into five bins, each 20 hours wide, and finds counts of 12, 10, 2, 9, and 7.
The local density in the third bin, from 40 to 60 hours, is \(2/20=0.10\) observations per hour. In the neighboring bins, the densities are \(10/20=0.50\) and \(9/20=0.45\) observations per hour. Thus, the third bin has substantially fewer observed predictor values per hour than either neighboring bin.
The third bin contains \(2/40=0.05\), or 5%, of the sample; each of its neighboring bins contains 25% and 22.5%, respectively. This identifies the 40-to-60-hour interval as relatively weakly supported at this bin width. It does not show that the curve is wrong there, nor does it tell us how much any particular fitted value might vary. The conclusion is specifically about the distribution of observed operating times.
Gaps and Spacing Reveal What Counts Can Miss
A bin count summarizes an interval but not the arrangement of observations inside it. For instance, a bin containing six observations could have them spread fairly evenly throughout, or have five bunched together at one end and one far away. Sorting the predictor values and examining adjacent spacings can expose large internal gaps that a count alone obscures.
The values \(x_{(j)}\) are the observed predictor values in sorted order, and \(g_j\) is the gap between consecutive values. A large gap relative to the typical gaps in the same region is evidence that predictor values are locally sparse. Tied observations produce a gap of zero, which may indicate clustering; they do not imply coverage of the intervening predictor values.
Worked Example: Detecting an Internal Gap
Consider a hypothetical set of 12 observed predictor values, measured in minutes: \(0, 2, 4, 6, 8, 10, 50, 52, 54, 56, 58, 60\). The observed range is 0 to 60 minutes, but the values form two clusters.
The successive gaps within the first cluster are all 2 minutes, as are the gaps within the second cluster. The gap between 10 and 50 minutes is \(50-10=40\) minutes. It is 20 times as large as the typical within-cluster gap of 2 minutes.
The interval between the clusters therefore has weak direct support, despite lying inside the observed range. For example, the value 30 minutes is between the minimum and maximum, but no observation is near it: its distance to either nearest observed value, 10 or 50, is 20 minutes. A curve evaluated at 30 minutes is not outside the observed range, yet interpreting its shape there should acknowledge the pronounced lack of nearby predictor values.
Scale, Units, and the Choice of Neighborhood
“Near \(x\)” has no meaning until a scale is chosen. A 10-unit window may be narrow when the predictor spans thousands of units and broad when it spans only 20. Choose window widths that are interpretable in the predictor’s units and relevant to the changes in the relationship being considered. A practical assessment often checks more than one width: a region that looks sparse under several reasonable choices is a clearer concern than one that appears sparse only under a single arbitrary cut.
For a descriptive calculation, if the predictor values can be regarded as independent draws from a population distribution with density \(g\), the expected count in an interval \(A\) is \(nP(X\in A)\). For an interval of width \(w\) centered at \(x\), this gives:
Dividing by \(nw\) shows why a count per observation and per unit width can indicate the average predictor density over that neighborhood:
This is an average of \(g\) over the window, not a precise measurement of density at the single point \(x\). Narrow windows describe finer local variation but can have unstable counts; wide windows smooth over gaps and clusters. The calculation motivates comparing neighborhoods, but it does not supply a universal threshold for declaring support adequate.
Worked Example: Checking Whether Sparsity Depends on Window Width
Suppose a hypothetical study observes 80 sites with predictor values measured as distance in kilometers along a route. In a 10-kilometer interval around a particular location, 3 sites are observed. In a 20-kilometer interval centered at the same location, 14 sites are observed.
For the 10-kilometer interval, the sample density is \(3/10=0.30\) sites per kilometer, and the sample fraction is \(3/80=0.0375\), or 3.75%. For the 20-kilometer interval, the density is \(14/20=0.70\) sites per kilometer, and the fraction is \(14/80=0.175\), or 17.5%.
The wider interval includes sites from beyond the immediate neighborhood, so its higher density does not erase the evidence that the narrower region is thinly sampled at that scale. A useful report would state both widths and counts rather than label the location simply “supported” or “unsupported.” If the 10-kilometer interval was chosen only after inspecting where the outcome curve looked unusual, the assessment is also less neutral; window choices are better set from predictor units and the sampling design before interpreting outcome patterns.
What Local Support Can and Cannot Tell You
Local support is about where the observations occur along the predictor axis. It helps identify parts of the fitted curve that are anchored by many nearby predictor values and parts whose shape is more dependent on the chosen curve form and on observations farther away. When describing a nonlinear fit, it is useful to show or summarize the predictor distribution alongside the curve so that an apparent bend is not mistaken for a well-sampled feature simply because it falls between the observed endpoints.
Sparse predictor values do not necessarily mean that the fitted relationship is inaccurate, and dense predictor values do not guarantee a correct model. Support summaries also do not measure residual variability or quantify uncertainty in a fitted curve. They answer a narrower but important question: where, and at what scale, are there relatively few observed predictor values to inform the local shape?
Common Mistakes
- Using only the minimum and maximum. Being between the endpoints does not mean observations are distributed throughout the interval. Check local counts and gaps.
- Calling a region dense because its bin contains many observations without stating the bin width. Counts are comparable only when the intervals are comparable; report width and units, or use a density rate.
- Assuming a zero gap or repeated predictor values imply broad coverage. Ties can create a cluster at one value while leaving nearby values unobserved.
- Treating one chosen window as definitive. Explain the choice and check whether the conclusion is sensitive to reasonable alternative widths.
- Equating predictor sparsity with high residual spread or a wrong curve. These are distinct issues. Local predictor support describes the locations of observations, not their outcome scatter or the correctness of the model.
Check Your Understanding
Try these questions using the definitions and methods in this tutorial.
- Why can a predictor value lie inside the observed range but still have weak local support?
- A 5-unit bin contains 4 observations, and a neighboring 10-unit bin contains 6. Calculate each bin’s sample density and explain why the raw counts alone are not directly comparable.
- What does a large gap between adjacent sorted predictor values suggest? What does it not prove about the fitted curve?
- Why might a region look sparse with a narrow window but not with a wider one?
- A data set has many repeated observations at one predictor value and no values near it. Explain why the repeats do not establish support across the gap.