A Short Summary of Smoothing Algorithms
When data are noisy, it’s our job as data scientists to listen for signals so we can relay it to someone who can decide how to act. To amp up how loudly hidden signals speak over the noise of big and/or volatile data, we can deploy smoothing algorithms, which though traditionally used in time-series analysis, also come into their own when applied on other sequential data.
Smoothing algorithms are either global or local because they take data and filter out noise across the entire, global series, or over a smaller, local series by summarizing a local or global domain of Y, resulting in an estimation of the underlying data called a smooth. The specific smoother you use depends on your analysis’ goal and data quirks, because as we’ll see below, there are trade-offs to consider. Below are a few options, along with their intuition, limitations, and formula so you can rapidly evaluate when and why to use one over the other.
1. Bin Smoothing
A bin smoother cuts your domain into discrete bins of a fixed-width over which an average is computed, which results in equally weighted estimations of Y across its domain.
Limitations: Sometimes it’s inappropriate to weight Y equally across its domain.
Math: Really depends on the specific binning algorithm you go with, but in general:
Source:
2. Simple Moving average
Arguably the simplest and most common smoother is the trusted simple moving average, which is similar to bin smoothing, except the mean value is computed over a variable bin-width with a fixed number of observations.
Limitations: Inflexible, reductive near the complexity
Math:
Source:
3. Running Line — local or global
Running line smoothers create a smooth that is more flexible and reliable than the simple moving average because it fits a line to Y over the fixed-width rather than a simple average.
Limitations: Inflexible, reductive near complexity.
Math:
4. Kernel Density
Like the running line smoothers, kernel density is similar to a moving average, except its average is weighted and has a fixed bin-width.
Limitations: Slow.
Math:
See Watson (1964; Theory and Probability Applications, 10, 186–190) and Nadaraya (1964; Sankha A, 26, 359–372).
5. LOESS — Locally estimated scatterplot smoother
Loess is the conceptual product of running line and kernel density smoothers. That is, Loess extends the idea of fitting a line over variable bin-widths but it’s a weighted regression line.
Limitations: computationally expensive, but this is an outmoded concern on modern machines, except in rare applications.
Math:
See: Cleveland 1979; Journal of the American Statistical Association, 84, 829–836.
6. Smoothing Splines
The intuition behind smoothing splines is to cut Y’s domain into partitions over which the algorithm computes a spline, which are joined at intersections called knots. These splines are piecewise polynomials that are typically restricted to being smooth at these knots such that the “knotty-ness” is unobservable to the human eye (but need not be).
Math:
Limitations: Untenably rough in large samples.
As you can see these smoothing techniques come in a host of flavors:
- Sides. One-sided or two-sided.
- Exponent. Exponential or linear.
- Weights. Weighted or unweighted.
- Bin-width. Fixed or variable.
- Observations. Fixed or variable.
Smoothing techniques are statistically diverse, helpful, and readily available. Get out there and smooth your teeth on a project!
Resources:
