Within-Array Loess Normalization for Two-Color Arrays

Why Use an MA Plot?

A two-color microarray measures two labeled samples on the same array, often called the red and green channels. Even when the underlying relative expression is not changing with intensity, a dye or measurement effect can make the observed channel ratio vary across the intensity range. A single correction applied to every spot can miss this pattern. Within-array loess normalization estimates a smooth intensity-dependent trend in the log ratios and removes it.

The key diagnostic is an MA plot: each spot is represented by its log ratio \(M\) and average log intensity \(A\). A systematic curve in \(M\) across \(A\) suggests that the ratio depends on signal strength. The method is useful when that dependence is plausibly technical and smooth, and when the bulk of spots provides a reasonable reference for estimating it.

A positive \(M_i\) means the red-channel intensity exceeds the green-channel intensity; a negative value means the reverse. The \(A_i\) coordinate describes the spot’s overall signal level, not its relative channel balance. An MA plot displays one \((A_i,M_i)\) point per spot. The ideal reference is not necessarily that every spot has \(M_i=0\), but that the technical component of \(M_i\) has no remaining systematic trend with \(A_i\).

Model and Loess Correction

Write the observed log ratio as the sum of a relative signal, an intensity-dependent technical effect, and residual variation:

Here \(\theta_i\) is the underlying log ratio of interest, \(b(A_i)\) is a smooth bias that depends on average intensity, and \(\varepsilon_i\) is residual error. The normalization target is to remove \(b(A_i)\), not to erase genuine differences represented by \(\theta_i\). To estimate the bias from one array, a central assumption is that, across most spots, the underlying ratios are balanced around zero at each intensity level: approximately, \(E(\theta_i\mid A_i)=0\). If this is reasonable, the conditional trend of \(M_i\) against \(A_i\) estimates the technical trend.

Loess (locally estimated scatterplot smoothing) estimates that trend by fitting a low-degree polynomial in a neighborhood of each intensity. For a local linear fit at target intensity \(A_0\), choose neighborhood weights \(w_j(A_0)\) and solve:

The fitted trend at \(A_0\) is \(\hat f(A_0)=\hat\beta_0\). A common choice assigns larger weights to nearby intensities using a tricube kernel: if \(d_j=|A_j-A_0|/h\), where \(h\) is the neighborhood radius, then \(w_j=(1-d_j^3)^3\) for \(0\le d_j<1\), and \(w_j=0\) otherwise. The span determines how large a fraction of spots contributes to each local fit. A smaller span follows more local variation but can be noisy; a larger span is smoother but may miss a real bend in the trend.

For each spot, evaluate the fitted curve at its intensity and subtract it from its observed log ratio:

Some implementations use robust reweighting so that large residuals have less influence on later fitting iterations. This can help when a few spots are outliers, but it does not protect against a broad group of genuinely changed spots that shifts the fitted trend.

Why the Average Log Signal Is Preserved

The loess adjustment is made to \(M_i\), while \(A_i\) is retained. Since \(A_i\) and \(M_i\) are the average and difference of the two channel logs, they can be converted back without approximation. For any adjusted ratio \(M_i^{\mathrm{norm}}\), define the corrected logs as follows:

Adding these equations and dividing by two gives \((r_i^{\mathrm{norm}}+g_i^{\mathrm{norm}})/2=A_i\). Subtracting them gives \(r_i^{\mathrm{norm}}-g_i^{\mathrm{norm}}=M_i^{\mathrm{norm}}\). Thus the transformation changes the log ratio exactly as intended while preserving the spot’s average log intensity. Equivalently, the geometric mean of the corrected channel intensities remains the original geometric mean.

This does not mean the two channel intensities, their arithmetic mean, or the total red and green signal over the whole array must remain unchanged. It means that each spot’s mean of the two channel log intensities is unchanged. This is the relevant sense of preserving overall signal when adjusting the ratio coordinate of the MA representation.

Assumptions, Diagnostics, and Failure Modes

The normalization is most defensible when the channel intensities are valid inputs, the technical bias changes smoothly with \(A\), and most spots at a given intensity are not shifted in the same biological direction. These conditions are assumptions, not consequences of the smoothing algorithm. A loess curve will be produced even when the assumptions are false.

• Check the raw MA plot. A systematic smooth curve is consistent with intensity-dependent bias. A roughly horizontal cloud may not need such a correction. A few extreme points can affect an ordinary fit, so inspect their frequency and whether robust fitting was used.
• Check the normalized MA plot. The broad smooth trend should be reduced. Residual scatter is expected; normalization does not require every \(M_i^{\mathrm{norm}}\) to equal zero.
• Look for unstable ends. There may be few spots at very low or high intensities. Local fits near the boundaries can therefore be sensitive to the span, available neighbors, or individual points.
• Vary the span as a sensitivity check. A highly wiggly fit can remove noise as if it were bias. An overly smooth fit can leave a systematic pattern. If the normalized conclusions change substantially across reasonable spans, the trend is not being estimated robustly.
• Consider the biological distribution of changes. If many spots truly shift together, the fitted curve may treat that real shift as bias. The correction can then remove part of the biological signal.

The central assumption can be particularly doubtful when the two samples differ globally or when changes are strongly associated with intensity. In those settings, the observed trend combines technical bias with biological structure, and one array alone may not distinguish them. A visually flatter MA plot is not, by itself, proof that the correction was appropriate.

How Loess Differs From Simpler Corrections

A global centering correction subtracts one constant from all log ratios, often a median or mean. It can remove an overall offset, but it cannot correct a curve that bends with intensity. Within-array loess instead estimates a varying correction \(\hat f(A_i)\). It can therefore address smooth intensity dependence, at the cost of needing enough spots across the intensity range and a credible assumption about the underlying ratio distribution.

No correction is also a legitimate comparison when the MA plot has no meaningful trend. Conversely, flattening a plot by applying a flexible curve is not automatically better: if the pattern is biological, loess can erase it. These choices concern the log-ratio structure within one two-color array. They should not be confused with methods that align intensity distributions across different arrays.

Common Mistakes

• Subtracting the fitted curve from \(A\). The loess trend is fitted to \(M\) as a function of \(A\); the adjustment is \(M_i-\hat f(A_i)\). Keep \(A_i\) unchanged when preserving average log signal.
• Assuming normalization makes every ratio zero. It removes the estimated trend, not all spot-level variation or genuine relative expression.
• Treating the trend as certainly technical. It is estimated from the observed ratios. A broad biological shift can be absorbed into the curve if the balancing assumption fails.
• Ignoring sparse intensity regions. Boundary fits can be unstable because their local neighborhoods contain fewer informative spots.
• Calling the preserved quantity “total signal.” The preserved quantity is the average of the two log intensities for each spot, equivalently their geometric mean—not necessarily the arithmetic total across channels.