Signal Distortion: Amplitude, Phase & Nonlinear

A signal is undistorted when the receiver gets the same waveform, multiplied by a constant and shifted in time. Anything else changes relationships among the signal's components. A linear channel does that by applying unequal gain or unequal delay across frequency; a nonlinear device does it by making gain or phase depend on instantaneous level and thereby creates frequencies that were never present at its input.

Those mechanisms demand different tests and different repairs. A frequency-response sweep cannot characterize compression, a two-tone IP3 test cannot reveal group-delay ripple, and a single EVM number cannot identify either root cause by itself.

The reference definition #

For an input $x(t)$, the ideal output is

$$y(t)=Kx(t-\tau_0),$$

where $K$ is a constant gain and $\tau_0$ a constant delay. Gain and delay may be inconvenient, but they do not change shape. Distortion is the residual after those two harmless freedoms are removed.

For a linear time-invariant (LTI) path with transfer function

$$H(f)=|H(f)|e^{j\phi(f)},$$

the distortionless condition over the occupied signal band of a real waveform is

$$H(f)=Ke^{-j2\pi f\tau_0}.$$

It requires two things at once:

• $|H(f)|=K$: constant magnitude, so every spectral component is scaled equally;
• $\phi(f)=-2\pi f\tau_0$ (up to a sign inversion): linear phase, so every component receives the same delay.

For a complex baseband envelope, the equivalent condition may include a constant rotation, $H_{bb}(f)=Ke^{j\phi_0}e^{-j2\pi f\tau_0}$, because a coherent receiver can absorb one carrier-phase offset. Applying an arbitrary common phase to all positive-frequency components of a general real waveform is not the same freedom: conjugate symmetry constrains the negative-frequency response. In either representation, phase curvature across the occupied band is what produces frequency-dependent delay.

Linear distortion changes existing components #

An LTI path never invents frequencies. It changes the complex coefficient multiplying each frequency already present.

Amplitude distortion #

Suppose

$$x(t)=\cos(2\pi\cdot1\,\text{MHz}\,t)+\cos(2\pi\cdot4\,\text{MHz}\,t).$$

If the path is 0 dB at 1 MHz and $-6.02$ dB at 4 MHz, its output, after removing common delay, is

$$y(t)=\cos(2\pi\cdot1\,\text{MHz}\,t)+0.5\cos(2\pi\cdot4\,\text{MHz}\,t).$$

No new line appears in the spectrum, yet the peaks, zero crossings, and crest factor of the sum change because one component is now half as large relative to the other. The same mechanism rounds digital edges: a transition needs a weighted set of harmonics, so attenuating the high-frequency terms changes the transition rather than merely making it smaller.

Phase and group-delay distortion #

Group delay is the negative slope of unwrapped phase:

$$\boxed{\;\tau_g(f)=-\frac{1}{2\pi}\frac{d\phi(f)}{df}\;}.$$

Linear phase gives constant $\tau_g$ and therefore only a time shift. Phase curvature makes $\tau_g$ frequency-dependent. Different portions of a pulse spectrum then emerge at different times and cannot reconstruct the original pulse shape.

Why delay ripple becomes ISI #

After the transmit filter, channel, and receive filter, symbol-rate samples obey

$$y[m]=\sum_k a_k h[(m-k)T+\tau]+n[m].$$

A Nyquist pulse has $h(nT)=0$ for nonzero integer $n$ at the intended sampling phase. Amplitude tilt or group-delay ripple changes the composite impulse response, moves energy away from the main sample, and makes those neighbouring samples nonzero. That is inter-symbol interference (ISI): linear distortion observed at the decision instants. Timing recovery can choose the best sampling phase, but it cannot make a dispersed impulse response collapse back to one sample; that requires equalization. See symbol timing and ISI for the receiver loop and equalizer choices.

Nonlinear distortion creates new components #

A memoryless weak nonlinearity can be expanded around its operating point:

$$y(t)=a_0+a_1x(t)+a_2x^2(t)+a_3x^3(t)+\cdots.$$

For a single tone $x(t)=A\cos\omega t$,

$$x^2(t)=\frac{A^2}{2}\left(1+\cos2\omega t\right),$$

$$x^3(t)=\frac{A^3}{4}\left(3\cos\omega t+\cos3\omega t\right).$$

The second-order term creates DC and a second harmonic; the third-order term changes the fundamental gain and creates a third harmonic. Those harmonics may fall outside a narrow channel and be filtered. With multiple tones, however, intermodulation products can land beside or inside wanted channels.

For equal tones at $f_1$ and $f_2$, third-order terms include

$$2f_1-f_2 \qquad\text{and}\qquad 2f_2-f_1.$$

P1dB and IP3 answer different questions #

1 dB compression point, P1dB

The input or output power where gain has fallen 1 dB below the extrapolated small-signal gain. It is an observed large-signal point and must be labelled IIP1dB or OIP1dB when the reference side is not obvious.

Third-order intercept, IP3

The hypothetical intersection of extrapolated fundamental and third-order-product lines: slopes of 1 and 3 dB per dB of input. IIP3 refers the intersection to input; OIP3 refers it to output.

AM-AM, AM-PM, and memory #

For a narrowband RF signal represented by complex envelope $x(t)=r(t)e^{j\theta(t)}$, a memoryless power amplifier can be described as

$$y(t)=A(r)e^{j[\theta(t)+\Phi(r)]}.$$

• AM-AM is $A(r)$: output magnitude as a function of input magnitude. Compression is AM-AM distortion.
• AM-PM is $\Phi(r)$: phase shift as a function of input magnitude. Envelope peaks rotate differently from small samples even when average carrier phase is correct.

Real wideband amplifiers also have memory. Transport delay, rapid thermal time constants in active devices, and envelope-dependent behaviour in biasing components make the present output depend on previous envelope values. A memoryless AM-AM/AM-PM curve then becomes a thick loop rather than a single-valued curve.

The Volterra series is the general extension of a power series to nonlinear systems with memory: each order contains a multidimensional convolution kernel. Its full term count is impractical for a wideband transmitter. Memory-polynomial and generalized-memory-polynomial (GMP) models retain selected delayed envelope terms such as

$$y[n]\approx\sum_{k\in\{1,3,5,\dots\}}\sum_{m=0}^{M} a_{k,m}\,x[n-m]|x[n-m]|^{k-1}$$

plus lagging and leading envelope cross-terms. This is why a two-tone IP3 measured at one spacing can look excellent while a wideband modulated waveform still has poor EVM or spectral regrowth: the wider waveform excites memory that the test did not represent.

Four measurements, four different questions #

flowchart LR
  X[Known stimulus] --> N{New spectral lines?}
  N -->|yes| NL[nonlinear distortion]
  N -->|no| H[measure complex H of f]
  H --> A{magnitude flat?}
  A -->|no| AD[amplitude distortion]
  A -->|yes| G{group delay flat?}
  G -->|no| PD[phase or delay distortion]
  G -->|yes| R[check noise, timing, and reference plane]
  NL --> T[two-tone plus power sweep]
  AD --> E[linear equalization]
  PD --> E
  T --> D[backoff or DPD]

The diagram is a first split, not a full test plan. Noise can obscure every branch, and a nonlinear DUT can corrupt a multitone group-delay measurement itself.

Frequency response and group delay #

Measure both magnitude and phase across the actual occupied band, including transition margin. A VNA estimates group delay from the phase difference over a finite frequency aperture:

$$\tau_g\approx-\frac{1}{2\pi}\frac{\Delta\phi}{\Delta f}.$$

A wider $\Delta f$ improves delay stability for a given phase-noise error but smooths narrow delay features; a smaller aperture improves frequency resolution but amplifies phase noise. Phase must be unwrapped before differentiation. Near a deep notch, low SNR and rapid phase rotation can make the numerical delay huge or meaningless, so inspect $|S_{21}|$ beside $\tau_g$ rather than reading delay alone.

Rohde & Schwarz documents a second trap: a multicarrier stimulus gives fast simultaneous group-delay measurement, but the DUT and instruments can create intermodulation products exactly on other carrier bins. What looks like delay ripple may then be nonlinear contamination. Repeat at lower total power or with stepped CW before blaming the channel.

Two-tone test #

Use equal tones, state their spacing and power per tone, and sweep level through the intended operating region. Verify that the sources, combiner, fixtures, and spectrum analyser have more linearity than the DUT. The close-in IM3 products give IP3; a separate single-tone gain sweep gives P1dB.

Tone spacing is part of the test condition. Closely spaced tones probe slow envelope variation; wider spacing excites different electrical and thermal memory. For a wideband PA, repeat with representative spacings and then with the real modulated waveform.

EVM #

For ideal complex symbols $s_k$ and aligned measured symbols $z_k$,

$$EVM_{rms}=\sqrt{\frac{\sum_k|z_k-s_k|^2}{\sum_k|s_k|^2}}.$$

EVM collects noise, residual frequency/phase error, I/Q imbalance, linear distortion left after allowed correction, clipping, and nonlinear memory into one in-band number. Error-vector magnitude versus time, phase versus magnitude, and the error-vector spectrum are more diagnostic than the scalar average.

The reference plane and allowed corrections are part of the definition. ETSI TS 138 104 V19.5.0 measures NR base-station EVM after the FFT and a zero-forcing per-subcarrier amplitude/phase equalizer. Its Release 19 limits for PDSCH are 17.5% for QPSK, 12.5% for 16-QAM, 8% for 64-QAM, 3.5% for 256-QAM, and 2.5% or 2.8% for 1024-QAM depending on whether frequency is at or below 4.2 GHz or above it. Those are compliance numbers for that defined procedure, not universal constellation thresholds.

Correct the mechanism #

Linear equalization #

If the path is stable and LTI, estimate $H(f)$ or its impulse response and apply an inverse-shaped filter at the transmitter or receiver. A linear equalizer can correct magnitude and phase together; an analog all-pass network is a narrower tool for group-delay shaping with ideally unchanged magnitude.

The literal zero-forcing inverse,

$$G_{ZF}(f)=\frac{1}{H(f)},$$

is dangerous near a channel null: it amplifies noise and model error without bound. An MMSE form regularizes the inversion,

$$G_{MMSE}(f)=\frac{H^*(f)}{|H(f)|^2+S_n(f)/S_x(f)},$$

and deliberately leaves residual distortion where exact inversion would cost more noise than it removes. Equalizer length must cover the significant channel impulse response; a short filter cannot undo a long echo.

Pre-emphasis spends transmit headroom to boost frequencies that the path attenuates. Receive equalization does not spend transmitter headroom but boosts in-band noise wherever it boosts the signal. Severe sampled channel memory belongs with the DFE or sequence-estimation methods described in symbol timing and ISI.

Nonlinear correction #

There are three practical levers:

1. Back off the drive. Peaks remain in the small-signal region, reducing compression and intermodulation at the cost of average RF power and efficiency.
2. Choose a more linear device or architecture. This moves P1dB and IP3 rather than trying to invert them later.
3. Predistort. Apply an approximate inverse ahead of the nonlinear device so that predistorter plus amplifier is close to linear.

A memoryless lookup table can invert stable AM-AM and AM-PM curves. A wideband PA generally needs a memory polynomial or GMP identified from a time-aligned observation of actual amplifier input and output. The observation receiver must sample enough bandwidth to represent the modeled nonlinear products, not merely the wanted channel. When the amplifier transfer changes, the model must be identified again or adapted; an inverse fitted to the old transfer is no longer its inverse.

In a transmitter, evaluate in-band EVM and out-of-band spectral regrowth together. A correction that makes constellation points look better while worsening adjacent-channel emissions is not a successful linearization.

Diagnostic signatures #

The shortest reliable workflow is: declare the signal band and reference plane → measure complex small-signal response → sweep power with one and two tones → measure the deployed waveform's EVM and adjacent-channel emissions → apply only the correction that matches the observed mechanism. Distortion is not one scalar defect; it is a family of transformations, and diagnosis succeeds when the test preserves that distinction.