Coherent Optics — the DSP era
For decades, going faster or farther meant fighting physics in the glass: dispersion-compensating fiber, tight power control, regenerators. Coherent optics changed the game by capturing the full light signal and cleaning it up in a silicon chip. Understand coherent detection and its trade-offs and you understand why modern optical networks look the way they do — and why a coherent pluggable can now live right inside a router.
A traditional direct-detect receiver only sees how bright the light is (on/off). A coherent receiver sees the light's full field — brightness, phase, and both polarizations — by mixing the incoming signal with a clean reference laser. That richer picture lets a DSP (digital signal processor) mathematically reverse the fiber's distortions after the fact. Coherent optics moved the hard work out of the glass (DCMs, careful spans) and into silicon (DSP).
How coherent detection works
Two ingredients make it coherent:
1. A local-oscillator laser
At the receiver, a clean local-oscillator (LO) laser is mixed with the incoming signal. Where the two line up in phase they reinforce; where they don't, they cancel. This "coherent mixing" recovers not just the signal's amplitude but its phase and polarization — information a simple photodetector throws away.
2. A DSP that undoes the fiber
The recovered field is digitized and fed to a DSP. In software it reverses chromatic dispersion, tracks and undoes PMD, corrects phase noise, separates the two polarizations, and runs the FEC decoder. Impairments that used to need physical compensation are now just math on samples.
Because the DSP compensates CD and PMD electronically, DCMs disappear from the line system, spans can be engineered more simply, and one transponder adapts to whatever the fiber presents. The same coherent engine also enables higher-order modulation (packing more bits per symbol) — because it can track phase precisely enough to tell dense signal points apart.
Modulation formats — packing more bits per symbol
Coherent systems encode data in symbols — distinct combinations of phase and amplitude. The more distinct points a format uses (its constellation), the more bits ride each symbol — but the points sit closer together, so they need a higher OSNR to tell apart, which means shorter reach. And coherent uses both polarizations (dual-pol, "DP-"), doubling the bits carried.
| Format | Bits / symbol / polarization | With dual-pol (×2) | OSNR need | Relative reach (illustrative) |
|---|---|---|---|---|
| DP-QPSK | 2 | 4 bits/symbol | Lowest | Longest — long-haul workhorse |
| DP-8QAM | 3 | 6 bits/symbol | Higher | Medium — regional |
| DP-16QAM | 4 | 8 bits/symbol | Higher still | Shorter — metro / DCI |
| DP-64QAM | 6 | 12 bits/symbol | Highest | Shortest — very short, high-OSNR links |
The net-rate formula — worked
How much data does a coherent wavelength carry? Multiply the symbol rate by the bits each symbol carries across both polarizations, then remove the FEC overhead (the redundant correction bits are not customer payload):
The lever is visible: to raise capacity you either spin the laser faster (more baud) or pack more bits per symbol (denser modulation). More baud needs more spectrum; denser modulation needs more OSNR. Both cost you something — capacity is never free.
SD-FEC vs HD-FEC
Every coherent link runs FEC, but the kind matters. Hard-Decision FEC (HD-FEC) first rounds each received bit to a firm 0 or 1, then corrects — throwing away the "how sure am I?" information. Soft-Decision FEC (SD-FEC) keeps that confidence value and feeds it to the decoder, which corrects far more effectively.
| HD-FEC | SD-FEC | |
|---|---|---|
| Coding gain (illustrative) | ~6 dB | ~11 dB+ |
| Tolerable pre-FEC BER | Modest | Very high (up to ~2×10⁻²) |
| What it buys | Some reach | Much more reach / denser format at same reach |
| Cost | Lower overhead / complexity | More overhead + DSP power, but far more capable |
Because every dB of coding gain lowers the required OSNR by about a dB, SD-FEC's ~5 dB advantage over HD-FEC translates into a large amount of extra reach (recall: doubling spans costs only ~3 dB of OSNR). SD-FEC is the standard for modern long-haul coherent.
Inside the DSP — the coherent pipeline
The DSP is where coherent optics actually earns its keep. Once the receiver has digitized the full field (in-phase and quadrature components of both polarizations), a fixed sequence of processing stages cleans it up. The order matters: the biggest, most predictable distortion is removed first with a static filter, then the time-varying effects are tracked, then the laser imperfections are corrected, and only then are bits decided and error-corrected.
Why CD first, with a fixed filter
Chromatic dispersion depends only on how much fiber the light crossed — it does not wander second to second. That makes it the ideal thing to remove with a single fixed FIR filter computed once for the span length. Doing it first also shrinks the pulse spreading before the adaptive stages, so they have an easier job.
Why the equalizer must adapt
PMD and polarization rotation do drift — with temperature, wind on aerial fiber, people touching patch panels. So the adaptive equalizer (butterfly structure, CMA/MMA algorithms) keeps re-tuning itself and untangles the two polarizations that the fiber has scrambled together. This is the stage that "tracks the fiber" in real time.
Why carrier recovery comes next
The transmit laser and the LO laser are two independent light sources, so their frequencies never match exactly and their phases drift (phase noise). Frequency recovery removes the colour offset; carrier-phase recovery then stops the constellation from rotating so symbols land where the decision logic expects them.
Then decide, then FEC
Only once the field is equalized and de-rotated can each sample be mapped to its nearest constellation point (symbol decision). Those raw bits still carry some errors, so FEC decode uses the redundant bits to correct them — and, with SD-FEC, uses the receiver's confidence values to correct far more.
Every impairment that used to need glass — dispersion-compensating modules, careful PMD-managed spans, phase-stable everything — is a stage in this pipeline now. Deterministic distortion (CD) gets a fixed filter; time-varying distortion (PMD, rotation) gets an adaptive one; laser imperfection (frequency, phase) gets carrier recovery. That is the whole reason a coherent line system can be simple: the complexity moved into these silicon stages.
Pulse shaping — Nyquist and roll-off
A coherent transmitter does not send abrupt square pulses; it shapes them so each symbol occupies as little spectrum as possible without smearing into its neighbours. The standard tool is Nyquist / root-raised-cosine (RRC) filtering, controlled by a single knob: the roll-off factor β (typically ~0.05–0.2 on modern coherent). The tighter the roll-off, the more rectangular ("brick-wall") the spectrum and the narrower the signal.
This is the direct link between the time domain (symbol rate) and the frequency domain (how wide a channel slot the signal needs). It is why the same baud rate can be "spectrally efficient" or "wasteful" depending purely on the shaping filter — and why flexgrid slot widths (75 GHz, 100 GHz, 150 GHz) are chosen to hold a target baud rate plus a little guard band.
Baud-rate growth and probabilistic constellation shaping
There are two independent levers for more capacity per carrier: spin the symbols faster (more baud) and shape the constellation smarter (PCS). They stack on top of the modulation-format choice above.
Baud-rate growth
Each generation of coherent DSP runs a higher symbol rate — roughly ~64 → ~96 → ~128 Gbaud across recent generations (illustrative). More baud means more bits per second on the same wavelength and the same modulation format. The cost is spectrum: at β ≈ 0.1, a ~96 Gbaud carrier occupies ~96 × 1.1 ≈ 106 GHz and a ~128 Gbaud carrier ~128 × 1.1 ≈ 141 GHz — so the higher-baud carriers need wider flexgrid slots (e.g. 150 GHz) to fit.
Probabilistic constellation shaping (PCS)
A plain constellation uses every point equally often. PCS biases the transmitter toward the lower-energy (inner) constellation points and uses the high-energy outer points more rarely. This does two things: it approaches the Shannon limit (the theoretical capacity ceiling of the channel) more closely than a uniform constellation, and it gives fine, near-continuous rate/reach granularity — instead of jumping QPSK → 8QAM → 16QAM in big steps, you can dial the shaping to hit almost any capacity, tuning the carrier to the exact reach the link can support.
Without PCS you pick from a handful of discrete formats and often leave margin (and capacity) on the table — the link could carry a bit more than QPSK but not enough for 16QAM, so you run QPSK and waste the gap. PCS lets you tune capacity almost continuously to match the OSNR the link actually delivers, squeezing out that wasted gap. It is the software equivalent of a dimmer switch where you used to have only a few fixed brightness settings.
400ZR and OpenZR+ — coherent in a plug
The biggest shift coherent enabled is shrinking a whole transponder into a pluggable module the size of a normal router optic. Two industry specifications define these:
400ZR (OIF)
An OIF-standardized coherent pluggable carrying 400G over an amplified link. Targeted at data-center interconnect (DCI) at roughly ~120 km amplified reach. It uses a fixed, interoperable configuration (DP-16QAM-class) so modules from different makers work together. Form factor: QSFP-DD or OSFP — the same slots a router uses for grey 400G optics.
OpenZR+ (industry)
An industry superset that extends reach and adds flexibility beyond 400ZR — multiple line rates (100G/200G/300G/400G), stronger FEC, and selectable modulation so it can reach farther (lower the rate to go longer). Same pluggable form factors. Think of ZR+ as "400ZR's longer-reach, more-configurable sibling."
Traditionally the router handed a grey signal to a separate transponder, which produced the coloured DWDM line wavelength. A ZR/ZR+ pluggable puts the coherent line laser and DSP directly in the router port — so the router emits the coloured wavelength itself and the standalone transponder collapses away. Fewer boxes, less power, less rack space. The router is the transponder.
Alien wavelengths and the optical line system
When a router's ZR pluggable emits its own coloured wavelength and that wavelength is carried across a third-party optical line system (OLS) — mux, amplifiers, ROADMs the pluggable's vendor didn't build — that line signal is an alien wavelength. The OLS amplifies and switches the light transparently; it never terminates or decodes it.
Because the line system cannot see inside an alien ZR wavelength, OSNR budgeting, reach validation, and fault correlation fall on the router/pluggable owner, not the line vendor's automated planner. You gain simplicity and box-count savings, but you take on the optical engineering the transponder vendor used to own. Confirm who owns OSNR/reach at the demarc before deploying router-integrated coherent across someone else's line.
IPoDWDM vs the transponder — and open line systems
The ZR pluggable creates a genuine architectural fork. In IPoDWDM ("IP over DWDM"), the coherent line optic lives inside the router as a 400ZR / ZR+ pluggable, and the router itself puts the coloured wavelength on the fibre. In the transponder model, the router hands a grey client signal to a separate optical shelf that produces the wavelength. Both work; they trade different things.
| Aspect | IPoDWDM (router ZR/ZR+ pluggable) | External transponder shelf |
|---|---|---|
| Boxes / power / rack | Fewer — the transponder collapses into the router port | More — a separate shelf, its own power and space |
| Cost per port | Generally lower (one device, one optic) | Higher (router optic + transponder + its optics) |
| Who owns the optical layer | The router / IP team inherits OSNR, reach, alien-λ planning | The transport / optical team keeps it, as before |
| Operational separation | Blurred — IP and optical faults land on one team | Clean — clear demarc between IP and optical domains |
| Reach & flexibility | Bounded by the pluggable (400ZR ~DCI; ZR+ farther) | Broader — transponders offer more formats / longer reach |
| Alien-wavelength management | Yours to own if it crosses a third-party line system | Typically managed within the vendor's own line system |
An open line system deliberately separates the terminals (transponders or router ZR/ZR+ pluggables) from the line system (mux/demux, amplifiers, ROADMs). Instead of one vendor's turnkey end-to-end system, you can put one vendor's pluggables/transponders onto another vendor's amplified line. That is exactly what makes router-integrated coherent practical at scale — but it is also what pushes the OSNR and reach engineering onto whoever owns the terminals, because the open line no longer bundles that planning for you. Open line + IPoDWDM is powerful and flexible; just be explicit about who owns the optical maths at the demarc.
Coherent moved the hard work from glass (DCMs and careful spans) into silicon (DSP) — and once the hard work was in a chip, that chip could shrink into a pluggable and move into the router. Dispersion compensation, high-order modulation, and router-integrated DWDM are all consequences of that single shift.
1. Why does DP-16QAM carry more capacity than DP-QPSK but reach less far?
2. Estimate the gross line rate of a ~60 Gbaud DP-16QAM signal, and why the net payload is lower.
3. What physically happens to the standalone transponder when you deploy a 400ZR pluggable in a router?
4. A ZR wavelength from your router crosses a third-party optical line system. What is that called and who owns OSNR planning?
5. Why is SD-FEC preferred over HD-FEC for long-haul coherent?
6. Why do coherent line systems omit DCMs?
7. In the DSP pipeline, why is chromatic dispersion removed by a fixed FIR filter while polarization effects need an adaptive equalizer?
8. A ~64 Gbaud carrier uses root-raised-cosine shaping at roll-off β ≈ 0.1. Estimate its occupied bandwidth and say whether it fits a 75 GHz slot.
9. What does probabilistic constellation shaping (PCS) buy you beyond simply picking QPSK, 8QAM or 16QAM?
10. Name one thing you gain and one thing you take on by choosing IPoDWDM (router ZR/ZR+ pluggable) over an external transponder shelf.