Recovering lost packets in remote play with nanors

Remote play gives each media frame a short window to arrive before playback. When packets go missing, the receiver needs a way to recover their contents while that frame is still useful. Waiting for the host to send the missing data again can take too long.

In the popular remote play systems Sunshine and Moonlight, nanors provides an efficient implementation of that recovery mechanism. The host sends repair packets alongside the original data, giving the client enough information to reconstruct a limited number of missing packets without requesting another transmission. This is forward error correction, or FEC.

A repair budget for every frame

The target bitrate is chosen in advance, with capacity reserved for repair data. Once a sustainable streaming rate has been selected, more protection means allocating less capacity to the encoded media.

Sunshine makes these allowances when Moonlight supplies the configured bitrate. It adjusts the video encoding rate for the FEC setting, then reserves capacity for audio and transport overhead. Caps and packet rounding make this a practical allocation rather than an exact bandwidth limit. ¹

For illustration, suppose 24 Mb/s remains for video and repair data after those other allowances. Encoding video at 20 Mb/s leaves 4 Mb/s for repair: 20% redundancy relative to the video payload, or one-sixth of the combined budget. These figures illustrate the tradeoff; they are not Sunshine configuration values.

At the packet level, the same ratio could be represented by 20 source packets and four repair packets of equal size. Reed–Solomon coding lets the receiver reconstruct the source block from any 20 of those 24 distinct, intact packets. Up to four can be lost, whether they carry source or repair data, provided the rest arrive in time. This exact recovery threshold makes a small repair budget useful. ²

The sender encodes each block once, sends its source and repair packets, then moves on. Sunshine divides an encoded video frame into a bounded number of FEC blocks and chooses the source and repair counts before encoding each one. Those counts can vary with frame size, but each block has a predetermined repair allowance. ³

This is a good fit for remote play’s short deadlines and continuous supply of new frames. Rateless codes can also work within a fixed budget, but their ability to generate further repair data offers less value once a frame’s transmission allowance and deadline are set.

Decoding guarantees also differ: some rateless codes may need extra packets beyond the source count, while Reed–Solomon’s maximum distance separable (MDS) property guarantees recovery at that count. ² With nanors, the sender calculates the chosen repair packets directly from the source block, and the receiver knows exactly how many intact packets it needs.

The tradeoff remains explicit. Losses beyond the allowance can make a block unrecoverable. Increasing protection requires either a lower video bitrate or more available bandwidth, and the application must still pace transmission to avoid congestion.

Reducing the work behind recovery

The host generates repair data for every protected block, so encoding efficiency matters throughout a streaming session. The Cauchy Reed–Solomon implementation in nanors accelerates the arithmetic that combines source data into repair packets and reconstructs missing data.

Sunshine adopted nanors in January 2023, replacing its previous Reed–Solomon implementation. ⁴ Later work expanded its use of CPU vector instructions, which process multiple data elements at once. In a 2024 comparison on a Ryzen 5950X, a Sunshine contributor reported roughly twice the Reed–Solomon performance with AVX2 as with SSSE3, and four times the performance with SSSE3 as with scalar C. ⁵

On the client, the value of faster recovery depends on network conditions. Moonlight’s normal video path can use a complete set of source packets directly. It needs Reed–Solomon decoding when source packets are missing, at which point recovery competes with the frame’s remaining time budget. Moonlight adopted nanors in its shared client library in February 2026, with the limited CPU capacity of older Xbox One hardware a principal motivation. ⁶

Improvement through collaboration

The work has moved in both directions. During Sunshine’s original integration, a contributor added support for small frames that needed more repair packets than source packets, then submitted the change and its tests to nanors. Once it was accepted, Sunshine used the upstream version. ⁴ Moonlight later adapted Sunshine’s optimized integration for its clients. ⁶ A focused library gave both projects something they could reuse and improve together.

Through the Sunshine/Moonlight ecosystem, nanors has been integrated into many other remote play systems, such as Apollo, Helios and Wolf, ⁷ Razer’s Cortex PC Remote Play Host, ⁸ and Tencent’s Polarlight Sunshine host. ⁹ Moonlight’s desktop, Android, iOS/tvOS, Xbox and webOS clients also use nanors through their shared core. ¹⁰

This is where projects like nanors fit best: a focused building block for fast, efficient data recovery. Developers can choose the protection their application can afford, inspect how it works, and contribute improvements that others can use. The source and build instructions are available on GitHub.

References