Recording
Upon starting a race the server will add a fake "ghost" racer I call the VCR. This exists only on the server, the client is not aware of it. The VCR has the same interface as a regular client connection, i.e. you can write packets to it, and it will be sent packets about other racers as a normal client would, these packets include `createRemoteCharacter`, `beginRace`, position updates `p` and `exactPos`, and everything a normal racer would receive.
EXCEPT: it will not receive a `createLocalCharacter`packet, this is explained in the playback section.

The VCR will save all the packets received during the race with unix millisecond timestamps prepended (this may be better as a delta timestamp), and once all players have left the race it'll save the replay with a unique ID, I used course ID + version + timestamp as the ID.

Playback
To play back the recording the server will read each packet from the replay file and schedule them to be sent at the appropriate intervals according to the unix millisecond timestamp at the beginning of the packet. I omitted the first 3 packet parameters `timestamp`, `subHash`, and `sendNum` from the packets before sending, you may want to remove `subHash` and `sendNum` entirely from being recorded.
Rather than a `createLocalCharacter` packet, a replay spectator will receive a `createSpectator` packet.
I've made a git diff of the (small amount) of changes made to the client https://pastebin.com/6t3Mzn17
The server is completely custom and in Go, so I'm not sure how useful it will be to see.

Here's the raw replay recording from the video (i.e. what the server reads and parses and sends to the spectating client) https://pastebin.com/CdmT5M1Z.

A note about versioning:
Replay recordings may be saved with a copy of the level to avoid issues with different versions or the level being deleted entirely.
Game versions should not be an issue unless a part/item is removed from the game, which is unlikely.