See this replay:

Rounds 2-5 don't load the correct round. For round 2, the same round as round 1 is loaded.

I don't see this problem in most other challenge replays. Since I know that the map used for this challenge does not pano ID the majority of the locations, one hypothesis is that the round doesn't load correctly because the replay function doesn't handle this case properly.