In replays, the map size only seems to change when the player actually changes the map size (by opening the map or leaving it) but NOT between rounds or when you switch to the other player.

For example here: https://www.geoguessr.com/duels/692a1d666052e10dfb6b2672/replay

If I watch the first round from the first player until the end or at least until he has fully opened his map, if I then want to go back a few seconds to check again what he saw before or switch to the next round or to the other player, the map just stays fully open and I can barely see anything of the round, so I have to refresh the page all the time to get rid of the huge map.