Here is a video i made re-producing the issue by simply moving a stack within a players pack over and over. The further from the players parent pack the item is "container in container" the worse the ping spikes become.
This looks and sounds amazing, i love that you buried many of the aspects of the towns and areas instead of just wiping the map clean. Gives a nice post-apocalyptic feel to the scenery love it! I could see camping & cooking skills having serious bonuses here this looks like an adventurers dream.