Welcome to iMore! That's odd. I have three AppleTVs in my home and don't have an issue with slow downloads on any of them. I might recommend restarting your AppleTV by unplugging it, waiting about 30 seconds, and then plugging it back in. If that doesn't work then a full restore might be in order. I provided a link (below) with details.
Also, make sure that your AppleTV has a good internet connection. I believe Speedtest.net's "Speed Test" app is available in the AppleTV App Store (for gen 4). If you're using wifi, can you switch to a wired connection? Is your AppleTV surrounded by electronics that could be blocking your wifi signal?