Great question. I use wifi all the time at home and at work. 500 meg fiber connection at both places (up and down) so there's never a "heavier time than others" issue. But that'e just my situation.
As for you, I don't think we can answer that question since we have no idea where you are, what ISP service you have nor what speeds you are getting.
I have only noticed a problem on my home wifi with my older devices. Late afternoon is when the kids get home from school, of course they’re all doing their homework !!
How congested is the channel you are using? If you are in a large apartment complex with several others around, the one channel just might be over used.