I’m not one who stores a lot of images and videos on my iPhone. In fact, I am quick to transfer my photos to Microsoft’s OneDrive AND my 2 TB external hard drive. Anyway, my recommendation is to offload some of your photos or make use of several cloud storage sites instead of just one.
The thing with that option, although I don't disagree, is that albums like 'people / faces' will only go back as far as the photos on the service you're using. So say you archive pre 2016 images on one drive, pre 2010 on google and the most recent on iCloud, you'll never see the chance old pictures you stumble across now and again.
I personally don't see any problems with a big photo library - as long as you have enough memory left.
If you come close to your maximum memory capacity, your device might be slightly slower.
But if you want to clear your memory, there are lots of options - exporting photos/videos to a cloud (as mentioned).
What I do is, just exporting all of my pictures to my laptop - so I have all backed up and delete all pictures or videos I don't really need to carry around.