Thanks. It ended up being a server side issue.
I couldn't install beta 5, and so I wanted to go down to 11.0.3. Well even though ipsw.me or iTunes itself would download the ipsw, it would fail the preparing stage.
So I went on Reddit and it was indeed a widespread issue with both the beta download and all restore procedures but not everyone was affected.
Lesson learned from this is be patient and don't rush into a restore if your beta update won't install and if a iTunes restore doesn't work, don't always panic and think it's your device(s)..
I'm now on 11.0.3 anyway and I'll stay there until 11.1 final.