I don't think you're missing anything. It seems wrought with problems There are some "fixes" posted on Apple forum. Disclaimer: I don't use outlook invites or calendar on my phone. I sync through Google. But I hear your frustration for a solution. Hope this helps...
https://discussions.apple.com/thread/5672404?start=120&tstart=0
Thanks, so I'm not crazy, don't know why people keep telling me, I can do xyz when clearly can not.