I think it's less about how you treat referees and more about how you treat people. If it was directed at a player or manager, it should have the same consequences in my book. It is a professional environment and can be legislated as such.
For what it's worth, expressing frustration and using foul language is really not a big deal to me, it's the specific kind of language and the clear intention to verbally brutalize someone that seems inappropriate.
