I've always loved the Bauhaus live collection PRESS THE EJECT AND GIVE ME THE TAPE.
Especially the versions of "Rosegarden Funeral of Sores" and "Terror Couple Kill Colonel".
[video=vimeo]
Not to be a complete dick about it, but I find their studio recordings a fraction insipid. I've just never loved them as much. The live anthology is wonderfully raw and showcases their sharp edges.
If you do like Bauhaus then Japan might be worth checking out. Art Goth to Proto-New Romantic, yeah, but with surprisingly similar tones and influences. Murphy ended up getting way more like Sylvian in some of his solo work.