Without the failed test (although he did actually fail a test once, but that's a story in itself) any evidence against him is going to be circumstantial. However, the weight of evidence is huge. It's not just disgruntled team mates and jealous rivals who have accused him of doping.
To me, the idea that he doped isn't a debate. He did dope, but so did the vast majority of the peleton. So the interesting debate is where that leaves his legacy?