The referees and the implementation are absolutely the problem but we can keep saying that until we're blue in the face, nothing will be done about it. Until the PL and other bodies get their act together and hold the PGMOL to account, nothing will change.
In the mean time, limit VARs powers so that they don't actively ruin the game as a spectacle. Not being able to celebrate a goal because you don't know what imagined infringement some fucker in a room somewhere is going to dream up is one of the worst things to ever happen to the game.