You can imagine a rule where only the player making the pass from an offside position (byline cross, cutback, corner etc) is permitted to be "inactive" and only if they don't touch a return ball, touch another player, or impede any player while they remain in an offside position.
But it's a bit of a can of worms isn't it.