Meatwad It's complex but yeah first generation migrants are often politically conservative and "pull the drawbridge up behind them" as it's sometimes called.
If you want the simplest possible interpretation of the No vote on this change, I'd say it's that Australia is a constitutively white supremacist state in the sense that exclusion and dispossession of its natural resources were two of the main reasons Federation happened. Its unconsidered political defaults are to reinforce that enclosure of value.