Biggest improvement on the Android side has without a doubt been the ARTS runtime where the Java bytecode of Android apks gets precompiled to machine code.
Made every subsequent Android phone at least twice as "snappy"—app launch and context switching got much much faster.
Other than that, the key is to stick to Google's devices and avoid any of the vendor window dressing from HTC and Samsung.
Getting a new phone always feels like a massive upgrade to me because it doesn't have a shit, nearly spent battery when it comes out of the box. I'm always like "wow, it lasts ten hours" 😃