If I'm hooking up an audio accessory, I should wire it to the ignition wire and not the power one, right?

The accessory in question is an FM modulator, like what satellite radios use.