@pendell Ages ago I used to have this problem with a built-in headphone jack and one specific headset; I think it's picking up a pulse that it's interpreting as "headset button" which modern android defaults to "invoke voice assistant"
@pendell In my case the problem was a wire inside the cable was loose and kept producing just the right electrical signal to be detected as a button-press. I've also seen that happen if the 3.5mm isn't plugged in tightly, but again, my experience is ages-old now (talking like iPhone 3G and Android 2.x era).
If your headphone cable is in good shape, and you're sure it's plugged in securely, only other thing I can think is make sure the 3.5mm is snug in the adapter before you plug in the USB end.