Different audio outputs - different audio delay figures?

[hyperjase]

Is it possible to vary the audio delay based on output, finding using HDMI over Cat5e, I have around 300ms delay, but on the TV it is by, without any delay, it's fine. I need to alter the HDMI audio and leave the Optical as it is.

Thanks