Do I have Ground Loop?

I am having issues with what I think is ground loop. When I have the system connected to a powersupply there is noise on the audio from accessing the SD Card and the WiFi dongle. This is how I have it wired at the moment.

If I connect the Pi to a separate 5V PSU I have none of the issues with the audio. Anyone have any idea how I can isolate the grounds to stop the feedback?