The headlines in short are don't buy a new Mac with a fusion drive if you're going to use it for audio. If you already have, and are experiencing the widely reported problems, there's a fix that has worked for me.
I bought a new late-2013 iMac with 3TB fusion drive to be the centre of a home recording setup. It is running Mavericks 10.9. However, with two different interfaces (Focusrite Scarlett 18i20 and Motu 896 mk3 Hybrid) a) all recordings in Logic would descend into a crackling mess after 5-10 seconds and eventually no sound would be recorded, and b) any audio played back in logic, iTunes, or through youtube would eventually do the same. Having updated the drivers on everything, tried every different USB and Thunderbolt port, including a USB hub, removed all hard drives, updated all the software and reinstalled Logic, the problem still persisted.
All the interfaces worked fine on a MB Air, and on my old (2009) iMac.
My local Apple service centre said that they couldn't see any reason why it would be the fusion drive, and thought it could be the USB bus or the interface. As I'd tried two different interfaces by different manufacturers, on FW/thunderbolt and USB, and my other peripherals were working fine, this seemed unlikely.
People on other forums had potentially tracked the problem down to the fusion drive, and many people seem to have returned their iMacs to have the drive replaced with a conventional drive. Another potential solution seemed to be to split the Fusion Drive into its constituent 128 GB flash drive and 3TB conventional drive. After a couple of weeks of frustration
Having done a full backup with time machine to a clean external drive, I tried splitting the fusion drive using the method explained here http://www.macworld.com/article/2015664/how-to-split-up-a-fusion-drive.html and, despite it being pretty nerve-wracking, 2 days in it seems to have worked fine. I also have the additional bonus of a spare internal flash drive as I restored to the conventional drive.
I don't know if this is reversible, so proceed with extreme caution, and make sure you have proper backups. I made sure i had a backup on a networked Time Machine drive and a second on a USB3 drive. I also unplugged everything from the iMac before starting. The process took 20 minutes, plus about 3 hours to restore the backup.
If you have this problem, this fix may work for you, but note that I have no idea whether or not it affects the AppleCare warranty. If you don't, and are thinking of buying a new mac, I'd advise avoiding getting a Fusion drive.