Copying from an ExFAT drive to Mac drive

I'm having a problem copying large AVCHD bundles from an ExFAT drive to a Mac Extended drive. In recent OS X versions the AVCHD structure is a bundle that appears like a single file. In this case I have one that's 36.5 GB. When I try to copy it to a Mac Extended drive it fails when it gets to copying this. Is this a known problem? If hardly ever used ExFAT drives, and I've not encountered this before.

Any light on this would be much appreciated.

Thanks.

Hi Tom, do you get an "error 36" when you try to copy?

- Ronny

I think that was the one. I did it yesterday on a different machine.

Could be a permissions problem on the Private folder.

When you connect the ExFAT drive to your system and you open FCP X > Import, can you create a Camera Archive from the AVCHD bundle? If that works I think that would be the easiest workaround.

- Ronny

Thanks Ronny. I'm able to archive and import from it without problem, just not copy.

Permissions for the bundle are listed as "You have custom access" and of course it can't be changed.

Thanks for confirming. The permission restrictions prevent the bundle from being copied over and unfortunately this cannot be changed. The archive and import option seems to override these restrictions.

- Ronny

Hi Tom and Ronny,
This is a common error going from Windows FAT to Mac disks.
You can use Terminal in several ways to overcome the issue. Sometimes it is with simple permissions and sometimes due to DS Store files.
Here is one way to use Terminal and the dot-clean command that I just ran across. Since they have kindly written it up, I won't.
osxdaily.com/2015/02/21/fix-error-code-36-finder-mac-os-x/
There are other ways to remove the DS. Store files as well.
Since you are already in Terminal, you can copy files via command line and this often works while drag and drop won't.
Hope this helps, Greg

Thanks Greg for following up on this.

Thank you, Greg. Very useful!

- Ronny

I have a side question, figured i'd ask now instead of making a new post for it, but are there any big drawbacks from having an external hard drive formatted in ExFAT to edit your projects from?

Huge drawbacks, if you're on a Mac system. FCPX doesn't like writing to FAT drives. You'll get bad performance, and wacky behaviors. Why would you want to use a FAT drive?

I work at an access center, and i'm trying to think of alternatives to make production a little faster. We have people shoot their shows inside our studio using the Newtek Tricaster system. However, Tricaster only runs on a windows OS, so all the files are saved on an NTFS drive inside the system.

We also have the citizens that use the facility edit on Premiere on iMac's. So we have external drives that are connected to those iMacs formatted as Mac Extended.

The issue comes with when they need to transfer files off the switcher and onto their editing computer. Since we can't connect their external drive to the windows computer since it won't recognize, we have to get an 'in between' drive, that is formatted as NTFS or ExFAT (we have several, some are just different. and we use ExFAT because the files are always to large for just regular FAT). After the files transfer to that 'in between' drive, we then have to connect it to their editing computer and transfer it to their external editing hard drive.

I just assumed working off an ExFAT drive would be bad, but i just wanted confirmation.

Reading from them is fine. There are writing issues. The Premiere folks can use regular Mac formatted drive, right. But yeah, I'd think you'd want two drives for the Tricaster stuff. One FAT one OS X. That is a dilemma...

bradywurtz wrote: ...I just assumed working off an ExFAT drive would be bad, but i just wanted confirmation.

I have over 100 terabytes of archival video material, much of which is on exFAT formatted drives. I've never had any problems reading or writing to those drives from a Mac due to the filesystem itself.

As the librarian for my video production group, I have done extensive performance and reliability testing on many different hard drives and disk formats. In general I haven't found any major issues with using exFAT on Macs. There might be a few rare "edge cases" where on a certain HDD or USB thumb drive HFS+ might be faster on some benchmarks than exFAT. However in general they perform about the same.

On an HDD you know will be Mac only, use HFS+. If there's a possibility the removable drive will be shared between Mac and Windows, use exFAT.

In general I'd suggest using portable bus-powered USB 3 hard drives for data transfer and archiving, not editing off of them (whether formatted HFS+ or exFAT). Many of them are too slow to edit from, although a few are considerably faster such as the HGST Touro S: www.touropro.com/en/product/touro-s/ There are also possible issues with spontaneous USB disconnects, which rarely happen on Thunderbolt drives.

Other filesystem options include Paragon NTFS on the Mac: www.paragon-software.com/home/ntfs-mac/
and HFS+ for Windows: www.paragon-software.com/home/hfs-windows/

HFS+ is a good file system but it is not without bugs. There is a particularly egregious stability and data integrity bug which was reported to Apple over a year ago and to my knowledge is still not fixed: macperformanceguide.com/blog/2014/201403...ericks-hang-bug.html

I have personal experience with it crashing my Mac and rendering it non-bootable.

Sounds like you've used the ExFAT drive though for archiving mostly. I know the general reading and writing off if it is probably fine, but I was talking more specifically reading and writing data while editing large HD video projects in NLE systems.

Sure, for archiving or just moving files, it's great, but i meant more specifically about reading and writing during editing. sorry if that wasn't clear. and sorry if maybe i didn't get from your post that you also use ExFAT formatted hard drives to do your primary editing off of while you work.

bradywurtz wrote: Sounds like you've used the ExFAT drive though for archiving mostly. I know the general reading and writing off if it is probably fine, but I was talking more specifically reading and writing data while editing large HD video projects in NLE systems...

I've used exFAT-formatted, AC-powered hard drives for editing, generally without problems. In general I prefer Thunderbolt or AC-powered USB 3 hard drives for editing, not bus-powered USB portable drives (whether formatted HFS+ or exFAT).

From the drive and filesystem standpoint there is generally no difference between I/O requests from Finder, from FCP X, from utilities, etc. The top side of the software stack is the API interface common to all apps and the bottom side passes the I/O requests to the device driver which in turn emits the commands to the device.

There is probably a greater difference in hard drive reliability than in filesystem used on those hard drives. E.g, you could theoretically have a Pegasus Thunderbolt RAID array formatted exFAT, since it's hardware RAID. I don't know why anyone would do that, but you could. Likewise you could have a USB 2 thumb stick formatted HFS+.

Just formatting as HFS+ some crumby little slow bus-powered USB portable drive doesn't magically transform it to being fast and reliable. Likewise formatting it as exFAT doesn't necessarily make it unusable for editing.

A potential problem with editing from a bus-powered USB hard drive (whether exFAT or HFS+ formatted) is back up. Where are your library files? Is anything being written to the portable drive that requires backup? If the content on the portable drive is read only, you are editing *from* that drive, not editing *on* it. In that case maybe it's not so bad.

If you require Windows/Mac data interchange on removable media, exFAT is a good file system for this. You can edit from an exFAT formatted HDD but the optimal practice is edit from an AC-powered Thunderbolt or USB 3 drive. Those tend to be HFS+ formatted -- not because exFAT won't work but they are typically dedicated and persistently connected to a Mac.

I see editors passing around media on bus-powered USB 3 exFAT portable drives and actually editing from those. In general I don't think that's the best approach but that's my opinion. The main problem isn't the disk format it's the speed and reliability of the HDD and interface.

So there is literally no difference between HFS+ formatted drives, NTFS, or exFAT drives other than the fact that HFS+ is not recognizable when plugged into an NTFS formatted computer, NTFS is read only when plugged into an HFS+ formatted computer, and an exFAT works perfect on all NTFS and HFS+ formatted computers?

So why not have everything you ever own or have, external hard drive speaking, be formatted in exFAT?

I feel like there has to be a reason. There has to be some performance issue or something or every external hard drive ever made would just come pre-formatted as ExFAT with a disclaimer on how it won't work with older OS's

There's one good reason at the start of this thread. ExFAT permissions are a major pain.

bradywurtz wrote: So there is literally no difference between HFS+ formatted drives, NTFS, or exFAT drives...So why not have everything you ever own or have, external hard drive speaking, be formatted in exFAT?...I feel like there has to be a reason. There has to be some performance issue or something or every external hard drive ever made would just come pre-formatted as ExFAT with a disclaimer on how it won't work with older OS's

There is no difference from the application interface to the file system API -- there's no separate API for doing I/O to exFAT vs NTFS vs HFS+. In the performance testing I've done on various portable drives and USB sticks I haven't seen much performance difference. However there are definitely differences in each file system's physical layout on disk, algorithms used to perform I/O, manage space, etc.

E.g, NTFS and HFS+ are journaling or transactional file systems so in theory metadata writes are logged and only persistently committed to disk at successful completion. This is for metadata (file create, delete, rename, index split, etc) not for data writes. In theory this provides better integrity if the system fails or crashes during a write. The degree to which that actually helps in the real world is less clear. You typically still are best served to run CHKDSK or Disk Admin to check the drives.

NTFS and HFS+ have supposedly better free space logic and are less susceptible to fragmentation.

A good example of theory vs reality is the serious data corruption bug which to my knowledge still exists in HFS+. It only happens under high stress coupled with async process termination, but it should never happen and Apple should have fixed it as a priority 1 bug a year ago.

There is no reason I'm aware of new hard drives could not come pre-formatted as exFAT. In fact most cameras automatically format large SDXC cards exFAT.

If you need portable two-way data interchange between Mac and unspecified Windows PCs, exFAT is good. Alternatives would include formatting them NTFS and using Paragon on the Mac, or formatting them HFS+ and using Paragon HFS+ on the Windows PC (assuming the PCs are all identified and unchanging).

If the portable drive is 100% dedicated to Macs you can just use HFS+. Is it better than exFAT? In my performance tests there's generally little difference. Is the data integrity better? Despite the above HFS+ bug, I would generally think HFS+ is a little better but this is hard to substantiate. I format Mac-dedicated portable drives HFS+ where data interchange isn't an issue.

As Tom said there could be permissions advantages to HFS+ but I've seen more permissions problems copying files across the network between Mac and Windows than I've seen from exFAT drives.

Consider also that FCPX uses soft and hard "links" when copying, moving and consolidating media. That is purely a UNIX thing that FAT/NTFS drives don't support. So there are things that will cause speed bumps and brick walls, eventually.