APFS

APFS

I want to start a discussion about Apple’s new file system APFS.
APFS is the first time all Apple devices will share common code between Apple devices, so this should allow greater interaction between devices and apps.
APFS will be introduced in High Sierra coming soon. I am highly supportive of a new file system and APFS has many features that I really like. It is unclear how Apple will integrate and change apps to take advantage of some of the features that APFS will contain, but looking forward to all the possibilities. Having said this, I have some concerns about the change.

Users should exercise some real caution when it comes to updating their main computers. It is also important to understand some “limitations” of implementing APFS.

First, if you upgrade to High Sierra and you have a computer that the internal main drive is a solid state flash drive, then your drive will be converted. There is no “opt out” for flash drives. Not so with “Fusion” drives or spinning hard drives(HDD). For Fusion drives and HDD’s, APFS is apparently optional.
This is a quote from an Apple KB… “When you install macOS High Sierra on the built-in solid-state drive (SSD) of a Mac, that drive is automatically converted to APFS. Fusion Drives and hard disk drives (HDDs) aren't converted. You can't opt out of the transition to APFS.”

Furthermore, it is interesting how Apple has limited what Macs can read/write to APFS drives…
“Devices formatted as Mac OS Extended (HFS+) can be read from and written to by devices formatted as APFS.
Devices formatted as APFS can be read from and written to by:
* Other devices formatted as APFS
* Devices formatted as Mac OS Extended, if using macOS High Sierra
For example, a USB storage device formatted as APFS can be read by a Mac using High Sierra, but not by a Mac using Sierra or earlier.”

Okay, I think this is going to cause issues. If you have drives formatted with HFS+(current Apple Extended {Journaled}) and you want to read and write to a them with APFS no problems. For folks that keep older OSs on external drives to use older hardware/software, then you cannot boot into say Sierra or earlier and read or write to a drive formatted with APFS. It does not appear that Apple plans on providing a way for older systems prior to High Sierra to interact with APFS formatted drives, Big WTF, this will cause major issues IMHO. You need to think carefully about formatting.
What also isn’t clear, is how files created in APFS will be sharable with older systems. Can you export files to an external HFS+ device and read write when using an older computer/OS? Here I am just talking about the files themselves.
Will there be limits/issues?
One possible issue is in naming files. HFS+ and APFS have some differences in naming conventions and UTC coding. While Apple has some work arounds for this, I think there will be some future problems and apparently when it occurs the only message you will get is basically a “404” “File not Found” pop-up. Figuring out why, is up to the user and deciphering that a file contains characters that are no longer supported and therefore not readable is going to be fun!

Speaking about file sharing, AFP will no longer be supported for share points. APFS will depend on SMB or NFS protocols for networks. SMB is the preferred protocol it seems.
I hope Apple ups their game on supporting NTFS immensely. NTFS on Macs is mostly a third party dependent support feature that has lots of issues. Will NTFS finally work on Macs as well as HFS+ formatted drives?? Hope so!
Will be interesting how various networking systems implement APFS. Will NASs with multiple macs be able to format to APFS and be accessible? Don’t think so, therefore, they may have to remain on older formats for older operating systems in the network or upgrade all Macs to High Sierra(???) That will be a real shame as APFS has some very nice features for file sharing/storage.

Installing High Sierra.

High Sierra (HS) will require a firmware update to all devices. The firmware updates are Mac/device specific and so at some point you must connect to Apple servers to have the specific firmware installed. Installing from a thumb drive or via network administration will require workarounds, mainly the device needs to be connected to the internet to get the firmware.
One question I have not seen answered is whether or not you can run older OSs from a Mac that has the firmware update installed. I think so, but can’t confirm it. Has anyone tried to boot into older system after High Sierra APFS install?? What if the older OS is on a partition of internal flash drive?

You can upgrade automatically to High Sierra and all your data will remain intact. But if you then decide to revert to an older non APFS OS, the installer will WIPE your Drive! Beware!
Here you really want to make a bootable clone before you upgrade, so if you want to go back to Sierra you can without too much pain. OK you can upgrade but not downgrade without losing data.

I always do a clone and a couple of other back-ups before doing a “Clean” install. I do not use Migration Assistant! Not ever for major OS updates. After installing OS, I go to the App Store and install purchased apps I still use with a fresh download, then I copy and paste any files I need from old drives. This makes a much better install, less problems and easier troubleshooting for any issues. It also saves me lots of space.
I do recommend you take the opportunity to “Archive” old emails, remove old messages from iCloud etc… to keep them from downloading and taking up space. I include archiving my gmail on Google's servers to avoid large downloads. I archive everything that is older than a year or so.
I keep my photos, videos, LPX sound files on external devices, so I am spared that pain and keep to free space on internal drive.
This is also a good time to change login passwords as you set up fresh.

So those are just some of my concerns and I look forward to comments and experiences from others.

There are lots of things to like about APFS and I do like what I see.

Dynamic sizing for partition volumes on drives. Any partition can expand to all available space on a drive as needed. No need to re-partition.
What Apple calls “Clones” and “Snapshots” for files. Clones are in essence like auto saves in FCPX where only changes are recorded not file duplicates, this saves space on your drive. Furthermore, Apple is implementing “Copy on Write” where files are directly written as appendages/delta changes and keeps old data, this provides a vastly more robust file crash protection from things like power interruption.
Snapshots are time locked and won’t change unless you specifically launch them and make changes.
Super fast copying and file updating and info.
Sparse image support with dynamic sizing.
New system wide and file specific encryption.

See these articles… support.apple.com/en-us/HT208018
developer.apple.com/library/content/docu...DontLinkElementID_19

So what do you think?

Hope this helps, Greg

The only really serious concerns I see are for those with 1- very specialized setups, 2- networks, 3- anyone running an OS version older than Sierra. And, no one every recommended an OS upgrade until all your apps and add-ons have been verified by the developer to be compatible. That's not exclusive to High Sierra. Don't upgrade your OS right away, ever. Unless you have legitimate reason to.

I've been running HS developer beta from the beginning on a rMBP, using a portable SSD drive, a portable spinning hard drive, and a P2 RAID5 on occasion. None has been reformatted by HS, just left as is (why would I reformat something that's already formatted) and I've had no issues.

Disk Warrior has already been updated to be compatible with HS.
FCPX is running just fine in HS, as are CoreMelt 'sadd-ons. I don't use a lot of 3rd party stuff aside from MotionVFX templates (a broadcaster's best friend).
Pixelmator seems fine.
Motion and Compressor seem fine.
But that's just my rMBP, I'd not use that as proof you can upgrade immediately without issues. I'm only running it to help a couple of developers beta test on it.

Again, I've never recommend anyone upgrade an OS (HS or other) until everything you use has been verified to be compatible.

All that aside, once you do safely move to HS, I believe you'll see noticeable performance and stability improvements.

Good references, VidGreg, thanks for posting them.

Hi Ben Thanks for your experiences
Yes I totally agree that folks should not update to new OS until all apps/plugins are updated and until the new version seems good to go. Don't update mid project! Speaking of which people should make sure that they do not have "Auto-Update" turned on in System Preferences>App Store>Update newly available updates in background turned on. Might get a surprise that way.

Not clear in your description where you installed High Sierra? Internal flash or external drive? AFAIK in all previous betas of High Sierra (HS) it was optional if the drive was reformatted to APFS. Latest GM candidate removes option, I think. Is your drive indeed formatted for APFS or is it still HFS+?
Also booting from a HS APFS drive would require a new firmware update.

AFAICT Disk Warrior is not fully HS compatible. Not listed on their site (Sierra latest). There are lots of sites listing issues with APFS and Disk Warrior. In fact you are the first post to say it works fine that I have found. Can you check that in fact DW working with APFS formatted drives? Thanks

Carbon Copy Cloner latest version for 4 and the new version 5 have been updated for HS. Version 5 will have some APFS specific features not to be found in vs4.

Not so with SuperDuper with APFS, and there is some interesting info on SuperDuper blog.… www.shirt-pocket.com/blog/index.php/comments/news_on_the_march
No firm date for full support for High Sierra with APFS formatting. For HS HFS+ Sept 25.
One interesting mention on this blog is that you cannot host a HFS+ partition on a APFS formatted drive, interesting and something I suspected. Also booting has not been documented. Think this is due to non-boot ability while under Sierra. Most likely firmware was not ready.

Disk Utility will get a big update. Many report issue with beta HS.

Apple will finally kill off legacy FCS and associated programs. This is expected but it should be pointed out that 32 bit programs will still work in HS, just that Apple has announced that will not be true for long. R.I.P. 32 bit apps.
Lots of under the cover code in Apple's own eco system is still 32bit, wonder if they will update and rewrite the code or replace it, or let it work as is??

I will definitely keep clones for current OS (EC). I have several audio interfaces that will need driver updates before I can use HS and some other software that I need. Still I am looking forward to HS. I avoided Sierra altogether, but at a good place to update soon.

Thanks for your input, be interested if your drive is formatted to APFS, available in DU or Get Info.
Greg

rMBP internal SSD.

VidGreg wrote: … should be pointed out that 32 bit programs will still work in HS, just that Apple has announced that will not be true for long. R.I.P. 32 bit apps.
Lots of under the cover code in Apple's own eco system is still 32bit, wonder if they will update and rewrite the code or replace it, or let it work as is??…

… out of curiosity, I checked my apps via System Reporter, just to find out Apple's own DVDplayer.app and actual Compressor (!) are 32bit …

Will be an interesting day, when updating: I 'upgraded' my historic 2012 Mini internally with some 3rd party SSD = no idea, if this will work. …

afasiu, ext. drives are not re-formated to APFS, am I right? Or, not 'automatically'??

murmuring 'backups, backups, backups' :lol:

And thanks, Greg for sharing your insights!

This is a VERY important topic that a lot of people won't know about until after they've upgraded. Might I suggest it's moved to the main FCPX forum?

Only 'boot' drive is APFS, external drives remain 'as is' until you choose. FCPX, Motion & Compressor run well on HS.

Compressor shows up as 32 bit because it can encode 32 bit codecs, but the app actually is 64 bit. I've had this conformed by an Apple engineer on several occasions for me. But Compressor runs just fine in HS.

FCPX.guru wrote: Compressor shows up as 32 bit because it can encode 32 bit codecs,….

What the heck is a '32bit codec' ??

I'm aware of 8, 10, 12 … but 32?

First I didn't mean to scare anyone regarding the 32bit app degradation that is just now beginning. Apple has stated that 32bits apps will run and be fully supported in HS.
No 32bit apps may be submitted to App Store starting in January and all updates must be 64bit starting in June of 2018. After HS, the following OS MAY not fully support 32bit apps. (Apple uses the term "compromises" {I think is the term they use}). So all 32bit apps are safe for now. Still I think iOS shows that Apple will move to end 32bits apps.

As to if the 32bit code inside Apple's own software will continue to be supported; who knows?? Tim hasn't told me
Apple has chosen to fully deprecate and not support legacy FCP starting with HS. I think this might also include Quick Time Player 7 as well, but do not know this for sure??
The writing in on the wall.

As far as Compressor goes, it still shows up as 32bit. Everything I have read says it is 32bit. System report and Activity Monitor all say it is 32bit and if it wasn't I think Apple would advertise it as 64bit and correct the info. AFAIK it is still based on older coding.

There is lots of code still 32bit inside Apple's program code.
Karsten, this includes almost everything having to do with optical disks. iDVD, DVD Player, Create Disk(Core software for creating optical discs code goes back to early 2000s) So I think Apple is about to just say no to optical discs altogether.

Lots of other apps, including most of Adobe and even CoreMelt's apps will have to be rewritten. I have lots of audio software that will be obsolete as well.

Karsten I think that as long as the internal boot drive is SSD then HS will convert the file system to APFS. Shouldn't matter if user installed or not, I think??? No opt out.
External drives whether SSD or spinning I think will be optional and you will have to manually format using Disk Utility. High Sierra is concerned with the boot drive. Don't know if you install it on an external SSD it will only format as APFS or not?? (I'm about to buy another external SSD so I might try installing HS on it and see it it gives me a choice).
See my cautions previous posts.

If we boot into an older OS with HFS+, will we be able to pull files off APFS drives? Looks like Apple is saying NO, absolutely NOT! You must be using High Sierra OS on drives formatted for HFS+ or APFS to Read/Write to APFS formatted drives. Worth reading again!

So as I interpret this, if I am using FCPX on HS with an internal drive formatted as APFS and create a project, I won't be able to then boot into an older OS in order to use say iDVD or DVDSP to create optical discs from a master file on the internal drive. What is not clear is if I export (share) that master file and move it to an external HFS+ drive, will I be able to use the file for creating a DVD. Hope so, if not, Apple just created a nightmare for lots of folks. Setting aside disc creation, this will apply to lots of other situations too.

It bears repeating, according to Apple, NO HFS+ based OS (all OS's prior to HS) can READ/WRITE to APFS, and only HS or later will be able to R/W to APFS formatted drives.

I'll just state here that it is Apple making this decision as right now Sierra can indeed R/W and format to APFS, you just can't boot from it. So Apple could allow R/W to APFS from earlier OS's.
Progress! Really I like APFS, I do!

Fusion drives still have some work to do. So does RAID. Seems APFS will see and work with RAIDs but not create them. One of the interesting things about ZFS (remember 2005-6) allows for dynamic nodes with multiple drives. APFS may have something similar in being able to span drives and dynamic space allocation. Not sure how it will fully work though?? Really like the idea of being able to add a drive and to span for more space. Not sure APFS will implement this or not.

Be interesting to see how LumaForge handles APFS and other forms of NAS's.

Need to spend more time on Apple's developer site. So much to do, so little time

Sorry for long posts
Greg

QT7 Pro is dead in HS, I can assure you of that.

Again, Coremelt's products run just fine in HS right now, I've been testing it for them, and I am using their stuff just fine in HS, no problems.

THANKS, Greg, again for sharing your profound insights & to thoughts!

And esp. answering my amateurish questions…

VidGreg wrote: …
As to if the 32bit code inside Apple's own software will continue to be supported; who knows?? Tim hasn't told me …

… and we all know, Tim and you are THAT close!
On pics of you two, first Q is always "who's the guy aside Greg?"

On other boards, it was mentioned, that this 'interim season', allowing 32bit code, is basically for Apple - to dig out 'left overs' in any software… (must be a hell of a job, digging thru zillions of lines of code!!)

VidGreg wrote: …
… Compressor … AFAIK it is still based on older coding.
…

yep, that Quicktime vs AVFoundation topic… I guess, Apple will publish the update as late as possible, because all those fancy new stuff, VR, AR with its new formats will disclosure too much future feature and products…

VidGreg wrote: …
…Karsten, this includes almost everything having to do with optical disks. iDVD, DVD Player, Create Disk(Core software for creating optical discs code goes back to early 2000s) So I think Apple is about to just say no to optical discs altogether.…

Yep, that will create the biggest outcry, all Wedding videographers will quit - but, wait? Where to go? No more Encore… Toast? …

… or... FCPX' own ability to burn 'beer mats' gets a little bump = more customisation.

VidGreg wrote: …
… Karsten I think that as long as the internal boot drive is SSD then HS will convert the file system to APFS. Shouldn't matter if user installed or not, I think??? No opt out.

… hmm, I well remember the discussion, if I should or have to use TrimEnabler or not…

Finally, afaik as a 'mouse pusher', AFPS/HighSierra asks for a new firmware… so, will launching from an ext. drive with some older PS work?? (not joking: I have a drive with an older OS + iDVD… exclusively for burning disks )
Will you be able to downgrade HS>>Sierra- just in case - from a TimeMachine backup? Or a cloned drive?

And I'm still using a soft-raid (poor mans 1TB Raid0 for <100€) … ok, just as a 'working drive'… meanwhile replaced with extSSDs …

VidGreg wrote: …
…See my cautions previous posts.
…Sorry for long posts
Greg

No, not lengthy, the more info we get the better!

… switch off any 'auto updates', backup int.drive, check for 'exotics' in your setup (as my soft raid)…

… lucky me I'm an amateur, no productive environment here, so no hurry updating - except FCPX v10_4 has new wonders I HAVE to have…

Fusion drives may have to wait.

www.macrumors.com/2017/09/18/apple-files...usion-drive-support/

And this

beta.apple.com/sp/betaprogram/apfsfusion