Time Machine and backing up in Catalina
Although some of us were hoping that Catalina would support making Time Machine backups to APFS volumes, it doesn’t. However, the version of Time Machine included in 10.15 doesn’t work the same as that in Mojave, and some users are reporting problems with their backups when upgrading to Catalina. In this article I try to pass on what has changed, and how you can best address problems if they arise.
Apple’s documentation hints that something has changed, but doesn’t explain what. Inone Help page, Apple points out that “if you create a backup on a Mac or partition with macOS Catalina, you can only use that backup on Mac computers or partitions with macOS Catalina.”
Additionally, inthis article: “If you upgraded to macOS Catalina on a Mac that uses a Time Capsule or other network storage device as the backup destination, your existing backups are also upgraded and can be used only on macOS Catalina. New backups that are created can be used only on macOS Catalina.”
Unfortunately, Apple doesn’t appear to document Time Machine for developers or advanced users, so that is all that I can find at present.
If you back up to a Time Capsule – or to another networked storage device – it appears that this change is reflected in the extension used by Time Machine backups, which change from being .sparsebundle to .backupbundle to reflect their being updated for 10.15. Once they have been changed, they can no longer be accessed from any earlier version of macOS, at least not recognised as containing a Time Machine backup. Bob kindly reports that the new .backupbundle can still be mounted using an older DiskImageMounter app in /System/Library/CoreServices, although whether you can use Time Machine to retrieve any of its contents remains unclear.
Many users have had no problems with their Time Machine backups following upgrading to Catalina. A range of problems have been reported by some users, typically that Time Machine refuses to make any new backups to an existing backup set. The best answer then is to start a new backup set, if possible.
There are good reasons for creating a new backup set for Catalina anyway. As all system files are replaced, most other files change paths, and Catalina has a complete new layout on two APFS volumes, the first backup after upgrading is likely to approach the size of a full backup anyway. Giving it a clean start also means that your existing backups shouldn’t be converted to the new format, allowing you access to them if you needed to revert to Mojave, for instance.
This is also a wise choice if you want to run your Mac as a dual-boot system, perhaps being able to boot back into Mojave for occasional use of 32-bit apps which Catalina has rendered inaccessible. Dual-boot systems don’t fit very well with Time Machine anyway, and you should avoid trying to run Time Machine backups in both versions of macOS for these reasons.
My summary recommendations are therefore:
- Before upgrading to Catalina, make one last backup (or two!), then turn Time Machine off.
- Once you have upgraded, turn Time Machine back on but using a new backup set which won’t then be accessible from macOS 10.14.x or earlier.
- If you experience problems with Time Machine backups in Catalina, turn it off, archive your old backups, then turn it back on to use a new backup set.
- If you back up to networked storage and want to retain access to your old backups from previous versions of macOS, follow a similar course to ensure that your Catalina backups go into a new backup set.
- When running a dual-boot system, only enable Time Machine backups in the system which you use for most of your work (most of the time). Disable Time Machine backups in the other version of macOS.
- If you experience problems with Time Machine or its backups in Catalina, take the trouble to contact Apple Support. Even if they don’t solve your problems, it will ensure that Apple is aware of them, and they will perhaps get addressed in the future.
If you use a third-party backup utility such as Carbon Copy Cloner or SuperDuper!, read their support information on backing up Catalina, and ensure that you’re using the latest version of the app. Mike Bombich runs an excellentpage about known issues with Carbon Copy Cloner and Catalina, and Dave Nanian gives insights into how SuperDuper! copes with Catalinain his blog. Both these products are extremely well supported: if you have issues with them, first check their online support pages, and then if you still can’t resolve them, contact the developer.
I welcome your comments and experience with Time Machine in Catalina, please.
Thanks to Hans-Peter, Bob and several others who have have been kind enough to share their experiences.
Share this:
- Click to share on X (Opens in new window)X
- Click to share on Facebook (Opens in new window)Facebook
- Click to share on Reddit (Opens in new window)Reddit
- Click to share on Pinterest (Opens in new window)Pinterest
- Click to share on Threads (Opens in new window)Threads
- Click to share on Mastodon (Opens in new window)Mastodon
- Click to share on Bluesky (Opens in new window)Bluesky
- Click to email a link to a friend (Opens in new window)Email
- Click to print (Opens in new window)Print
83Comments
Add yours- 1
- 3
Michele Galvagno Thanks for this, Howard. Unfortunately it comes a week too late as I’ve already experienced the issues first hand.
In the end my TM completed a backup roughly as big as a full one, probably just rewriting all filepaths… the interface in the menu bar is slightly different as it doesn’t show progress towards a goal anymore rather just how much was backed up.
The biggest issue I had was with cloning using CCC. I used a spinning drive for my clone and in Mojave I had no issue, including converting it to APFS. As soon as I started cloning in Catalina, following MB instructions on the blog, it started converting my disk from HFS+ to APFS (impossible as it was already so…) and, after 2 hrs, failed (as expected from me at least) burning out my old clone and letting me in the mud.
I therefore thanked heaven for having a spare SSD I used to test Catalina with, I erased and formatted it manually via Disk Utility and then used it for cloning.Overall not a nice experience but well… still not as bad as the current iCloud crisis.
Let me know if I may be able to give more insight into what happened on my system.
- 4
hoakley Yes, my apologies, but of all the major features in macOS, Time Machine is the hardest to gather information about, and the least tested in beta, I suspect.
I’m surprised that you had such problems with CCC, though. Have you informed its developer?
I’m going to be running some tests here with TM over the next few days, and will be writing more about this in due course.
Howard.
- 4
- 6
- 7
Jerry Fritschle After upgrading to Catalina and finding all to be well, I wiped my CCC volume and created it from scratch. Otherwise, if the clone system has FileVault enabled (and mine did), one at least has to boot into it and turn FV off. With FileVault on, CCC cannot rejigger the volume into the new System and Data partitions.
I noticed no issues with TM, but decided later to start anew with it as well.
- 9
Alberto.G Dear Howard, I have carefully read this article about Time Machine and would like to contribute to a shared discussion.
I am a user who has chosen to work with a dual-boot system for more than a year and I am absolutely convinced and happy with the choice I made.
My system is defined as follows: on the internal disk (factory mechanical HD 1 TB capacity) there is Mojave (with the last supplementary update installed) while on the external disk, a 2.5 “SSD with 1 TB capacity, Catalina is already updated to version 10.15.1.
In his article, he writes:
“Dual-boot systems don’t fit very well with Time Machine anyway, and you should avoid trying to run Time Machine backups in both versions of macOS for these reasons.”
Instead, in my case, Time Machine behaved very, very well!
After switching to Catalina on 8 October last, I already ran a dozen backups, on two separate disks used only for backups, without ever detecting any error in the operation of Time Machine.
I back up this way and did it even before switching to Catalina. I always leave both boot disks active (mechanical HD + external SSD) so Time Machine reads them both and backs up both by copying them to the exclusive backup disk. Not only that, I do this twice, a first time when I boot the system with the internal HD disk with Mojave and from here I start Time Machine using a first backup disk, a second time when I boot the system with the external SSD disk on which Catalina is present and from here I start Time Machine using a second backup disk. It should be noted that this double execution of Time Machine runs it with expiration every 5/6 days.
In conclusion, in doing so I find myself having two separate backup disks on which both my system volumes have been saved.
The transition from Mojave to Catalina was absolutely transparent for Time Machine, to note also that Time Machine simultaneously backs up two system disks with different macOS: Mojave and Catalina!
Always, for greater security, I NEVER keep backup disks connected to the iMac when I’m not using them.
Alberto- 10
hoakley Thank you, Alberto.
I’m not saying that dual-booting with TM active in both systems is impossible. However, few users would ever devise such an elaborate scheme as the one which you use.
For me, the whole advantage of TM is simplicity: I leave it turned on, with my backup drive (a RAID) connected. Every hour it automatically makes a backup to the RAID system, and it all just works. I never disconnect that RAID, it’s not partitioned, but a single HFS+ volume. I never run manual backups.
If I were to try making this a dual-boot system, Catalina would convert my existing backups to its new format, according to Apple. Once converted to that, Mojave can neither access them through the TM app, nor make any new backups to them. So a simple TM system doesn’t work with a dual-boot system.
I note with interest that you keep connecting and disconnecting your backup disks. Whilst I’m sure that you’re happy doing that, that isn’t what you’re supposed to do with backup disks, particularly when running automatic backups every hour.
We had a similar experience when trying to dual-boot Sierra and High Sierra: yes, it was possible, and some users even kept TM running, but the elaborate processes that you have to go through to prevent one from corrupting the other’s backups made TM so complex that it simply wasn’t worth the effort, for the great majority of users.
Thank you for explaining your intricate solution – it’s an ingenious workaround. Congratulations!
Howard.LikeLike
- 11
Alberto.G Dear Howard,
first of all, I thank you very much for the congratulations you wished to express to me!
I am certainly proud of this.
However, yes, I must admit to having come up with a rather intricate backup solution…
However, for me it was completely intuitive, almost natural, to arrive at this… without thinking too much.
The amount of daily work I do on my machine is certainly not comparable to his!
For me it is more than sufficient to perform manual backups, but not every hour, but every 5/6 days.
For my work requirements, I frequently change the boot disk from Catalina to Mojave, and then from Mojave to Catalina, without any difficulty since Apple allows me to do it. And on these occasions I take the opportunity to make my backups on separate discs, as already mentioned.
I had the need to restore documents, which I had deleted by mistake, taking them from a backup disk and quietly bringing them back both to the boot disk with Mojave and also to the one with Catalina which, however, I chose as the default boot disk .
I will keep you informed of any occurrence.
With regard,
Alberto
- 11
- 10
- 12
macOS CatalinaでのTime Machineとバックアップ | 酔いどれオヤジのブログwp […] (Via Eclectic Light Company .) […]
LikeLike
- 13
Alberto.G As you surely will have guessed, I am Italian and for my writing I use the Google Translate app for English translation.
I therefore hope that Google behaves well, very well indeed …
My greetings,
Alberto - 15
Tim Cox I’ve found Time Machine quite broken in Catalina. I always backup my MacBook Pro to a Time Capsule and ever since Catalina there have been issues. I started a fresh backup because it didn’t look like I was ever going to get a backup on top of the original Mojave one. But starting a new one hasn’t helped. They are always very very slow – I get perhaps one backup run a day. And the ‘Cleaning Up’ step typically takes another 24h. Right now a backup that started about 9 am this morning is still only on ‘40.4 MB of 268 MB’ and estimates about 10 hours remaining. (I believe it, after my recent experience!) Something is just wrong in Catalina and I have no idea what. Even when it is so slow I see continual network activity, both uploading (e.g. right now at 1.8 MB/s according to iStat Menus) and downloading to the Time Capsule, but the progress bar is not moving at all. So what on earth is being transferred?! Quite discouraging. I am not happy with Apple right now because I also suffer from the ‘Notes’ failing to sync across devices issue in Catalina and that has completely messed up my everyday workflow as well as causing me to lose information and waste enormous amounts of time trying to sort it out (I’m in the process of moving my 2000+ notes to Bear).
- 16
hoakley I’m sorry to hear that.
Is your Time Capsule over 3 years old? If so, I’d strongly recommend that you replace it anyway, as it is increasingly likely to suffer hard drive failure. That would give you the opportunity to purchase, say, an external hard drive or similar, which would make your backups so much quicker, even when things are working more normally.
Howard.LikeLike
- 16
- 17
Tim Cox Thanks for the suggestion. Hard for me to blame the Time Capsule when it was perfectly fine, and Time Machine was running perfectly, up to the Catalina upgrade. I’d expect a server-grade drive to last more than 3 years :) Frustrating too that there is no clear way to check it. I reinitialized with Airport Utility, as Apple recommends, when I started a fresh backup in Catalina. One day I have to learn how to decode the Console logs. I can never find anything useful any more. (Can you tell I prefer the old Unix days when one could actually diagnose, understand and debug such issues without an arcaneness barrier?!)
- 18
hoakley My rule of thumb is to try to replace all hard drives before their warranty expires. It’s now very hard to find reasonably priced drives with 5 year warranties – almost all offer only 3 years. As that means they’re engineered to last 3 years, even down to component specs, expecting them to last any longer is trusting a lot to them being from a ‘good’ bad which outlives the warranty.
It’s not hard to look in the unified log – I have a free utility Consolation 3 which does that. The trouble is that, when using a networked drive, it’s the log of that which you need to see. And Apple doesn’t oblige there.
Howard.LikeLike
- 19
Tim Cox Just to complete my Time Machine saga, I finally backed up to an external drive, erased my MacBook Pro and reinstalled Catalina (10.15.1) from scratch. I also cleared my Time Capsule. I can report that Time Machine between the MBP and the Time Capsule now works ‘normally’ again. It is as fast as it used to be pre-Catalina, and I no longer have issues with very slow copies or very long cleanup times. I am backing up about 500 GB of my user documents. It really looks like the Catalina upgrade from Mojave broke something, which has not occurred during a completely fresh Catalina installation.
Oh, and thanks for Consolation! It’s very useful for getting to grips with the unified log. (And also thanks for T2M2.) I really appreciate your postings – always something interesting to learn.
- 19
- 18
- 20
Alberto.G Dear Howard,
With regards to the timeless speech of Time Machine, I report below an excerpt of his answer that he gave me on November 11th 2019 at 11:18 pm:
“I note with interest that you keep connecting and disconnecting your backup disks. Whilst I’m sure that you’re happy doing that, that isn’t what you’re supposed to do with backup disks, particularly when running automatic backups every hour.”I decided to put his advice into practice!
Since last night, therefore, I keep the backup disk of Catalina always connected to my machine (iMac 21.5 “, Retina 4K, Late 2015) and I will no longer perform manual backups, as I was used to.
However, I continue to operate with a dual-boot system, using as default (for daily use) the external unit consisting of a Crucial SSD disk, with 1 TB of capacity, connected via USB 3.0.
Catalina 10.15.1 is obviously installed on this SSD.
Absolutely the best backup solution!
So a heartfelt thanks to you, Doctor Howard!
With regard,
Alberto - 22
delay Thanks for the article. Good to know I’m not alone. I asked a question on StackExchange and I have got no useful information. My specific problems are here:https://apple.stackexchange.com/questions/373241/time-machine-freezes-while-copying-files-with-no-error-message-on-catalina-10-1
A question that I have not been able to find a good answer to is if I can rename some folder in on my backup disk, such that time time machine will start over, and I can still keep my old backups?
- 23
hoakley The best way to keep your old backups is to archive them to somewhere else, if you can. If you have to keep them on the same disk, then you should turn TM off, try renaming, then turn it back on and see if it prompts you for a backup volume again. It’s going to be a matter of trial and error before you fool TM into not recognising the old backups and starting a fresh set.
I have this same problem to face when I upgrade this iMac Pro to Catalina. I’m intending to try changing the name of the machine folder first, e.g. fromMy MacBook Pro to something different. If that doesn’t work, I’ll go up to theBackups.backupdb folder above and change that next.
Please let me know what works for you, so I can steal it!
I wish you success,
Howard.LikeLike
- 24
Tony Have you tried using Consolation with the Time Machine filter option to see approximately where it was before it got stuck? In Catalina, Time Machine now logs its progress roughly every 10 minutes, including how much was backed up and what the last file was.
You also mention it “quietly stops.” Does this mean in Time Machine preferences, the backup task is shown as no longer in progress, or does it appear to hang at a certain point? I’ve found TM backups can go fairly fast on large files but there are chokepoints where it’s wading through a large number of small files, whether copying them, creating hard links to them, or deleting them where it may appear nothing is happening. The main ones I’ve encountered are your Mail library if you have a lot of archived emails and attachments, and Messages attachments.
Consolation should help you find out what it’s currently doing. I also like to use the following command when troubleshooting Time Machine:
sudo fs_usage -wf filesys backupd | grep HFS_update
It should display all the HFS file operations being performed by Time Machine and can give a real time update on what files are currently being accessed.
LikeLike
- 25
delay Thanks for your comment Tony! I tried running the sudo command you suggested. And the backup got stuck at a particular folder inside the tree of the hidden directory ‘.DocumentRevisions-V100’. First it took a long time just locating where this folder was, and then I didn’t even have permission to go inside it. So I started to add permissions (step by step) for me to go down the tree. I ended up in a folder containing several ‘.app’ files with long names made of hexadecimal numbers. Then just a little later – after the ‘backupd’ process having had no CPU usage for almost 2 hours – it starts processing again(!) I have no idea if it was just a coincidence that it started after I added permissions.
But then gets stuck at some other location…. And this location is much easier to get into, and seems to contain some automatic update software for Google’s products (such as Chrome). It took a couple of minutes and then it moved on and got stuck again.. Another app. And then in some Safari database.
So I guess the thing is actually running. But for some reason it gets randomly stuck for no apparent reason…. I will continue to investigate!
- 26
delay It turns out that the backup was probably never actually stuck, but just running extremely(!) slow. I left the computer on over night and after 14 hours the backup had finished. The command for printing out which file is being copied helped me a lot to get a more detailed view of what’s going on. Thanks!!
- 25
- 23
- 28
Stephan Catalina Migration Assistant consistently destroys TimeMachine backups – BEWARE – HELP NEEDED
I have 3 TimeMachine NAS backups of my MacBook Pro, each one in a different location, two on a Synology NAS and one on a Time Capsule. So when I got my new MBPRO 16“ I thopught it would be safe to use Target Disk Mode and Migration Assistant to transfer my data. The process started normally and got stuck at 107,000 documents or so. Then, the target computer crashed and restarted. The source computer was unable to boot after that and I could not repair it using recovery mode tools (I tried everything).
Next I tried to restore from my latest NAS Time Machine Backup. Migration Assistant would not recognize the (encrypted) backup so I had to manually mount and open the backup file. Migration assistant went to work and after one night crashed (did not complete), with the same result: the new MBPRO would not boot and the backup data on the NAS was destroyed – deleted!
Thinking there might be an issue with the Synology NAS and TimeMachine, I went to my TimeCapsule backup. Again the Migration Assistant would not recognize the backup at first, mounted it manually, then started. This morning I found the MBPRO with a boot screen, entered my credentials and it … crashed. Recovery mode told me nothing had been copied and the OS is unable to boot, I had to reinstall.
Then I tried to access the TimeCapsule backup manually. It was destroyed, only 25MB or so left of a 2TB backup file.
I have one NAS backup left which I am copying to an external HD …
Can someone help me? Apple support just wants me to try the things that already failed 3 times. For example, how can I restore from an encrypted backup file that I copied from a NAS to an external HD?
For everybody else … be forewarned. TimeMachine / Migration assistant is causing irreparable data loss!
- 29
hoakley To the best of my knowledge, Migration Assistant doesn’t write to the source being used for migration. I’m therefore at a loss to explain how you have “lost” these backups.
I also wouldn’t recommend that you migrate from network backup storage. When it works, migration is slow enough from a directly-attached Thunderbolt disk (or Target disk). I think a better strategy would be to clone your existing MBP to a Thunderbolt disk, and migrate from that to your new MBP.
Unfortunately, because of their size and complexity, full migrations are always going to be prone to errors, so it’s best to plan ways which are most robust in the face of such problems.
So I would rebuild the original MBP first, then clone its internal storage to a locally-attached disk using CCC, and use that as the local source for migration. It might also be wise to cut back on what you’re trying to migrate: remember that you don’t have to migrate everything in one go, but can have several separate migration sessions. I would also hold off any migration until your new MBP is fully updated with macOS, which could make a big difference to its success.
Howard.LikeLike
- 29
- 30
Barry The error I get when TM fails is that there are 2 disks with the same name, ie Macintosh HD and Macintosh Data. This a weirdness with Catalina though it supposedly a feature, but I’m not sure why. Catalina has done a few things that I don’t really like. Hard to back-up in TM and splitting the Music and video etc into separate apps just makes it more work to track content in 2 programs instead of just having it as a tabs in 1 program. It seems Apple is a little wrong minded in some of changes they’ve implemented since Mojave.
- 31
hoakley Are you sure those are the volume names given? In Catalina, as Ihave explained, you should have a Volume Group consisting of
Macintosh HDandMacintosh HD - Data. TM should back both of those up, as they are the read-only System volume and the read-write Data volume respectively.
Howard.LikeLike
- 32
Barry Thanks Howard. Yep, those are the names, Macintosh HD and Macintosh HD Data. Data has no files in it. TM will not backup for me if the Macintosh HD Data drive is mounted. It does if its not. Not sure why.
- 33
- 34
Barry hoakley, thanks for that, when I look at it through the disk utility I see the Macintosh HD and then two Macintosh – Data HDs. Each of the named data drives have different amount of memory used, but they both have the same name. Also, when you click on and open the drive in finder, they appear to both be empty.
Should I delete one of those drives? Or take it back to the Mac store and have them look at it.
- 35
hoakley Oh. That should not be, and is most probably causing your problems with backing up.
Did you upgrade this from an earlier version of macOS, or is this a new Mac? If the latter, then it would be worth taking to an Apple store and getting it fixed properly.
If this is an upgrade, then you have a choice. If neither of the Macintosh HD – Data volumes has any files in it, you’ll need access to the backup you made before upgrading, as you’ll be best wiping the drive and installing Catalina from scratch. The best way to do that is by booting up into remote recovery, using Disk Utility to wipe the drive, then installing a fresh copy of 10.15.2.
Howard.LikeLike
- 33
- 32
- 31
- 36
Lars Baumstark Here´s more “woes” with Catalina and Time Machine backups:
We have a small office running with team members backing up their machines to a server running El Capitan (stable, robust, serviceable on cMP hardware).
After upgrading all but one client Macs from Mojave to Catalina, I noticed that the old Server.app does not list the time machine backups any more – just the remaining Mojave user…
After first brief shock and disbelief I noticed that the name change of the backups (mentioned in your blog article) is the reason: The old Server.app just does not recognize the new name format.
While creating an object alias of a *.backupbundle and renaming it back to *.sparsebundle makes it appear in the list of user backups again on the server side, all meta information is still not available (creation date, last modification, size etc.).
Backups (even the un-listed ones) seem to work and provide retrievable data to the client side, so the main functions are still given – but perhaps there is some Unix/Posix way of getting the meta data over to the alias link?Any observations on current hardware and Server.app (stripped down as it now is in comparison to the max. El Cap version 5.2) would be welcome to read…
- 37
hoakley Thank you.
I don’t think you can solve that with the server running El Capitan, I’m afraid, because of the changes made by Catalina. If you want a compatible server for it, then it would need to be running Catalina, as a shared TM server in the Sharing pane of macOS client.
Howard.LikeLike
- 38
Lars Baumstark Success with Catalina and Time Machine Backups to El Cap running Server 5.2
I´ve managed to satisfy both ends of that data pipe the following way:
1. Let Catalina clients do the conversion of the .sparsebundle to .backupbundle
(as noticed, the result is that the backups work, are still retrievable from Catalina but do not show up in the Server App any more)
2. On the server side, create symbolic links for all backups by terminal command ln -s with the .backupbundle and the link again being named as .sparsebundle
Ceveat! The name of the link has to be slightly different also (not just the extension!) because otherwise the Catalina clients will start to convert the image again to the new format.
Example: You have a “Sam Tramiel Macbook Pro.backupbundle” to begin with, then create the link as i.e. “ST Macbook Pro.sparsebundle”
3. Restart Server App to update the backups being shownOutcome:
There will be an actual listing of all backups, including correct owners, machine types (icons) and last modification date. Only thing not working yet is the overall size info.
But based on the info that is correctly shown, alerts can be automatically generated for failed backups or too long intervals without user backups!
- 38
- 37
- 40
Ji Rui-fu Just to share our experience:
My wife got a new 16″ MB Pro which came with Catalina installed. She had a TM backup on an external HD and a Mojave clone (using CCC) on another external HD.
I updated my 2015 iMac to Catalina.
She heard about problems with Catalina and TM and so erased her TM external HD before turning TM back on. TM is now running perfectly.
Now this: we both SWEAR that we erased TM AND reformatted our external HDs as APFS but when we just checked, they are both formatted as Mac OS Extended (journaled).
I am baffled, but…TM is working fine on both Macs.
Lastly, love and appreciate and have told friends about your great site. - 43
Barry Hoakley, thanks so much for your help and accurate info! Just to finish out. I went back to the Apple Store and the Genuis and I checked it out and it turns out that I did have 2 Data Drives. 1 showed that the volume had material on it and the other was empty, so we held our breaths and deleted the empty one. All is good now. TM works fine and there’s only 1 data drive. Again, thanks for your help. Great site you have.
- 45
Jorge Kusnir Thank you for your comprehensive article.
I would like to duplicate my experience: the only way to see old backups.
Only when time machine is backing up (in Catalina), and ONLY DURING THE BACK UP IS RUNNING, you can click in “enter time machine”. This will stop the backing up process but you will be able to see all your backups from before. If during that momentum you enter Disk Utility you will see Sparsebundle mounted.
I am not IT, just an user, so I am unable to interpret it and have conclusions. I am sure you might…
Apple assistant was unable so far to fix it, so they escalate to engineering, and I am currently awaiting an answer. - 47
Jon L. I have been getting a message that the drive used for TM backups is full. Before updating to Catalina, I used to be asked about deleting the oldest backup. I do not get that anymore. I tried to manually delete the oldest backups, but have had issues deleting all the files. Fortunately, I use Backblaze to remotely backup my computer, but how do I get Tm to work like it did? Thanks.
- 48
hoakley Thank you.
I’m not aware that TM has changed in this respect: it can only thin backups to a certain extent before their integrity is lost. Over time, even with the most diligent of thinning, it’s inevitable that the minimum size of backups steadily rises until eventually the backup store becomes full.
Catalina makes this worse, because TM must preserve the last pre-upgrade state carefully, and the first post-upgrade backup is necessarily large, and can’t be reduced in size because of the new files and structures in Catalina.
How much free space is left on your backup volume now?
You shouldn’t try deleting anything in backups from the Finder: it’s much better to use the Time Machine app itself, ortmutilin Terminal.This article explains more.
Howard.LikeLike
- 48
- 49
John Steinmuller In the past if one had a problem with a new OS one could simply use their Time Capsule backup to regress to a previous OS. Not so with Catalina. In an almost virus-like manner it changes extensions and makes it impossible to roll-back to a previous OS that works with 32-bit programs. Just having Catalina connect to my Time Capsule corrupted all backups of all our machines. I had to bring my MacBook to an Apple store, install Sierra, and then rebuild my machine from a hard disk I had pulled from my Time Capsule that had backups untouched by Catalina. My take: Tim Cook wants you to really, really like Catalina. And if not, too bad, you’re stuck with it.
- 50
hoakley Thank you.
Yes, because of the changes made to backups and volume architecture, rolling back from Catalina to an earlier version of macOS isn’t easy. However, I’d love to know why you consider that Apple has done thisdeliberately to stop users from rolling back.
Howard.LikeLike
- 51
John Steinmuller I found that I did not like the 64-bit revision to an older 32-bit program I regularly use and needed to revert to High Sierra. Furthermore, my 2012 MacBook strained to run Catalina. Even Mohave ran too slow and I had previously reverted to Sierra with the push of a button. Not so after upgrading to Catalina.
Regardless of the changes to volume architecture, if you have a backup which in the past allowed you to revert to a previous OS with the push of a button you expect that same functionality. You should be notified before installing the new OS that this no longer possible–especially if the new OS is not backward compatible with older programs. To roll back from Catalina is more than “isn’t easy”. Try it yourself. I’d like to see an easier solution than what I needed to do.
Luckily, I had upgraded the hard drive in my Time Capsule last year and kept the old disk intact. All attempts of reinstalling from the new drive “hung up” for a day before failing. As described here and else, Catalina changes more than the extension names of Time Machine backups. Attempts to retrieve backups from a fresh install of High Sierra were not permitted.
The only thing that worked was to reinstall Catalina, retrieve the Catalina backup, copy the home directory to an external drive, reformat my MacBook, do a fresh install of High Sierra, reinstall the old hard drive back into my Time Capsule, retrieve an uncorrupted backup from that drive and then transfer interim files from the saved external home directory back to the MacBook. And then reinstall any Apps that had been installed in the interim. Finally, reinstall the new drive into Time Capsule, format it, and resume backing up our four machines with a drive that had never shook hands with Catalina.
Again, if anyone has -actually- rolled back from Catalina in easier manner I’d love to hear it. (The Apple Store techs were clueless as to extension changes in Time Machine and said they were given zero guidance. They did help me do a clean reinstall of High Sierra.)
I moved from WINTEL to Apple years ago because, with Apple, things just worked. Now, more and more, things change in the Apple ecosystem without notice and I have to search the internet to find out what just happened. Usually it seems to be something that provides more service revenue for Apple.
Specifically, Catalina further implements “Project Catalyst” to allow iOS apps to be easily ported to/built as Mac apps. This (along with the Mac adopting the iOS security model) is a central part of Apple’s push into enterprise IT, as many businesses have their own internal iOS apps. (Hammen, R. (2019, June 6). Significant Changes in macOS 10.15 Catalina of Interest to Mac Admins. Retrieved fromhttps://medium.com/@hammen/significant-changes-in-macos-10-15-catalina-of-interest-to-mac-admins-fbc3865c055e)
Apple can’t sell IOS apps to MacBook users of previous OS versions.
So maybe Apple didn’t deliberately try to stop users from rolling back but they certainly made it very hard to roll back. (It took me days to devise my described work around which only worked because I periodically put backups into cold storage.)
Things at Apple don’t just work like they used to and coincidentally the end result is more service revenue.
- 51
- 50
- 52
Perfectlyfadeddelusions Reblogged this onZero to Hero Perfectlyjadeddelusions.
LikeLike
- 53
Technolong Well, I thought all was well, maybe not. It backed up several times yesterday, and once at 6:24 AM this morning. When I got home from work this afternoon, there was a backup fail message again. It says. Time Machine couldn’t complete the backup to Seagate Backup Plus Drive Two of the disks to back up have the same name. Rename one of the disks named Macintosh HD . Last successful backup: Today, 6:24 AM The drives are listed as: Macintosh HD Macintosh HD – Data This is the same thing it did before. It backed up for ten days successfully after installing Catalina, then started the backup fail. One other thing, and I don’t know if it is related or not, and it has been ocurring for a long time. I go through periods where the computer says, Backup not ejected properly . Sometimes the backup has worked properly even with this message occuring. Then it quits saying this for a time. I just never know. Any thoughts? Thanks Tom
LikeLike
- 54
Fernando G Rendina Thanks for the article! I have been scouring the internet for a solution to my backups to no avail. After upgrading, they have been crawling. For instance, it is now on 15 GB out of 45 GB and I have had it running for about a day! Crazy! I currently do not have another disk to start a new backup, so I am having to deal. Is that the only way to speed them up?
Thanks in advance,
Fernando
- 55
hoakley Thank you, Fernando. I have detailed how to use my free utility T2M2 to discover whether this is the hidden version database, or something else, which is choking it. You can then exclude that folder from the backup, and it should run much more quickly.
Some users use an unthrottling command in Terminal, but I wouldn’t recommend that – and if you do, you’ll need to set it back to normal afterwards anyway.
Howard.LikeLike
- 55
- 56
Worries Speculation was rife among users, rather than developers, that last June’s WWDC might have brought announcement of a major update to Time Machine coming in Catalina. Now that all Macs running Mojave have to boot from APFS, and with Catalina’s additional conjuring trick of a read-only APFS system volume, for many systems now the only drives still using HFS+ are those for Time Machine’s backups. So why hasn’t Apple fixed Time Machine so that it can back up to APFS yet?
LikeLike
- 57
hoakley The simple answer is that it’s a lot more complicated than it might seem.
One fundamental problem is that TM relies heavily on directory hard links, which are only available in HFS+ and not APFS. So TM 2.0 can’t use the same basic tricks as at present, and would need a completely different backup structure. That’s non-trivial to design and implement.
Howard.LikeLike
- 57
- 58
Lady As advised, the best thing to do is start over with a new Time Machine backup. As noted above by the article that Ian referred to, it’s a good idea to start fresh. Unless you really need those older Time Machine backups, just erase the drive and go from there. How useful are older backups going to be for you now that you have updated to Catalina? Catalina has also formatted your internal drive to APFS. You might also want to consider using another backup strategy. Most of us use and recommend cloning software: Carbon Copy Cloner or SuperDuper. Both programs offer a very effective way for backing up and restoring data.
LikeLike
- 59
defjaf Does anyone know what it means when the Time Machine prefpane passes the nominal size of the backup it was making and then says “Backed up: XX.yy GB” for a (long) while? (I see it when backing up to a Synology NAS).
- 61
ErikH I held off upgrading to Catalina due to persistent problems reported with having multiple accounts in Mail (ao Michael Tsai of SpamSieve). Last month I decided to opt for a Mail alternative (Postbox), made it my default Mail agent and upgraded tot Catalina.
Although I do open Apple Mail occasionally to download messages, I noticed Time Machine didn’t make a backup of my Apple Mail content since I upgraded to Catalina. When I enter Time Machine from Mail, it shows only the current content of my Mail accounts … I cannot go back in time (it also shows only hexadecimal encoded mail subjects like dffe, 7587, a6fe, …).Is this expected behaviour of Time Machine (the backup resides on a Synology NAS)?
- 62
hoakley Thank you.
No – as far as I’m aware, the Mail folder should back up correctly, unless of course it has been added to the exclude list. Have you checked for errors using T2M2 or Ulbow?
Howard.LikeLike
- 63
ErikH Thanks for the response.
I also checked my mail folder in ~/Library/Mail/V7/MailData and it appears that my mail data is properly backed up by Time Machine (which is a relieve). Only the backed up mails appear not to be accessible after “Enter(ing) time machine”. I have no problem seeing other backed up files in my Time machine backup at various backup moments in the past.
Next I checked T2M2 which showed “No error messages found”.
Ulbow, however, showed 2 identical errors less than a second apart:
Couldn’t create MailLibraryItem at path: /Volumes/com.apple.TimeMachine.localsnapshots/Backups.backupdb/mac-pro-2014/2020-05-09-182616/Macintosh_HD – Data/Users//Library/Mail/V7 libraryRealm 3 fullPathToLibrary (null) pathRelativeToLibrary (null)Unfortunately searching the internet for this message does throw up any further information. I deleted and recreated the 3 Envelope-index files but that did not make any difference (got same error message).
Please note I created a fresh Time Machine backup after upgrading to Catalina using 10.15.4.
I would appreciate any help.
- 63
- 62
- 65
Cyclo Pontivy Excellent article, thank you.
With a backup on an SMB network share (and you have to use SMB instead of AFP) there is a very good reason to recreate a new backup with Catalina.
A sparsebundle package or a sparsebundle migrated to backupbundle by Catalina contains tens of thousands of files in the “Bands” directory. These files are 8.4 MB and over time they accumulate until they lead to very slow backup (SMB does not like directories where there are tens of thousands of files).
A Backupbundle created by Catalina contains 268.4MB files in its “bands” directory, which decreases the number of files and greatly speeds up the SMB-sharing backup. - 67
Rob I’ve been having issues with TM not running automatically. I’ve tried all sorts, removing the .plist file, starting a new backup and formatting my Time Capsule but still the backups don’t happen automatically. They work fine when I click “Backup Now” but then they won’t run again until I do a manual Backup Now. Very occasionally more than one backup will happen before they stop again.
TM reports the time and the next backup and I can see the Time Capsule in Finder no problem.
- 68
- 69
TheFSG Cheers, Howard – as requested…Rob
Analysis from 2020-05-14 21:06:27 +0000 to 2020-05-16 21:06:27 +0000 for 48 hours:
Backing up to Time Machine Backups (/dev/disk2s2): /Volumes/Time Machine Backups
Current free space on backup volumes:
✅/Volumes/Time Machine Backups = 2.69 TB
No auto backups started in the period, last backup completed successfully 311.2 minutes ago,
currently still making a manual backup,
last manual backup started 41.0 minutes ago,
backed up a total of 53337 files, range 1 to 14601 in each backup,
total data for each backup was 1.12 GB, Zero KB, 69 bytes, 1.35 GB, Zero KB, 69 bytes, 735.7 MB, Zero KB, 69 bytes, 2.98 GB, Zero KB, 69 bytes, 635.8 MB, Zero KB, 69 bytes.
Created 4 new backups, and deleted 0 old backups,
Created 10 new snapshots, and deleted 8 old snapshots,
Of 15 volume backups:
0 were full first backups,
5 were deep scans,
10 used FSEvents,
0 used snapshot diffs,
0 used consistency scans,
0 used cached events.✅ No error messages found.
⚠️ Running check on scheduling system (CTS and DAS).
DAS appears to be rescoring normally over the last 90 minutes.
Running CTS check next.
CTS has not been told to run a backup in the last 90 minutes. Scheduling may have stopped.
Consider restarting your Mac.- 70
hoakley Thank you. If you’re running Catalina, something’s wrong with its task scheduling system CTS, which is causing it to fail to run backups periodically. For further details, you’ll need to look in the log (I have several articles here about watching DAS and CTS at work scheduling automatic backups).
An alternative is simply to download and install the latest Combo update, then apply all later security updates. That is quite likely to fix that sort of issue.
I wish you success,
Howard.LikeLike
- 70
- 69
- 68
- 71
- 72
BillJ Thanks for this. My issues came up as soon as I upgraded to Catalina. For whatever reason my USB drive that I use for Time Machine will sporadically unmount itself. The USB drive that I use to store the bulk of my data never does this. Thinking it was the drive, I bought a new one. Same issue. Swapped cables, swapped USB ports, same issue. The only constant is Catalina and TIme Machine. :(
- 73
ErikH Continuation of my post 63 above dated May 9th 10:17 pm.
I thought I’ll provide a (short?) update on my problem as this may help others as well.
I contacted Apple support and was quickly connected to someone from second line support. We agreed there are essentially 3 problems which the support agent could not solve and these have to be escalated to Apple engineering. These are visible when you enter TM from within Apple Mail:
1) I can’t retrieve deleted mail messages from Time Machine (TM won’t let me go back in time)
2) TM shows mail subjects as hexadecimal codes
3) when starting in the Inbox in Mail (ie not in a inbox folder if you have multiple e-mail accounts) TM doesn’t show any e-mailsThe problems persisted also when I created a fresh TM backup on an attached USB-drive (normally my backups are on a Synology NAS).
The support agent recorded several screen actions in Quicktime movie and also captured about 1GB worth of system information before all was submitted to Apple. I may not hear anything until it gets solved and incorporated in a future macos release. All in all we spent about 2 time 90 minutes on the phone.
Fortunately there is a workaround. In Mail, go to File/Import Mailboxes … and then navigate in your TM backup to // – Data///Library/Mail/V7/. Next you have to find the right folder with your mail account from which you wish to recover one or more mails. You can also import all your mails but that may take a while depending on the number of mails.
Hope this helps readers with the same/similar problems.
- 75
porg SUSPICION: Introduction of APFS volume group with split System+Data volumes in macOS Catalina enforces a full backup which may enforce thinning your backup history.
IN DETAIL:
I updated from 10.14.6 to 10.15.4 in May 2020. My initial Time Machine backup was a deep scan unsurprisingly as an OS upgrade changes way more files than in the FSEvents log.
After a multiple hour long initialization phase (!) the file transfer size was shown which was 100% of existing data. I aborted b/c in reality only about 10% (OS files, libs, app bundles) could have changed, my data was mostly unchanged, some extra 5% at most.
I suspected that maybe the disk ID association broke for some reason, although no hardware was changed, therefore ran “sudo tmutil associatedisk”. Gave Time Machine another try. Again showed a transfer size of 100%. I was annoyed at Apple’s QA, but accepted with sighing. As a nasty side effect this thinned out my entire backup history! At least Time Machine’s safety rule of “never deleting the latest backup” held.
Meanwhile I read about macOS Catalina’s change: In your main APFS container your bootup volume i.e. “Macintosh HD” gets split into two volumes, i.e. “Macintosh HD” and “Macintosh HD – Data” (same name with the “- Data” suffix)) which are grouped together in a so called “volume group”.https://bombich.com/kb/ccc5/working-apfs-volume-groups
I now suspect, the following has happened behind the scenes:
Time Machine goes the safe way and says “Everything before the macOS Catalina epoch is treated as legacy, we leave that unchanged (if enough disk space exists!), and the new volume group representation is created as a full backup, not as an incremental one to the old disk (which consisted of only one volume). The latter would have been technically possible. But possibly too many edge case, for which Apple choose the safer strategy. Although “safe” only as in “safe for the latest backup” but if disk space is low “willingly sacrificing elder backups due to required backup thinning”. I have my understanding for that strategy, but at least a warning before performing that radical procedure of “thinning out everything before the latest backup version” is what I would have minimally expected!
What do you think?
- 76
hoakley Thank you.
There’s actually even more to it than that, as I have described in several articles here, notablythis one.
In short: yes, the first TM backup made by Catalina is essentially a full backup, which includes both the volumes in the boot Volume Group unless you exclude System files. When you let TM back up to an existing series of backups, all the previous backups will also be modified to bring them up to the new format, which is incompatible with previous versions of Time Machine. That may also involve some backup thinning, and has a significant risk of causing problems with those old backups.
You can read more about the new boot disk structure in Catalinahere.
For many months now, I have been recommending that, if at all possible, users should archive their old TM backups before upgrading to Catalina, and then starting a fresh set of backups. I also recommend that users should exclude the versions database (a hidden folder at the root level of the Data volume, and any external volumes), which can cause Catalina’s backups to choke.
Howard.LikeLike
- 76
- 77
tslarkin I have an external drive which is mounted, using an entry in /etc/fstab/, at /Users. This drive had been backed up correctly until I installed Catalina last October. I did not discover this until I needed to recover some files this morning. (Fortunately I also had a CrashPlan backup.) The boot drive has been backed up correctly.
Tmutil tells me that /Users is excluded from backup. I have tried to remove the exclusion with “sudo tmutil removeexclusion -v /Users”, but this does not change the exclusion status.
/Library/Preferences/com.apple.TimeMachine.plist has an array for IncludedVolumeUUIDs which lists the UUID for /Users. So /Users is both an included volume and an excluded volume.
I looked in .exclusions.plist in the directory for the most recent TM backup. It includes /Volumes.
Any idea how I can get TM to revoke this exclusion?
- 78
hoakley Thank you.
I’m not sure that arrangement is going to work with Catalina’s new Volume Group. You might try replacing that with a synthetic firmlinkas explained here.
Howard.LikeLike
- 79
tslarkin Thanks for the suggest to look into the volume group.
Using fstab a device can be mounted at /Users, but it is mounted on the System volume, at /System/Volumes/Data/Users, not the Data volume. Due to group magic, I can then write to /Users, but Time Machine excludes /Users from backup. Other root directories can be included, whether they are at /System/Volumes/Data or at /, but I could not find a simple rule for predicting includability: This is a distinction made by Time Machine according to its internal logic.
For instance, considering your diagrams of the layout of System and Data, it appears that /home is a symlink on System, while the directory is on Data. Nonetheless, /home is irrevokably excluded from backup. System/Volumes/Data/mnt, however, is included in backup. I think I could mount my external drive there, and it would be backed up. I’m also certain that my understanding of this is primitive if not substantially wrong, but trying to sort it makes my head hurt.
The original goal was to mount at /Users to take advantage of the default behavior of the OS with regard to that directory. I am reasonably confident that I cannot mount a device there and get a TM backup of it. If I can’t mount at /Users, then there’s no alternative to modifying each user account’s home directory property, and so no reason not to do the simple thing, and mount at /Volumes.
I don’t know yet how modifying my account’s home directory will work out. I remember that I went the fstab route many years ago when I discovered that doing the account modification caused problems in some applications. I know, for instance, that changing my home from /Users to /Volumes/Users will cause problems when opening old documents belonging to QGIS, a GIS application, which refer to constituent data files using fixed paths from the root.
I have made some progress, even if I didn’t get the solution I wanted. Thanks for your help in moving me forward.
- 80
hoakley Thank you.
I think the problem here is that /Users is firmlinked from the System volume (at its root level) to that directory on the Data volume. Anything which looks for /Users at the top level should follow that firmlink and end up on your Data volume, rather than your external device. Because of these architectural changes in the Volume Group, I would have thought that you’d be much better off mounting the contents of that external device in the /Users directory of your Data volume – then all paths are preserved, and the firmlink to /Users on the Data volume holds good.
There’s no option to change the system firmlinks, only to add synthetic firmlinks to them.
Howard.LikeLike
- 80
- 79
- 78
- 81
qednet Thanks for posting about this. Recently I updated my MacBook Pro from High Sierra to Catalina which broke a lot of my legacy apps and ran like a dog. I tried to revert using TimeMachine and that’s where the fun began.
Here’s what I discovered.
Apple (documented but not advised during the update) no longer support HFS+ and consequently the update to Catalina also converts your old HFS+ file system to the new APFS. This creates a problem if you want to restore because TimeMachine wont play along.
The second thing is in the process your old timemachine archive get’s converted (as noted here) to a .backupbundle which you need to change back to a sparsebundle to be readable by TimeMachine using OSX 10.13. Unfortunately I also discovered I could only do that as root on my Mac. This had me completely lost for a bit as I returned the Mac to HFS+ and installed a base version of High Sierra but still couldn’t access the old backups until the extension was changed. I could have used the host system to make the change (my TimeMachine vault is a linux box) but I was a bit stubborn about using OSX to fix it’s own problems. Anyway once I activated root access on the Mac I could easily change the backup to a .sparsebundle and everything suddenly worked as it should.
So if you continue to use HFS+ and have an older version of OSX be warned. TimeMachine will save you eventually if you want to unwind an system update but it’s not as simple as you might think.
LikeLike
- 82
hoakley Thank you.
It’s no secret that upgrading to Mojave or Catalina converts all boot disks to APFS – that’s been well-known since the release of Mojave. What’s more, Catalina changes the volume layout of boot disks completely to split the single startup volume of Mojave into a Volume Group, consisting of a System volume which is read-only and protected by SIP, and a Data volume, which is writeable. Those can’t be restored to any version of macOS prior to Catalina, as they didn’t exist then and are unsupported by 10.14 and earlier. Again, this is no secret, and has been well-described even before the release of Catalina last year.
I have dozens of articles here which look at these and related issues.
Howard.LikeLike
- 82
- 83
Joe Thank you for the advices, very useful.
Unfortunately, I have upgraded to new version before reading your article. I have to clear the TM disk and re-backup from zero.
As you suggested, I contacted Apple support and let them pay attention to this.
And more importantly, I will DO turn off TM before upgrading macOS to next big version.
·Comments are closed.
The Eclectic Light Company