Making NSFetchRequest.fetchBatchSize Work With Swift

Apple Frameworks Engineer:

Set in Swift is an immutable value type. We do not recommend making Core Data relationships typed this way despite the obvious convenience. Core Data makes heavy use of Futures, especially for relationship values. These are reference types expressed asNSSet. The concrete instance is a future subclass however. This lets us optimize memory and performance across your object graph. Declaring an accessor asSet forces an immediate copy of the entire relationship so it can be an immutable SwiftSet. This loads the entire relationship up front and fulfills the Future all the time, immediately. You probably do not want that.

It’s so convenient, though, and often it doesn’t matter because it’s a small relationship or one that you will be fully accessing anyway. Perhaps the answer is to provide a duplicate set ofNSSet accessors for use when you want the lazy behavior enabled by the class cluster.

Similarly for fetch requests with batching enabled, you do not want a SwiftArray but instead anNSArray to avoid making an immediate copy of the future.

Needless to say, thedocumentation doesn’t mention this, but it does do a good job of explaining whatfetchBatchSize does:

If you set a nonzero batch size, the collection of objects returned when an instance ofNSFetchRequest is executed is broken into batches. When the fetch is executed, the entire request is evaluated and the identities of all matching objects recorded, but only data for objects up to thebatchSize will be fetched from the persistent store at a time. The array returned from executing the request is a proxy object that transparently faults batches on demand. (In database terms, this is an in-memory cursor.)

You can use this feature to restrict the working set of data in your application. In combination withfetchLimit, you can create a subrange of an arbitrary result set.

Under the hood, this works by eagerly fetching the object IDs and lazily fetching and caching the objects, in batches, as they are accessed. The implementation is more optimized than what you could implement yourself, passing the object IDs to SQLite via temporary tables rather than as parameters to the SQL statement. There are some caveats to be aware of:

• If you’re using a coordinator with multiple stores, it will get the sorting right, fetching multiple batches and merging them together without ever doing a giant fetch. However, it does seem to eventually load all the objects into memory at once, which mostly defeats the purpose of batching. If youcan hold everything in memory but just prefer not to, I guess you could refault all the objects after the sorting has completed and let the special array bring them back as needed. Or, you can avoid combiningfetchBatchSize with multiple stores and instead use a dictionary fetch request to get just the object IDs and the properties needed for sorting, save the IDs, and manually fetch batches of full objects as needed.

• I’m a little worried that there are bugs related to multiple stores. Disassembling_PFBatchFaultingArray shows code that anticipates sometimes receiving more object IDs than it expected to fetch, and this hasoccurred in the wild. It’s looks as if Core Data is querying theZ_PK without regard for which store it’s supposed to be in. However, I tried to reproduce this situation by deliberately creating objects in multiple stores with the sameZ_PK and everything seemed to work as expected on macOS 10.15.7.

So, how do you get the optimizedfetchBatchSize behavior when using Swift? The Apple engineer suggests using anNSArray, which I take to mean casting the result of the fetch viaas NSArray to disabling automatic bridging and give your Swift code the originalNSArray. However, my experience is that this doesn’t work. All the objects get fetched before your code even accesses the array. I think it’s because the specialas behavior is for disabling bridging when calling Objective-C APIs from Swift, butNSManagedObjectContext.fetch(_:) is an overlay method implemented in Swift, not just a renaming of-[NSManagedObjectContext executeFetchRequest:error:].

This can be worked around by using an Objective-C category to expose the original method:

@interface NSManagedObjectContext (MJT)- (nullable NSArray *)mjtExecuteFetchRequest:(NSFetchRequest *)request error:(NSError **)error;@end@implementation NSManagedObjectContext (MJT)- (nullable NSArray *)mjtExecuteFetchRequest:(NSFetchRequest *)request error:(NSError **)error {    return [self executeFetchRequest:request error:error];}@end

Then you can implement a fetching method that preserves the batching behavior:

public extension NSManagedObjectContext {    func fetchNSArray<T: NSManagedObject>(_ request: NSFetchRequest<T>) throws -> NSArray {        // @SwiftIssue: Doesn't seem like this cast should be necessary.        let protocolRequest = request as! NSFetchRequest<NSFetchRequestResult>                return try mjtExecute(protocolRequest) as NSArray    }    func fetch<T: NSManagedObject>(_ request: NSFetchRequest<T>,                                   batchSize: Int) throws -> MJTBatchFaultingCollection<T> {        request.fetchBatchSize = batchSize        return MJTBatchFaultingCollection(array: try fetchNSArray(request))    }}

The first method gives you theNSArray, but that is not very ergonomic to use from Swift. First, you have to cast the objects back to yourNSManagedObject subclass. Second, it doesn’t behave well when an object is deleted (or some other SQLite error occurs) between your fetch and when Core Data tries to fulfill the fault.

If you’re using Swift, you can’t catch theNSObjectInaccessibleException, so you should be usingcontext.shouldDeleteInaccessibleFaults = true. This means that instead of an exception you get a sort of tombstone object that’s of the right class, but with all its properties erased.

But it’s hard to remember to check for that each time you use one of the objects in theNSArray, and you probably don’t want to accidentally operate on the empty properties. So the second method uses a helper type to try to make the abstraction less leaky, always giving you either a valid, non-fault object ornil:

public struct MJTBatchFaultingCollection<T: NSManagedObject> {    let array: NSArray    let bounds: Range<Int>    // array is presumed to be a _PFBatchFaultingArray from a fetch request    // using fetchBatchSize.    public init(array: NSArray, bounds: Range<Int>? = nil) {        self.array = array        self.bounds = bounds ?? 0..<array.count    }}extension MJTBatchFaultingCollection: RandomAccessCollection {    public typealias Element = T?    public typealias Index = Int    public typealias SubSequence = MJTBatchFaultingCollection<T>    public typealias Indices = Range<Int>        public var startIndex: Int { bounds.lowerBound }    public var endIndex: Int { bounds.upperBound }           public subscript(position: Index) -> T? {        guard            let possibleFault = array[position] as? T,            let context = possibleFault.managedObjectContext,            // Unfault so that isDeleted will detect an inaccessible object.            let object = try? context.existingObject(with: possibleFault.objectID),            let t = object as? T else { return nil }        return t.isDeleted ? nil : t    }    public subscript(bounds: Range<Index>) -> SubSequence {        MJTBatchFaultingCollection<T>(array: array, bounds: bounds)    }}extension MJTBatchFaultingCollection: CustomStringConvertible {    public var description: String {        // The default implementation would realize all the objects by printing        // the underlying NSArray.        return "<MJTBatchFaultingCollection<\(T.self)> bounds: \(bounds)>"    }}

It’s still a bit leaky, because you have to be careful to only access the collection from the context’s queue. But this is somewhat obvious because it has a separate type, so you’ll get an error if you try to pass it to a method that takes anArray.

The batch faulting behavior and batch size are preserved if you iterate over the collection or slice it. (When iterating theNSArray directly, small batch sizes don’t work as expected becauseNSFastEnumerationIterator will always load at least 16 objects at a time.)

Previously:

• Replacing vs. Migrating Core Data Stores
• Apple Developer Forums Can Now Monitor Threads
• Surprising Behavior of Non-optional @NSManaged Properties
• Swift 4: Bridging Peephole for “as” Casts
• Core Data Type Safety With Swift
• Double Core Data Accessors by Omitting @NSManaged
• Core Data in El Capitan
• Using Core Data With Swift

Core DataDocumentationiOSiOS 14MacmacOS 10.15 CatalinamacOS 11.0 Big SurObjective-COptimizationProgrammingSwift Programming Language

CommentsRSS ·Twitter

Leave a Comment

←  →

Support this site viaPatreon.

Try my Mac apps:

DropDMG
EagleFiler
SpamSieve
ToothFairy