- Notifications
You must be signed in to change notification settings - Fork302
Feature/nested prefetching#964
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Closed
Closed
Uh oh!
There was an error while loading.Please reload this page.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
Closing. Closed PR will exist as a record of "future-ish plan". Will spin up some cleaner, smaller PRs to trickle these changes in in smaller chunks. See:#921 (reply in thread) |
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading.Please reload this page.
Addresses#921,#337
Problem:
We need n+1 handling to work when nesting includes.
Currently the two offerings in drfjsonapi don't cut it:
PreloadIncludesMixin
forces unnecessary manual effort, and is fundamentally flawed in that it cannot be set up to consider nesting properly. Either you don't prefetch and incur n+1, or you do prefetch, but you do so even when the given request doesn't need the data. Also the manual configuration is a headache to maintain.AutoPrefetchMixin
doesn't recurse, nor does it offer any way of opting out of fetching the additional data if sparsefieldsets haven't asked for it. Specifically, for reverse relations and M2Ms, there are three cases to consider: [1] don't return the field at all (no prefetch), [2] return only the ids (simple prefetch), and [3] return the entire relations (deep prefetch and recurse this same logic at the next level down).This PR introduces a no-config approach, which'll
INCLUDE_EXPENSVE_FIELDS
, to still include expensive relations if one so wants (negating the first point, allowing an easier upgrade path).prefetch_related
andselect_related
calls to prepare exactly the data that's needed.Obvs this is a large change and would require an appropriately-sized version bump to boot.
We're actually using this in production presently, having rolled it out not so long ago. It's cleaned up a lot of mess in our serializer definitions, and also provided a much better peace of mind that nothing is being missed.
I'm happy to babysit this through to getting merged, improving test coverage and backwards compatibility (it's currently only py39 compliant, but i can strip back some things to fix that), but wanted to get an early set of eyes on it, so that i can have a guarantee it'll be merged and my efforts aren't for nought.
Cheers!