- Notifications
You must be signed in to change notification settings - Fork1k
feat: add a paginated organization members endpoint#16835
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
coderd/coderd.go Outdated
r.Route("/paginated-members",func(r chi.Router) { | ||
r.Get("/",api.paginatedMembers) | ||
}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Really unfortunate we cannot just use the/members
endpoint without it being a breaking change.
Since organizations are new, I think it's worth considering.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I was originally going to to augment the existing route to be paginated but@aslilac brought up the issue of backwards compatibility. I can circle back with her, and other stakeholders on this
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
if we were having this discussion like two weeks ago I think I'd agree with you@Emyrk, but 2.20 is out now and marketing organizations as stable. :\
coderd/database/dbauthz/dbauthz.go Outdated
} | ||
func (q*querier)PaginatedOrganizationMembers(ctx context.Context,arg database.PaginatedOrganizationMembersParams) ([]database.PaginatedOrganizationMembersRow,error) { | ||
returnfetchWithPostFilter(q.auth,policy.ActionRead,q.db.PaginatedOrganizationMembers)(ctx,arg) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Unfortunately this does not work.
By default, amember
can only read themselves, not other org members. So if you were paginating 10 users at a time, and the user was the 12th. Your query would return 10 rows, all would fail theauthz
check, and you would get an empty set as your return.
You would need tooffset = 2
to find yourself, but that is not intuitive.
Pagination is a general problem for authorization engines. The user is requesting "The next 10 elements I can access". If you cannot join the authz data into the query itself then:
- Query 10 items, if some are blocked by the authz, fetch some more. Repeat until you get 10 or exhaust the table.
- Fetch all elements, then filter (large table this is not doable)
- Overfetch by some, say 50 elements. Keep filtering and re-fetching until you get 10.
These solutions kinda suck. So we have 2 methods in our codebase to solve this.
Quick anddirty
The first is the quick solution. We just restrict access to this endpoint to users who can read the entire table (withorganization_id = @org_id
). This functionally changes the query to only usable by a certain class of users. Regular members now get aForbidden
instead of a list of["me"]
. If we go this route, we just need to fix the page that currently displays just yourself in these cases.
func (q*querier)PaginatedOrganizationMembers(ctx context.Context,arg database.PaginatedOrganizationMembersParams) ([]database.PaginatedOrganizationMembersRow,error) {iferr:=q.authorizeContext(ctx,policy.ActionRead,rbac.ResourceOrganizationMember.InOrg(arg.OrganizationID));err!=nil {returnnil,err}returnq.db.PaginatedOrganizationMembers(ctx,arg)}
Thecorrect solution
The more correct solution is to properly filter the list, so if there exists a user who can read a subset of members, this query still works.
A good example is workspaces
coder/coderd/database/dbauthz/dbauthz.go
Lines 3018 to 3024 in902538c
func (q*querier)GetWorkspaces(ctx context.Context,arg database.GetWorkspacesParams) ([]database.GetWorkspacesRow,error) { | |
prep,err:=prepareSQLFilter(ctx,q.auth,policy.ActionRead,rbac.ResourceWorkspace.Type) | |
iferr!=nil { | |
returnnil,xerrors.Errorf("(dev error) prepare sql filter: %w",err) | |
} | |
returnq.db.GetAuthorizedWorkspaces(ctx,arg,prep) | |
} |
The functionprepareSQLFilter
actually createsWHERE
clause expressions that return the truthy value of anauthorize()
call for the given user. So we apply the filter in SQL, and you can paginate through the rows you can access.
There is some extra code required, but we have precedent to copy from if we go this route.
I think we should just go the quick and dirty route in this case. We don't have any roles that grant subset access to members. It is also strange to request organization members, and only return a subset.
For workspaces/templates this makes sense, as the resources have access control. However in this case, I think it makes more sense to return all or nothing. And we can update the UI to be in line with this decision.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Thanks for the feedback/explanation! I agree that from a product perspective the quick and dirty route makes the most sense
coderd/coderd.go Outdated
r.Route("/paginated-members",func(r chi.Router) { | ||
r.Get("/",api.paginatedMembers) | ||
}) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
r.Route("/paginated-members",func(r chi.Router) { | |
r.Get("/",api.paginatedMembers) | |
}) | |
r.Get("/paginated-members",api.paginatedMembers) |
I wouldn't bother with the subrouter here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I saw this was a pattern we were doing in other places so copied that. Will change
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I had an idea that could avoid breakage, avoid needing an extra endpoint, but that I'm not sure is actually a good idea:
what if we did just update the existing /members route, butonly if specify some pagination params?
maybe that's too weird to have it return two different types, but it'd also be unfortunate to have to maintain these two routes independently
or maybe we should just make this breaking change, call it out in the release notes, and I'm over thinking it and being too cautious
check.Args(database.PaginatedOrganizationMembersParams{ | ||
OrganizationID:uuid.UUID{}, | ||
OrganizationID:o.ID, | ||
LimitOpt:1, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think this different check means we also don't need this field any more
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Accidentally pushed some test code in the test so not sure which one you were looking at but that's needed.
Steven and I had the same thought, but agreed that it's kinda jank, and also wouldn't play well with the generated docs. For now we are deprecating the route in favor of the new paginated option that can still return all members. |
iferr:=q.authorizeContext(ctx,policy.ActionRead,rbac.ResourceOrganizationMember.InOrg(arg.OrganizationID));err!=nil { | ||
returnnil,err | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Just throw a comment
iferr:=q.authorizeContext(ctx,policy.ActionRead,rbac.ResourceOrganizationMember.InOrg(arg.OrganizationID));err!=nil { | |
returnnil,err | |
} | |
// Required to have permission to read all members in the organization | |
iferr:=q.authorizeContext(ctx,policy.ActionRead,rbac.ResourceOrganizationMember.InOrg(arg.OrganizationID));err!=nil { | |
returnnil,err | |
} |
s.Run("PaginatedOrganizationMembers",s.Subtest(func(db database.Store,check*expects) { | ||
o:=dbgen.Organization(s.T(),db, database.Organization{}) | ||
check.Args(database.PaginatedOrganizationMembersParams{ | ||
OrganizationID:o.ID, | ||
LimitOpt:1, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
Can you add back the org member in the setup? And check it in theReturns
? And set the limit to 0.
That was breaking in thedbmem
becauselimit = 0
was not being treated as "no limit"
coderd/database/dbmem/dbmem.go Outdated
iflen(tmp)>=int(arg.LimitOpt) { | ||
break | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
I think this needs to check if the limitOpt is not 0. If it is zero, skip this check. To match the SQL
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
feel free to wait for steven, but it looks good to me 👍
actionAsString,ok:=inputs[i+1].(string) | ||
if!ok { | ||
panic(fmt.Sprintf("action '%q' not a supported action",actionAsString)) | ||
panic(fmt.Sprintf("action '%T' not a supported action",inputs[i+1])) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
nice catch 😂
8c0350e
intomainUh oh!
There was an error while loading.Please reload this page.
Uh oh!
There was an error while loading.Please reload this page.
Closescoder/internal#460