Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

feat: add a paginated organization members endpoint#16835

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
brettkolodny merged 19 commits intomainfrombrett/org-members-pagination-backend
Mar 10, 2025

Conversation

brettkolodny
Copy link
Contributor

@brettkolodnybrettkolodny commentedMar 6, 2025
edited
Loading

@brettkolodnybrettkolodny changed the titleBrett/org members pagination backendfeat: Add a paginated organization members endpointMar 6, 2025
@brettkolodnybrettkolodny changed the titlefeat: Add a paginated organization members endpointfeat: add a paginated organization members endpointMar 6, 2025
@brettkolodnybrettkolodny marked this pull request as ready for reviewMarch 6, 2025 22:58
@brettkolodnybrettkolodny requested a review fromEmyrkMarch 6, 2025 23:26
Comment on lines 1005 to 1007
r.Route("/paginated-members",func(r chi.Router) {
r.Get("/",api.paginatedMembers)
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Really unfortunate we cannot just use the/members endpoint without it being a breaking change.

Since organizations are new, I think it's worth considering.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I was originally going to to augment the existing route to be paginated but@aslilac brought up the issue of backwards compatibility. I can circle back with her, and other stakeholders on this

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

if we were having this discussion like two weeks ago I think I'd agree with you@Emyrk, but 2.20 is out now and marketing organizations as stable. :\

}

func (q*querier)PaginatedOrganizationMembers(ctx context.Context,arg database.PaginatedOrganizationMembersParams) ([]database.PaginatedOrganizationMembersRow,error) {
returnfetchWithPostFilter(q.auth,policy.ActionRead,q.db.PaginatedOrganizationMembers)(ctx,arg)
Copy link
Member

@EmyrkEmyrkMar 6, 2025
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Unfortunately this does not work.

By default, amember can only read themselves, not other org members. So if you were paginating 10 users at a time, and the user was the 12th. Your query would return 10 rows, all would fail theauthz check, and you would get an empty set as your return.

You would need tooffset = 2 to find yourself, but that is not intuitive.

Pagination is a general problem for authorization engines. The user is requesting "The next 10 elements I can access". If you cannot join the authz data into the query itself then:

  • Query 10 items, if some are blocked by the authz, fetch some more. Repeat until you get 10 or exhaust the table.
  • Fetch all elements, then filter (large table this is not doable)
  • Overfetch by some, say 50 elements. Keep filtering and re-fetching until you get 10.

These solutions kinda suck. So we have 2 methods in our codebase to solve this.

Quick anddirty

The first is the quick solution. We just restrict access to this endpoint to users who can read the entire table (withorganization_id = @org_id). This functionally changes the query to only usable by a certain class of users. Regular members now get aForbidden instead of a list of["me"]. If we go this route, we just need to fix the page that currently displays just yourself in these cases.

func (q*querier)PaginatedOrganizationMembers(ctx context.Context,arg database.PaginatedOrganizationMembersParams) ([]database.PaginatedOrganizationMembersRow,error) {iferr:=q.authorizeContext(ctx,policy.ActionRead,rbac.ResourceOrganizationMember.InOrg(arg.OrganizationID));err!=nil {returnnil,err}returnq.db.PaginatedOrganizationMembers(ctx,arg)}

Screenshot From 2025-03-06 17-38-01

Thecorrect solution

The more correct solution is to properly filter the list, so if there exists a user who can read a subset of members, this query still works.

A good example is workspaces

func (q*querier)GetWorkspaces(ctx context.Context,arg database.GetWorkspacesParams) ([]database.GetWorkspacesRow,error) {
prep,err:=prepareSQLFilter(ctx,q.auth,policy.ActionRead,rbac.ResourceWorkspace.Type)
iferr!=nil {
returnnil,xerrors.Errorf("(dev error) prepare sql filter: %w",err)
}
returnq.db.GetAuthorizedWorkspaces(ctx,arg,prep)
}

The functionprepareSQLFilter actually createsWHERE clause expressions that return the truthy value of anauthorize() call for the given user. So we apply the filter in SQL, and you can paginate through the rows you can access.

There is some extra code required, but we have precedent to copy from if we go this route.


I think we should just go the quick and dirty route in this case. We don't have any roles that grant subset access to members. It is also strange to request organization members, and only return a subset.

For workspaces/templates this makes sense, as the resources have access control. However in this case, I think it makes more sense to return all or nothing. And we can update the UI to be in line with this decision.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks for the feedback/explanation! I agree that from a product perspective the quick and dirty route makes the most sense

Emyrk reacted with thumbs up emoji
Comment on lines 1005 to 1007
r.Route("/paginated-members",func(r chi.Router) {
r.Get("/",api.paginatedMembers)
})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Suggested change
r.Route("/paginated-members",func(r chi.Router) {
r.Get("/",api.paginatedMembers)
})
r.Get("/paginated-members",api.paginatedMembers)

I wouldn't bother with the subrouter here

Emyrk and brettkolodny reacted with thumbs up emoji
Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I saw this was a pattern we were doing in other places so copied that. Will change

Copy link
Member

@aslilacaslilac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I had an idea that could avoid breakage, avoid needing an extra endpoint, but that I'm not sure is actually a good idea:

what if we did just update the existing /members route, butonly if specify some pagination params?

maybe that's too weird to have it return two different types, but it'd also be unfortunate to have to maintain these two routes independently

or maybe we should just make this breaking change, call it out in the release notes, and I'm over thinking it and being too cautious

check.Args(database.PaginatedOrganizationMembersParams{
OrganizationID:uuid.UUID{},
OrganizationID:o.ID,
LimitOpt:1,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think this different check means we also don't need this field any more

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Accidentally pushed some test code in the test so not sure which one you were looking at but that's needed.

@brettkolodny
Copy link
ContributorAuthor

I had an idea that could avoid breakage, avoid needing an extra endpoint, but that I'm not sure is actually a good idea:

what if we did just update the existing /members route, butonly if specify some pagination params?

maybe that's too weird to have it return two different types, but it'd also be unfortunate to have to maintain these two routes independently

or maybe we should just make this breaking change, call it out in the release notes, and I'm over thinking it and being too cautious

Steven and I had the same thought, but agreed that it's kinda jank, and also wouldn't play well with the generated docs. For now we are deprecating the route in favor of the new paginated option that can still return all members.

aslilac reacted with thumbs up emoji

Comment on lines +3585 to +3587
iferr:=q.authorizeContext(ctx,policy.ActionRead,rbac.ResourceOrganizationMember.InOrg(arg.OrganizationID));err!=nil {
returnnil,err
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Just throw a comment

Suggested change
iferr:=q.authorizeContext(ctx,policy.ActionRead,rbac.ResourceOrganizationMember.InOrg(arg.OrganizationID));err!=nil {
returnnil,err
}
// Required to have permission to read all members in the organization
iferr:=q.authorizeContext(ctx,policy.ActionRead,rbac.ResourceOrganizationMember.InOrg(arg.OrganizationID));err!=nil {
returnnil,err
}

brettkolodny reacted with thumbs up emoji
Comment on lines 988 to 993
s.Run("PaginatedOrganizationMembers",s.Subtest(func(db database.Store,check*expects) {
o:=dbgen.Organization(s.T(),db, database.Organization{})

check.Args(database.PaginatedOrganizationMembersParams{
OrganizationID:o.ID,
LimitOpt:1,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Can you add back the org member in the setup? And check it in theReturns? And set the limit to 0.

That was breaking in thedbmem becauselimit = 0 was not being treated as "no limit"

brettkolodny and aslilac reacted with thumbs up emoji
Comment on lines 9609 to 9611
iflen(tmp)>=int(arg.LimitOpt) {
break
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

I think this needs to check if the limitOpt is not 0. If it is zero, skip this check. To match the SQL

brettkolodny and aslilac reacted with thumbs up emoji
Copy link
Member

@aslilacaslilac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

feel free to wait for steven, but it looks good to me 👍

actionAsString,ok:=inputs[i+1].(string)
if!ok {
panic(fmt.Sprintf("action '%q' not a supported action",actionAsString))
panic(fmt.Sprintf("action '%T' not a supported action",inputs[i+1]))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

nice catch 😂

@brettkolodnybrettkolodny merged commit8c0350e intomainMar 10, 2025
35 checks passed
@brettkolodnybrettkolodny deleted the brett/org-members-pagination-backend branchMarch 10, 2025 18:42
@github-actionsgithub-actionsbot locked and limited conversation to collaboratorsMar 10, 2025
Sign up for freeto subscribe to this conversation on GitHub. Already have an account?Sign in.
Reviewers

@aslilacaslilacaslilac approved these changes

@EmyrkEmyrkAwaiting requested review from Emyrk

Assignees

@brettkolodnybrettkolodny

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add paginated organizaton members backend route
3 participants
@brettkolodny@aslilac@Emyrk

[8]ページ先頭

©2009-2025 Movatter.jp