Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Improve email regexp on edge cases#10601

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
sydney-runkle merged 3 commits intopydantic:mainfromAlekseyLobanov:fix.email-regex
Oct 11, 2024

Conversation

AlekseyLobanov
Copy link
Contributor

@AlekseyLobanovAlekseyLobanov commentedOct 10, 2024
edited
Loading

  • Drastically improves performance on cases like"<" + " " * N
  • Last spaces are not needed anyway because this group is stripped later. Also spaces will be caught by. anyway.

Change Summary

I found that one single change in email regexp solves slowdowns on special invalid email strings. See related issue for details

Related issue number

Fixes#10600

Checklist

  • The pull request title is a good summary of the changes - it will be used in the changelog
  • Unit tests for the changes exist
  • Tests pass on CI
  • Documentation reflects the changes where applicable
  • My PR is ready to review,please add a comment including the phrase "please review" to assign reviewers

Selected Reviewer:@sydney-runkle

- Drastically improves performance on cases like `"<" + " " * N`- Last spaces are not needed anyway because this group isstripped later. Also spaces will be caught by `.` anyway.
@github-actionsgithub-actionsbot added the relnotes-fixUsed for bugfixes. labelOct 10, 2024
@AlekseyLobanov
Copy link
ContributorAuthor

please review

pydantic-hooky[bot] reacted with thumbs up emoji

@codspeed-hqCodSpeed HQ
Copy link

codspeed-hqbot commentedOct 10, 2024
edited
Loading

CodSpeed Performance Report

Merging#10601 willnot alter performance

ComparingAlekseyLobanov:fix.email-regex (e35c507) withmain (c772b43)

Summary

✅ 38 untouched benchmarks

@AlekseyLobanov
Copy link
ContributorAuthor

AlekseyLobanov commentedOct 10, 2024
edited
Loading

How performance changes?
I use my own POC in#10600 and run it as/usr/bin/time python pydantic-poc.py 500

  • Before: 5.35user 0.01system 0:05.37elapsed 99%CPU (0avgtext+0avgdata 36840maxresident)k
  • After: 0.20user 0.01system 0:00.22elapsed 99%CPU (0avgtext+0avgdata 37004maxresident)k

About25x speed improvement.

@github-actionsGitHub Actions
Copy link
Contributor

github-actionsbot commentedOct 10, 2024
edited
Loading

Coverage report

This PR does not seem to contain any modification to coverable code.

Copy link
Member

@ViicosViicos left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks, this looks reasonable but to be extra careful we'll wait for other reviews as well.

Could you add the following to thetest_address_valid test?:

        ('Samuel Colvin < s@muelcolvin.com>','Samuel Colvin','s@muelcolvin.com'),        ('Samuel Colvin <s@muelcolvin.com >','Samuel Colvin','s@muelcolvin.com'),        ('Samuel Colvin < s@muelcolvin.com >','Samuel Colvin','s@muelcolvin.com'),

Co-authored-by: Victorien <65306057+Viicos@users.noreply.github.com>
@AlekseyLobanov
Copy link
ContributorAuthor

Thanks, this looks reasonable but to be extra careful we'll wait for other reviews as well.

According toWikipedia it is one of the valid DoS attack vectors. And at least some of known to me rate limiters will work onlyafter validation step.

Could you add the following to the test_address_valid test?

I think that existing tests are already covering this edge cases (spaces before/after the group). Should I still add yours?

        ('foo BAR <foobar@example.com >','foo BAR','foobar@example.com'),        ('FOO bar   <foobar@example.com> ','FOO bar','foobar@example.com'),        ('Whatever < foobar@example.com>','Whatever','foobar@example.com'),
sydney-runkle reacted with thumbs up emoji

@sydney-runklesydney-runkle added the relnotes-performanceUsed for performance improvements. labelOct 11, 2024
@sydney-runkle
Copy link
Contributor

Yep let's add those extra tests and fix the lints, but otherwise LGTM.

@Viicos
Copy link
Member

According toWikipedia it is one of the valid DoS attack vectors. And at least some of known to me rate limiters will work onlyafter validation step.

I agree, just wanted to be careful as changing regex can be a source of breaking changes.

I think that existing tests are already covering this edge cases (spaces before/after the group). Should I still add yours?

Missed these ones, then maybe only add these ones after it:

        ('Whatever <foobar@example.com >','Whatever','foobar@example.com'),        ('Whatever < foobar@example.com >','Whatever','foobar@example.com'),
sydney-runkle reacted with thumbs up emoji

Covering name + email case with spaces surrounding the email
@AlekseyLobanov
Copy link
ContributorAuthor

Missed these ones, then maybe only add these ones after it:

Done.

sydney-runkle reacted with heart emoji

Copy link
Contributor

@sydney-runklesydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Nice, thanks for the help here! We appreciate the thorough explanations / refs :).

@sydney-runklesydney-runkleenabled auto-merge (squash)October 11, 2024 14:15
@sydney-runklesydney-runkle merged commit37d98a8 intopydantic:mainOct 11, 2024
57 checks passed
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers

@ViicosViicosViicos left review comments

@sydney-runklesydney-runklesydney-runkle approved these changes

Assignees

@sydney-runklesydney-runkle

Labels
ready for reviewrelnotes-fixUsed for bugfixes.relnotes-performanceUsed for performance improvements.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Email parsing slowdown on edgecases
3 participants
@AlekseyLobanov@sydney-runkle@Viicos

[8]ページ先頭

©2009-2025 Movatter.jp