Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

Implement a safe_url based on all standards#221

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Draft
Gallaecio wants to merge36 commits intoscrapy:master
base:master
Choose a base branch
Loading
fromGallaecio:safer-url
Draft
Changes from1 commit
Commits
Show all changes
36 commits
Select commitHold shift + click to select a range
4798aaf
test_safe_url_idna: adjust test data to the test scope
GallaecioNov 3, 2022
86711dd
Fix pytest-cov warning
GallaecioNov 4, 2022
030a996
Implement safe_url and provide partial test coverage
GallaecioNov 8, 2022
6f0dc16
Merge remote-tracking branch 'upstream/master' into safer-url
GallaecioNov 8, 2022
17ee8d6
Progress on typing
GallaecioNov 8, 2022
aef0532
Address typing issues
GallaecioNov 9, 2022
abe4032
Fix unicode encoding in userinfo and IPv6 parsing
GallaecioNov 9, 2022
86435df
Progress on tests
GallaecioNov 11, 2022
79193bf
Address issues reported by tests and static checkers
GallaecioNov 15, 2022
82f6e66
Complete test coverage for safe_url
GallaecioNov 15, 2022
81fc830
Progress on passing upstream tests
GallaecioNov 15, 2022
7e72474
Solve issues previously marked as expected failures
GallaecioNov 16, 2022
2c62859
Validate host parsing according to upstream URL living standard tests
GallaecioNov 16, 2022
3623c35
Cover port and path in upstream tests
GallaecioNov 17, 2022
2513ea1
Cover query and fragment in upstream tests
GallaecioNov 17, 2022
7d92740
Add upstream tests for domain_to_ascii
GallaecioNov 17, 2022
6a080da
Add upstream tests for percent encoding
GallaecioNov 17, 2022
79eafeb
Add a performance test
GallaecioNov 17, 2022
3028165
Add idna.txt to source distribution
GallaecioNov 17, 2022
091905e
2x speed-up at the cost of mishandling invalid code points
GallaecioNov 17, 2022
fea33a6
_percent_encode_after_encoding: 0.2 speed-up
GallaecioNov 17, 2022
184e5a7
0.1 speed-up
GallaecioNov 17, 2022
4c21a0e
x1.2 speed up
GallaecioNov 17, 2022
c317f9e
x1.67 speed up
GallaecioNov 17, 2022
7152d0b
Remove unused code
GallaecioNov 17, 2022
9daca87
x1.125 speed up
GallaecioNov 17, 2022
88f32ae
Use urllib.parse.unquote
GallaecioNov 18, 2022
63a2f2f
Initial use of Cython (x7 speedup)
GallaecioFeb 12, 2024
eefbb9e
Use Cython for additional files (x1.5 speedup)
GallaecioFeb 12, 2024
8185cdc
Merge _encoding into _url and use uchar for state control
GallaecioFeb 13, 2024
bd15eb9
Update test expectations
GallaecioFeb 13, 2024
5b18c9a
Merge remote-tracking branch 'scrapy/master' into safer-url
GallaecioFeb 13, 2024
4960889
WIP
GallaecioFeb 13, 2024
25ecb97
Refactor _URL
GallaecioFeb 13, 2024
66b9154
Minor improvements
GallaecioFeb 13, 2024
8f1f809
Minor change
GallaecioFeb 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
PrevPrevious commit
NextNext commit
Cover query and fragment in upstream tests
  • Loading branch information
@Gallaecio
Gallaecio committedNov 17, 2022
commit2513ea1b62c025cfe41d4e896f0547b60b416246
25 changes: 22 additions & 3 deletionstests/test_url.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -53,7 +53,7 @@


@pytest.mark.parametrize(
"input,base,failure,href,protocol,username,password,hostname,port,pathname",
"input,base,failure,href,protocol,username,password,hostname,port,pathname,search,hash",
(
case
if case[0] not in URL_TEST_DATA_KNOWN_ISSUES
Expand All@@ -70,14 +70,27 @@
i.get("hostname"),
i.get("port"),
i.get("pathname"),
i.get("search"),
i.get("hash"),
)
for i in URL_TEST_DATA
if not isinstance(i, str)
)
),
)
def test_parse_url(
input, base, failure, href, protocol, username, password, hostname, port, pathname
input,
base,
failure,
href,
protocol,
username,
password,
hostname,
port,
pathname,
search,
hash,
):
if failure:
with pytest.raises(ValueError):
Expand All@@ -93,7 +106,13 @@ def test_parse_url(
# TODO: Find out why we do not always get right whether path is supposed to
# be / or an empty string.
assert (_serialize_url_path(url) or "/") == (pathname or "/")
# TODO: Cover additional fields
# TODO: Find out why we do not always get right whether query is supposed
# to be an empty string or None.
assert (url.query or "") == (search[1:] if search else "")
# TODO: Find out why we do not always get right whether fragment is
# supposed to be an empty string or None.
assert (url.fragment or "") == (hash[1:] if hash else "")
# TODO: Address the TODOs above.
# assert _serialize_url(url) == href


Expand Down

[8]ページ先頭

©2009-2025 Movatter.jp