Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

gh-116738: Make _json module safe in the free-threading build#119438

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
kumaraditya303 merged 53 commits intopython:mainfromeendebakpt:json_ft
Aug 31, 2025

Conversation

@eendebakpt
Copy link
Contributor

@eendebakpteendebakpt commentedMay 22, 2024
edited
Loading

(updated description)

Writing JSON files (or encoding to a string) is not thread-safe in the sense that when encoding data to json while another thread is mutating the data, the result is not well-defined (this is true for both the normal and free-threading build). But the free-threading build can crash the interpreter while writing JSON because of the usage of methods likePySequence_Fast_GET_ITEM. In this PR we make the free-threading build safe by adding locks in three places in the JSON encoder.

Reading from a JSON file is safe: objects constructed are only known to the executing thread. Encoding data to JSON needs a bit more care: mutable Python objects such as a list or a dict could be modified by another thread during encoding.

  • When encoding a list usePy_BEGIN_CRITICAL_SECTION_SEQUENCE_FAST to project against mutation the list
  • When encoding a dict, we use a critical section for iteration over exact dicts (PyDict_Next is used there). The non-exact dicts usePyMapping_Items to create a list of tuples.PyMapping_Items itself is assumed to be thread safe, but the resulting list is not a copy and can be mutated.

Update 2025-02-10: refactored to avoid using Py_EXIT_CRITICAL_SECTION_SEQUENCE_FAST

  • The script below was used to test the free-threading implementation. Similar code was added to the tests.
Test script
import jsonfrom threading import Threadimport timeclass JsonThreadingTest:        def __init__(self, number_of_threads=4, number_of_json_dumps=10):            self.data = [ [], [], {}, {}, {}]        self.json = {str(ii): d for ii, d in enumerate(self.data)}        self.results =[]        self.number_of_threads=number_of_threads        self.number_of_json_dumps =number_of_json_dumps                def modify(self, index):        while self.continue_thread:            for d in self.data:                if isinstance(d, list ):                    if len(d)>20:                        d.clear()                    else:                        d.append(index)                else:                    if len(d)>20:                        try:                            d.pop(list(d)[0])                        except KeyError:                            pass                    else:                        if index%2:                                                        d[index] = index                        else:                            d[bytes(index)] = bytes(index)                        def test(self):        self.continue_thread = True        self.modifying_threads = []        for ii in range(self.number_of_threads):            t = Thread(target=self.modify, args=[ii])            self.modifying_threads.append(t)        self.results.clear()        for t in self.modifying_threads:            print(f'start {t}')            t.start()                    for ii in range(self.number_of_json_dumps):            print(f'dump {ii}')            time.sleep(0.01)                        indent = ii if ii%3==0 else None            if ii%5==0:                try:                    j = json.dumps(self.data, indent=indent, skipkeys=True)                except TypeError:                        pass            else:                j = json.dumps(self.data, indent=indent)            self.results.append(j)        self.continue_thread= False                print([hash(r) for r in self.results])            t=JsonThreadingTest(number_of_json_dumps=102, number_of_threads=8)t0=time.time()t.test()dt=time.time()-t0print(t.results[-1])        print(f'Done: {dt:.2f}')
  • The test script witht=JsonThreadingTest(number_of_json_dumps=102, number_of_threads=8) is a factor 25 faster using free-threading. Nice!

@nineteendo
Copy link
Contributor

You need to include the file that defines that macro.

Copy link
Contributor

@nineteendonineteendo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Revert newlines

Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com>
@eendebakpteendebakpt changed the titleDraft: gh-116738: Make _json module thread-safe #117530gh-116738: Make _json module thread-safe #117530May 31, 2024
@eendebakpteendebakpt changed the titlegh-116738: Make _json module thread-safe #117530gh-116738: Make _json module thread-safeMay 31, 2024
@eendebakpteendebakpt changed the titlegh-116738: Make _json module thread-safegh-116738: Make _json module safe in the free-threading buildAug 14, 2024
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Copy link
Contributor

@kumaraditya303kumaraditya303 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

Thanks!

@kumaraditya303kumaraditya303 merged commit4357302 intopython:mainAug 31, 2025
45 checks passed
lkollar pushed a commit to lkollar/cpython that referenced this pull requestSep 9, 2025
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

@kumaraditya303kumaraditya303kumaraditya303 approved these changes

+1 more reviewer

@nineteendonineteendonineteendo left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@eendebakpt@nineteendo@kumaraditya303

[8]ページ先頭

©2009-2025 Movatter.jp