Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

FIX memoryview leaks and retrofit memory-manager as context-managers#33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Open
ankostis wants to merge19 commits intogitpython-developers:master
base:master
Choose a base branch
Loading
fromankostis:leaks2

Conversation

ankostis
Copy link
Contributor

BREAKING API:

  • retrofitgit.util.mman as context-manager,
    to release memory-mapped regions held.

    Themmap-manager(s) are re-entrant, but not thread-safecontext-manager(s),
    to be used within awith ...: block, ensuring any left-overs cursors are cleaned up.
    If not entered, :meth:StaticWindowMapManager.make_cursor() and/or
    :meth:WindowCursor.use_region() will scream.

    Get them fromsmmap.managed_mmaps().

  • Simplify :class:SlidingWindowMapBuffer as create/close context-manager
    (nobegin_access(), orend_access()).

+ Possible fixinggitpython-developers#31 by stop decrement ondestruction, rely only on `with...` resources.+ feat(mmap): utility to check if regions have been closed (PY3-only).
+ Add PY3 compat utilities+ doc(changes, tutorial): update on mman usage
+ doc(tutorial): update use-cases+ doc(changes): new bullet.
+ All gitdb TCs now pass without explit release!
@coveralls
Copy link

Coverage Status

Coverage decreased (-0.7%) to 93.116% when pulling144891b on ankostis:leaks2 into6e55a1c on gitpython-developers:master.

1 similar comment
@coveralls
Copy link

coveralls commentedOct 27, 2016
edited
Loading

Coverage Status

Coverage decreased (-0.7%) to 93.116% when pulling144891b on ankostis:leaks2 into6e55a1c on gitpython-developers:master.

@ankostis
Copy link
ContributorAuthor

ankostis commentedOct 27, 2016
edited
Loading

@Byron can you check these lineswhere amemoryview is crated?

It took me the half month to realize that this is leaking badly, and I want you to check the history of this commits leading up to it, because I cannot understand whether they were mistake or intentional, and what would be the repercussions (ie, performance?).

@ByronByron self-assigned thisDec 8, 2016
@Byron
Copy link
Member

@ankostis Sorry for the very late reply. I am no holiday now, and am working through everything that piled up.
About the matter: I don't understand why this would leak in the first place. Does that mean it's a python 3 only issue? Also I wonder how anything can leak in the first place unless there are dependency cycles.
The comment in the respective are of the code also contains an alternative. It appears that it was chosen just for performance reasons. Maybe alternatives do by now exist too.

@ankostis
Copy link
ContributorAuthor

Yes, it is a python-3 issue, because the destructors are not deterministic.
Untill allmemoryview for some mmemp have close, neither the memmap nor its file can be collected.

Now, in python3, you can slice memmaps as if they werememoryview, so they are not strictly needed, unless it is a performance issue. From some quick experiments I had done, I could not see any benefits, quite the contrary, but it's been a long time and I don't remember exactly.

@Byron
Copy link
Member

If using direct access would be a solution, then it probably should be used instead. Correctness over performance I would say.
The main driver behind using the 'buffer' interface was to prevent copying of the slices that are read, and thus maintain the awesome properties of a memory map even in python. I do remember me digging around in python source code to see if copies were made.
However, now is a different time, and slicing memory maps directly in python 3 sounds appealing to me.

Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment
Reviewers
No reviews
Assignees

@ByronByron

Labels
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

3 participants
@ankostis@coveralls@Byron

[8]ページ先頭

©2009-2025 Movatter.jp