Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

getDbiNames() method requires a caller-side write lock to avoid crashes #195

Closed
@dacr

Description

@dacr

It is not intuitive that in order to get all defined databases in an environment we must use awrite lock to prevent any other platform thread or virtual thread or effect... to do the same operation at the same time in order to avoid random crashes under heavy concurrent situation which is my case.

I'm using lmdbjava 0.8.2 (from scala 3 with ZIO 2 library), with an environment configured with the following flags :

                               Env                                 .create()                                 .setMapSize(100_000_000_000L)                                 .setMaxDbs(10_000)                                 .setMaxReaders(100)                                 .open(                                   databasesPath,                                   EnvFlags.MDB_NOTLS,                                   EnvFlags.MDB_NOLOCK, // Locks managed using ZIO ReentrantLock                                   EnvFlags.MDB_NOSYNC                                 )

So in my databases list implementation, I replacedreentrantLock.withReadLock byreentrantLock.withWriteLock:

  def databases(): Task[List[String]] = {    reentrantLock.withWriteLock(      ZIO.logSpan("databases") {        for {          databases <- Task                         .attempt {                           env                             .getDbiNames()                             .asScala                             .map(bytes => new String(bytes))                             .toList                         }                         .tap(l => ZIO.log(s"${l.size} databases found : ${l.mkString(",")}"))        } yield databases      }    )  }

which allows to only have one simultaneous call to getDbiNames and prevent all the crashes I had until now. After taking a look to getDbiNames() implementation I've understood why such a user side write lock must be implemented.

  public List<byte[]> getDbiNames() {    final List<byte[]> result = new ArrayList<>();    final Dbi<T> names = openDbi((byte[]) null);    try (Txn<T> txn = txnRead();    ....

this is related to the use ofopenDbi((byte[])null) which requires only one simultaneous access. So what can we do about this ? I spent something like more than ~20 hours to identify the issue as I think it is quite easy to fall into this trap. But thanks to it a performancement enhancement has been made to the ZIO library (zio/zio#6674).

  • I'm not sure that it is easily fixable, is there an other way to get all defined databases within an environment ?
  • I can prepare a Pull Request with various information in order to help to deal with concurrency (in the wiki :https://github.com/lmdbjava/lmdbjava/wiki/Concurrency) ?

thanks in advance for your response,
David.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions


      [8]ページ先頭

      ©2009-2025 Movatter.jp