- Notifications
You must be signed in to change notification settings - Fork124
Open
Description
Hi,
I'm running into a weird issue where callingnioBuffer on aByteBuf received fromlmdb-java results in aNIO ByteBuffer with the desired size but a value of all zeros.
I'm onNetty 4.1.90.Final andlmdbjava 0.8.3 and JDK 19 . The repro is as follows:
package foo;import io.netty.buffer.ByteBufUtil;import io.netty.buffer.PooledByteBufAllocator;import io.netty.buffer.Unpooled;import org.junit.jupiter.api.Test;import org.junit.jupiter.api.io.TempDir;import org.lmdbjava.ByteBufProxy;import org.lmdbjava.DbiFlags;import org.lmdbjava.Env;import java.io.File;import static java.nio.charset.StandardCharsets.UTF_8;import static org.junit.jupiter.api.Assertions.assertNotNull;public class ByteBufMinimalReproTest { @TempDir File tmp; @Test void minimalRepro() { var env = Env.create(ByteBufProxy.PROXY_NETTY) .setMapSize(10_485_760) .setMaxDbs(1) .open(tmp); var db = env.openDbi("testdb", DbiFlags.MDB_CREATE); var alloc = PooledByteBufAllocator.DEFAULT; var key = alloc.directBuffer(env.getMaxKeySize()); var value = alloc.directBuffer(700); key.writeCharSequence("greeting", UTF_8); value.writeCharSequence("Hello World", UTF_8); System.out.println("Sanity -- Key"); System.out.println(ByteBufUtil.prettyHexDump(key)); System.out.println("Sanity -- Key NIO"); System.out.println(ByteBufUtil.prettyHexDump(Unpooled.wrappedBuffer(key.nioBuffer()))); System.out.println("Sanity -- Value"); System.out.println(ByteBufUtil.prettyHexDump(value)); System.out.println("Sanity -- Value NIO"); System.out.println(ByteBufUtil.prettyHexDump(Unpooled.wrappedBuffer(value.nioBuffer()))); db.put(key, value); // need a transaction to read values. try (var txn = env.txnRead()) { var found = db.get(txn, key); assertNotNull(found); System.out.println("Found Value"); System.out.println(ByteBufUtil.prettyHexDump(found)); System.out.println("Found Value - NIO (Bad: all zeros)"); System.out.println(ByteBufUtil.prettyHexDump(Unpooled.wrappedBuffer( found.nioBuffer()))); var fetchedVal = txn.val(); System.out.println("val via txn"); System.out.println(ByteBufUtil.prettyHexDump(fetchedVal)); System.out.println("val via txn - NIO (Bad: all zeros)"); System.out.println(ByteBufUtil.prettyHexDump(Unpooled.wrappedBuffer( txn.val().nioBuffer()))); } env.close(); }}Which prints:
Sanity -- Key +-------------------------------------------------+ | 0 1 2 3 4 5 6 7 8 9 a b c d e f |+--------+-------------------------------------------------+----------------+|00000000| 67 72 65 65 74 69 6e 67 |greeting |+--------+-------------------------------------------------+----------------+Sanity -- Key NIO +-------------------------------------------------+ | 0 1 2 3 4 5 6 7 8 9 a b c d e f |+--------+-------------------------------------------------+----------------+|00000000| 67 72 65 65 74 69 6e 67 |greeting |+--------+-------------------------------------------------+----------------+Sanity -- Value +-------------------------------------------------+ | 0 1 2 3 4 5 6 7 8 9 a b c d e f |+--------+-------------------------------------------------+----------------+|00000000| 48 65 6c 6c 6f 20 57 6f 72 6c 64 |Hello World |+--------+-------------------------------------------------+----------------+Sanity -- Value NIO +-------------------------------------------------+ | 0 1 2 3 4 5 6 7 8 9 a b c d e f |+--------+-------------------------------------------------+----------------+|00000000| 48 65 6c 6c 6f 20 57 6f 72 6c 64 |Hello World |+--------+-------------------------------------------------+----------------+Found Value +-------------------------------------------------+ | 0 1 2 3 4 5 6 7 8 9 a b c d e f |+--------+-------------------------------------------------+----------------+|00000000| 48 65 6c 6c 6f 20 57 6f 72 6c 64 |Hello World |+--------+-------------------------------------------------+----------------+Found Value - NIO (Bad: all zeros) +-------------------------------------------------+ | 0 1 2 3 4 5 6 7 8 9 a b c d e f |+--------+-------------------------------------------------+----------------+|00000000| 00 00 00 00 00 00 00 00 00 00 00 |........... |+--------+-------------------------------------------------+----------------+val via txn +-------------------------------------------------+ | 0 1 2 3 4 5 6 7 8 9 a b c d e f |+--------+-------------------------------------------------+----------------+|00000000| 48 65 6c 6c 6f 20 57 6f 72 6c 64 |Hello World |+--------+-------------------------------------------------+----------------+val via txn - NIO (Bad: all zeros) +-------------------------------------------------+ | 0 1 2 3 4 5 6 7 8 9 a b c d e f |+--------+-------------------------------------------------+----------------+|00000000| 00 00 00 00 00 00 00 00 00 00 00 |........... |+--------+-------------------------------------------------+----------------+My repro is using the prettyHexDump from ByteBuf, which is why i need to re-wrap nioBuffer. I'm pretty sure this isn't causing any problems because when I write the bytebuf to file, I see all zeros.
The reason I'm doing nioBuffer on this netty buffer is because I need to pass it to Zstd java for decompression.
I'm new to both lmdb java and netty, and it's possible i'm doing something wrong, but I'd appreciate your help with this.