Repository files navigation

Java has a lot of different ways to represent a stream of bytes. Depending on the author and age of a library, it might usebyte[],InputStream,ByteBuffer, orReadableByteChannel. If the bytes represent strings, there's alsoString,Reader, andCharSequence to worry about. Remembering how to convert between all of them is a thankless task, made that much worse by libraries which define their own custom representations, or composing them with Clojure's lazy sequences and stream representations.

This library is a Rosetta stone for all the byte representations Java has to offer, and gives you the freedom to forget all the APIs you never wanted to know in the first place. Complete documentation can be foundhere.

Leiningen:

[org.clj-commons/byte-streams"0.3.4"]

deps.edn:

org.clj-commons/byte-streams {:mvn/version"0.3.4"}

To convert one byte representation to another, useconvert:

clj-commons.byte-streams> (convert"abcd" java.nio.ByteBuffer)#<HeapByteBuffer java.nio.HeapByteBuffer[pos=0 lim=4 cap=4]>clj-commons.byte-streams> (convert *1 String)"abcd"

(convert data to-type options?) converts, if possible, the data from its current type to the destination type. This destination type can either be a Java class or a Clojure protocol. However, since there's no direct route from a string to a byte-buffer, under the coversconvert is doing whatever it takes to get the desired type:

clj-commons.byte-streams> (conversion-path String java.nio.ByteBuffer)([java.lang.String [B] [[B java.nio.ByteBuffer])

While we can't turn a string into aByteBuffer, we can turn a string into abyte[], andbyte[] into aByteBuffer. When invoked,convert will choose the minimal path along the graph of available conversions. Common conversions are exposed viato-byte-buffer,to-byte-buffers,to-byte-array,to-input-stream,to-readable-channel,to-char-sequence,to-string, andto-line-seq.

Every type can exist either by itself, or as a sequence. For instance, we can create anInputStream representing an infinite number of repeated strings:

clj-commons.byte-streams> (to-input-stream (repeat"hello"))#<InputStream byte_streams.InputStream@3962a02c>

And then we can turn that into a lazy sequence ofByteBuffers:

clj-commons.byte-streams>   (take2 (convert *1                   (seq-of java.nio.ByteBuffer)                   {:chunk-size128}))(#<HeapByteBuffer java.nio.HeapByteBuffer[pos=0 lim=128 cap=128]> #<HeapByteBuffer java.nio.HeapByteBuffer[pos=0 lim=128 cap=128]>)

Notice that we describe a sequence of a type as(seq-of type), and that we've passed a map toconvert describing the size of theByteBuffers we want to create. Available options include:

• :chunk-size - the size in bytes of each chunk when converting a stream into a lazy seq of discrete chunks, defaults to4096
• :direct? - whether anyByteBuffers which are created should bedirect, defaults tofalse
• :encoding - the character set for any strings we're encoding or decoding, defaults to"UTF-8"

To create aManifold stream, use(stream-of type). To convert a core.async channel, convert it usingmanifold.stream/->source.

While there are conversions defined for all common byte types, this can be extended to other libraries viadef-conversion:

;; a conversion from byte-buffers to my-byte-representation(def-conversion [ByteBuffer MyByteRepresentation]  [buf options]  (buffer->my-representation buf options));; everything that can be converted to a ByteBuffer is transitively fair game now(convert"abc" MyByteRepresentation)

This mechanism can even be used for types unrelated to byte streams, if you're feeling adventurous.

Simple conversions are useful, but sometimes we'll need to do more than just keep the bytes in memory. When you need to write bytes to a file, network socket, or other endpoints, you can usetransfer.

clj-commons.byte-streams> (deff (File."/tmp/salutations"))#'clj-commons.byte-streams/fclj-commons.byte-streams> (transfer"hello" f {:append?false})nilclj-commons.byte-streams> (to-string f)"hello"

(transfer source sink options?) allows you pipe anything that can produce bytes into anything that can receive bytes, using the most efficient mechanism available. Custom transfer mechanisms can also be defined:

(def-transfer [InputStream MyByteSink]  [stream sink options]  (send-stream-to-my-sink stream sink))

print-bytes will print both hexadecimal and ascii representations of a collection of bytes:

clj-commons.byte-streams> (print-bytes (-> #'print-bytes meta:doc))5072696E7473206F7574207468652062      Prints out the b7974657320696E20626F746820686578      ytes in both hex20616E64204153434949207265707265       and ASCII repre73656E746174696F6E732C2031362062      sentations,16 b7974657320706572206C696E652E            ytes per line.

(compare-bytes a b) will return a value which is positive ifa is lexicographically first, zero if they're equal, and negative otherwise:

clj-commons.byte-streams> (compare-bytes"abc""abd")-1

bytes= will return true if two byte streams are equal, and false otherwise.

conversion-path returns all the intermediate steps in transforming one type to another, if one exists:

;; each element is a conversion pair of to/fromclj-commons.byte-streams> (conversion-path java.io.File String)([java.io.File java.nio.channels.ReadableByteChannel] [#'byte-streams/ByteSource java.lang.CharSequence] [java.lang.CharSequence java.lang.String]);; but if a conversion is impossible...clj-commons.byte-streams> (conversion-path java.io.OutputStream java.io.InputStream)nil

possible-conversions returns a list of possible conversion targets for a type.

clj-commons.byte-streams> (possible-conversions String)(java.lang.String java.io.InputStream java.nio.DirectByteBuffer java.nio.ByteBuffer (seq-of java.nio.ByteBuffer) java.io.Reader java.nio.channels.ReadableByteChannel [B java.lang.CharSequence)

byte-streams/optimized-transfer? returns true if there is an optimized transfer method for two types:

clj-commons.byte-streams> (optimized-transfer? String java.io.File)true

There exists an older, top-level namespace called justbyte-streams that has been deprecated. Top-level namespaces can cause problems, particularly with Graal. Please switch toclj-commons.byte-streams in all your requires.

Copyright © 2014 Zachary Tellman

Distributed under theMIT License