Repository files navigation

Only C types can be stored: supported types are currently number, strings,the data structures themselves (seenesting: e.g. it ispossible to have a Hash containing a Hash or a Vec), and torch tensors andstorages. All data structures can store heterogeneous objects, and supporttorch serialization.

It is easy to extend the support toother C types.

Note thattds relies currently on FFI, and works both withluajit orLua 5.2, providedthe latter is installed withluaffi. The dependency on FFI willbe removed in the future.

If a lua tabletbl is provided, the Hash will be filled up withcorresponding elements. Tables inside thetbl will be also converted(recursively) totds.Vec (if they contain only number keys) ortds.Hash otherwise.

Store the given (key,value) pair in the hash table. Ifvalue isnil, remove thekey if it exists.

Returns thevalue at the givenkey, andnil if thekey does not exist in the hash table.

Returns the number of key-value pairs in the hash table. Note that this acts different than lua tables, the latterreturning the number of elements stored in numbered indices starting from 1.

Returns an iterator over the hash tabled. The iterator returns akey-value pair at each step, or nil if reaching the end. Typical usagewill be:

fork,vinpairs(d)do-- <do something>end

Note: as for Lua standard tables, the iterator behavior is undefined if anew key is inserted in the hash while iterating. Modifying existing keys ishowever allowed.

If provided,tbl must contain only number keys. Tables inside thetbl(or passed as arguments) will also be converted (recursively) totds.Vec (if they contain only number keys) ortds.Hash otherwise.

A vector can contain any element (as value) supported bytds, as well asthenil value.

Store the givenvalue at the givenindex (which must be a positivenumber). If the index is larger than the current size of the vector, thevector will be automatically resized.value may benil.

Returns thevalue at the givenindex ornil if it does not exist.

Returns the current size of the vector (note that it includesnil values, which are not treated as holes!).

Resize the current vector to the given size. If the size is larger than the current size, the vector will be filled withnil values.

Insertvalue in the vector, at positionindex, shifting up all elementsaboveindex. Ifindex is not provided, insert the element at the end ofthe vector.

Remove the element at positionindex, shifting down all elements aboveindex. Ifindex is not provided, remove the last element of the vector.

Sort the vector in-place, according to the givencompare function.

Compare can be either a lua function or a C function.

In the lua case,compare is a function which takes two vector elements,and returns true when the first is less than the second.

If the C case,compare must be a FFI typeint (*compare)(const tds_elem *, const tds_elem *).It must return an integer less than, equal to, or greater than zero if thefirst argument is considered to be respectively less than, equal to, orgreater than the second. See the include filetds_elem.h for more detailsabout thetds_elem structure.

Note that havingcompare as a lua function will lead to a (relatively)slow sort: elements of the vector will need to be moved in the lua userland(and thus handled by the GC) in order to be compared.

In the FFI case,compare might be a FFI callback, but will also lead to aslow sort, FFI callbacks being slow. Fastest speed are obtained whencompare is a true compiled C function loaded through FFI.

Asconcat(), but returns atorch.CharStorage() instead.

Returns an iterator over the vectord. The iterator returns a index-valuepair at each step, or nil if reaching the end. Typical usage will be:

fori,vinpairs(d)do-- <do something>end

Alias for ipairs(d).

tds=require'tds'require'torch'-- create a vector containing heterogeneous datad=tds.Vec(4,5,torch.rand(3),nil,"hello world")-- serialize in a bufferf=torch.MemoryFile("rw")f:writeObject(d)-- unserializef:seek(1)print(f:readObject())

tds.Vec[5]{    1 : 4    2 : 5    3 :  0.1665         0.8750         0.7525        [torch.DoubleTensor of size 3]    4 : nil    5 : hello world}

tds=require'tds'require'torch'-- create a vector containing heterogeneous datad=tds.Vec(4,5,torch.rand(3),tds.Hash(),"hello world")-- fill up the hash table:d[4].foo="bar"d[4][6]=torch.rand(3)d[4].stuff=tds.Vec("how","are","you","doing")print(d)

tds.Vec[5]{    1 : 4    2 : 5    3 :  0.1958         0.5663         0.2777        [torch.DoubleTensor of size 3]    4 : tds.Hash[3]{        foo   : bar        6     :  0.0105                 0.7496                 0.5241                [torch.DoubleTensor of size 3]        stuff : tds.Vec[4]{                    1 : how                    2 : are                    3 : you                    4 : doing                }        }    5 : hello world}

localelem=require'tds.elem'

tds typechecking is achieved using this function. You can override it foryour own purposes. If torch is detected,tds will setelem.type totorch.typename(), so in general (if you are using torch!) you should notworry about this part.

Add a new C type intotds:

• ttype must be the typename understood by the currentelem.type() function.
• free_p is a C FFI pointer to a destructor of the C object.
• setfunc(luaobj) takes alua object and returns a FFI C pointer on this object, as well as a FFI functionfree_p to free this object.
• getfunc(cpointer) takes aC FFI pointer and returns a lua object of the corresponding object.

One must be careful to handle properly reference counting and garbage collection insetfunc() andgetfunc():

• setfunc() will convert a lua object into a C pointer which will bestored into the data structure: the reference count on this object mustbe increased. When removed from the data structure,tds will call thegivenfree_p() function.
• getfunc() will convert a C pointer and push it into lua memory space:one must again increase properly the reference count on this object,and make sure lua will garbage collect it properly.

Here is a typical example showing how support fortds.Hash elements is supported:

elem.addctype('tds.Hash',C.tds_hash_free,function(lelem)C.tds_hash_retain(lelem)returnlelem,C.tds_hash_freeend,function(lelem_p)locallelem=ffi.cast('tds_hash&',lelem_p)C.tds_hash_retain(lelem)ffi.gc(lelem,C.tds_hash_free)returnlelemend   )