Repository files navigation

(m)Ruby + PostgreSQL = <3

Holycorn makes it easy to implement a Foreign Data Wrapper using Ruby.

It is based on top of mruby, that provides sandboxing capabilities the regularRuby VM "MRI/CRuby" does not implement.

Holycorn is the combination of the mruby VM, some supporting gems (some basiclibraries and some more advanced ones), and custom code that implement the actualforeign data wrappers.

All the following wrappers are currently linked against Holycorn:

• Redis, using themruby-redis gem

• PostgreSQL 9.4+

Simply run

rake

to vendor and buildmruby.

Now thatmruby is built, buildingholycorn requires to run

make

and installing it only requires to run

make install

Now connect to PostgreSQL and install the extension:

λ psqlpsql (9.5.14)Type "help" for help.DROP EXTENSION holycorn CASCADE;CREATE EXTENSION holycorn;

A set of builtin FDW are distributed with Holycorn for an easy setup. All oneneeds to provide are the options that will allow the FDW to be configured:

λ psqlpsql (9.5.14)Type "help" for help.CREATE SERVER holycorn_server FOREIGN DATA WRAPPER holycorn;CREATE FOREIGN TABLE redis_table (key text, value text)  SERVER holycorn_server  OPTIONS ( wrapper_class 'HolycornRedis'          , host '127.0.0.1'          , port '6379'          , db '0');

Alternatively, Holycorn supportsIMPORT FOREIGN SCHEMA and the same can beaccomplished by using that statement instead:

λ psqlpsql (9.5.14)CREATE SERVER holycorn_server FOREIGN DATA WRAPPER holycorn;IMPORT FOREIGN SCHEMA holycorn_schemaFROM SERVER holycorn_serverINTO holycorn_tablesOPTIONS ( wrapper_class 'HolycornRedis'        , host '127.0.0.1'        , port '6379'        , db '0'        , prefix 'holycorn_'        );

Please note that custom data wrappers have to use theholycorn_schema schemaas it would otherwise result in an error at runtime.

Note:IMPORT FOREIGN SCHEMA requires us to use a custom target schema andHolycorn encourages users to use aprefix to help avoiding name collisions.

AsHolycorn doesn't supportINSERTs yet, let's create some manually:

λredis-cli127.0.0.1:6379>select0OK127.0.0.1:6379>keys*(empty list or set)127.0.0.1:6379>set foo 1OK127.0.0.1:6379>set bar 2OK127.0.0.1:6379>set baz 3OK127.0.0.1:6379>keys*1) "bar"2) "foo"3) "baz"

Now that the table has been created and we have some data in Redis, we canselect data from the foreign table.

SELECT*from redis_table; key | value-----+------- bar |2 foo |1 baz |3(3 rows)

Alternatively, custom scripts can be used as the source for a Foreign Data Wrapper:

DROP EXTENSION holycorn CASCADE;CREATE EXTENSION holycorn;CREATE SERVER holycorn_server FOREIGN DATA WRAPPER holycorn;CREATE FOREIGN TABLE holytable (some_datetimestamptz) \  SERVER holycorn_server  OPTIONS (wrapper_path'/tmp/source.rb');

(IMPORT FOREIGN SCHEMA is also supported here.)

And the source file of the wrapper:

# /tmp/source.rbclassProducerdefinitialize(env={})# env contains informations provided by Holycornenddefeach@enum ||=Enumerator.newdo |y|10.timesdo |t|y.yield[Time.now]endend@enum.nextenddefimport_schema(args={})# Keys are:#   * local_schema: the target schema#   * server_name: name of the foreign data server in-use#   * wrapper_class: name of the current class#   * any other OPTIONS passed to IMPORT FOREIGN SCHEMAendselfend

For more details about#import_schema, please take a look at the exampleslocated in in thebuiltin_wrappers directory.

Now you can select data out of the wrapper:

λ psqlpsql (9.5.14)Type "help" for help.franck=# SELECT * FROM holytable;      some_date--------------------- 2015-06-21 22:39:24 2015-06-21 22:39:24 2015-06-21 22:39:24 2015-06-21 22:39:24 2015-06-21 22:39:24 2015-06-21 22:39:24 2015-06-21 22:39:24 2015-06-21 22:39:24 2015-06-21 22:39:24 2015-06-21 22:39:24(10 rows)

Pretty neat.

Any type of Ruby object can act as a FDW. The only requirements are that it canreceive.new (with arity = 1) and return an object that can receiveeach (arity = 0).

It doesn'thave to be aClass, and there's currently no will to provide asuperclass to be inherited from.

In future versions, there will be many more callbacks to interact with PG's FDWinfrastructure throughHolycorn.

Also, the script can only be a single word - likeMyClass - as long asMyClass has been defined and exists within your compilation ofmruby.

A hash is passed byHolycorn to the Ruby script. Its current keys are:

• PG_VERSION
• PG_VERSION_NUM
• PACKAGE_STRING
• PACKAGE_VERSION
• MRUBY_RUBY_VERSION
• WRAPPER_PATH

• MRB_TT_FREE =>null
• MRB_TT_FALSE =>Boolean
• MRB_TT_TRUE =>Boolean
• MRB_TT_FIXNUM =>Int64
• MRB_TT_SYMBOL =>Text
• MRB_TT_UNDEF => Unsupported
• MRB_TT_FLOAT =>Float8
• MRB_TT_CPTR => Unsupported
• MRB_TT_OBJECT =>Text (to_s is called)
• MRB_TT_CLASS =>Text (class.to_s is called)
• MRB_TT_MODULE =>Text (to_s is called)
• MRB_TT_ICLASS => Unsupported
• MRB_TT_SCLASS => Unsupported
• MRB_TT_PROC =>Text (inspect is called)
• MRB_TT_ARRAY =>Text (inspect is called)
• MRB_TT_HASH =>Text (inspect is called)
• MRB_TT_STRING =>Text
• MRB_TT_RANGE =>Text (inspect is called)
• MRB_TT_EXCEPTION => Unsupported
• MRB_TT_FILE => Unsupported
• MRB_TT_ENV => Unsupported
• MRB_TT_DATA => See "Arbitraty Ruby objects" section
• MRB_TT_FIBER =>Text (inspect is called)
• MRB_TT_MAXDEFINE => Unsupported

• Time (Ruby) =>timestamptz

None (yet).

Eitherwrapper_class orwrapper_path can be used to defined whether anexternal script or a built-in wrapper will manage the foreign table.

• wrapper_class: Name of the built-in wrapper class
• wrapper_path: Path of a custom script

In both case, any other option will be pushed down to the wrapper class via theconstructor.

• Array type
• JSON type
• Range type
• Support PG 9.5'sIMPORT FOREIGN SCHEMA for easy setup

• Fork the project.
• Make your feature addition or bug fix.
• Add tests for it. This is important so I don't break it in a future version unintentionally.
• Commit, do not mess with version or history. (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
• Send me a pull request. Bonus points for topic branches.

Copyright (c) 2015-2018 Franck Verrot

Holycorn is an Open Source project licensed under the terms of the LGPLv3license. Please seehttp://www.gnu.org/licenses/lgpl-3.0.html for licensetext.

Franck Verrot, @franckverrot