Repository files navigation

npm i tinypg

import{TinyPg}from'tinypg'// Options default to PG environment variablesconstdb=newTinyPg()asyncfunctionmain(){constresult=awaitdb.query(`SELECT 'Hello, World'`)console.log(result)}main()// Prints:// { row_count: 1,//  rows: [ { '?column?': 'Hello, World' } ],//  command: 'SELECT'}

TinyPg makes it possible to use objects as the source for parameters in a query. For example:

db.query(`SELECT *FROM customer   INNER JOIN address ON address.customer_id = customer.customer_idWHERE address.state = :state   AND customer.first_name = :first_name   AND customer.last_name = :last_name;`,{first_name:'Joe',last_name:'Andaverde',state:'Kansas',})

Now that we're past the mess of managing parameters the next step is organizing our SQL statements. It's kind of ugly to embed SQL in your JavaScript syntax. TinyPg allows for specifying a directory for which it will load all files with the.sql extension. The path to the file will normalized into a key for which you can look up and execute the SQL in the file as a prepared statement. For our projects we have hundreds of different SQL queries. It's worth noting that we don't subscribe to tools that generate SQL on your behalf. Many of our queries require use of Postgres features that no SQL generator can provide. Therefore, we don't bother trying to force these libraries to be smarter and stick to the language that's best suited for retrieving data from relational data stores: SQL.

Consider the following directory structure:

/app   /db      /address         create.sql      /customer         create.sql         fetch.sql         search.sql

If you provide/app/db as a root directory to TinyPg it will load and parse parameters from all sql files beneath that directory. These files are keyed using the path to the file e.g.address.create orcustomer.fetch. Here's an example usage:

db.sql('customer.search',{first_name:'Joe',last_name:'Andaverde',state:'Kansas',})

TinyPg checks for the existence of required parameters when each query is executed. Instead of placing db null in it will fail with an error message describing the missing parameter. I highly recommend using an object literal to specify parameter in order to use some of the static analysis tools like tslint or the VS Code plugin.

If you've ever looked at handling transactions with node-postgres you'll quickly realize that it's easy to get into deadlock. Tiny handles the re-use of the same connection for all queries performed within the same transaction provided you use the new database object provided by the call to.transaction. Here's how to create a customer and associate an address in the same transaction.

db.transaction(asynctransaction_db=>{// BEGINconstcreate_result=awaittransaction_db.sql('customer.create',{// INSERTfirst_name:'Joe',last_name:'Andaverde',})constcustomer=create_result.rows[0]awaittransaction_db.sql('address.create',{// INSERTcustomer_id:customer.customer_id,street:'123 W SomeStreet',city:'SomeCity',state:'SomeState',zip:12345,})returncustomer.customer_id})// COMMIT.then(asynccustomer_id=>{constfetch_result=awaitdb.sql('customer.fetch',{// SELECTcustomer_id:customer_id,})returnfetch_result.rows[0]})

Whenever your promise succeeds it'll automatically executeCOMMIT and in the event of failure it will executeROLLBACK. Callingtransaction on a Tiny instance that's already in a transaction will be a no-op. It is important that the lambda given to the transaction function return a promise. OtherwiseCOMMIT may be called at an unexpected time. Synchronous errors thrown in a transaction lambda will be caught and result in aROLLBACK.

Here's the sequence of SQL statements that would be executed:

BEGININSERT INTO customer ...INSERT INTO address ...COMMITSELECT*FROM customer ...

Notice the select AFTER the transaction has been committed. This is very important in order to return data that's actually persisted in the database.

Events are emitted for the beginning and end of a query. This has been helpful for us to diagnose slow running queries. Our practice is to create a single instance of TinyPg per process and attach handlers to thequery andresult events to log all database queries.

Sometimes you need to associate several database calls with some context e.g., a web request. TinyPg provides a way to create a brand new event emitter that can emit events separately from the global handlers. This functionality isn't thoroughly flushed out and may not fit all use cases but works great for us thus far. Here's an example:

constdb=newTinyPg(options)functionApiRequestHandler(request,reply){constisolated_db=db.isolatedEmitter()isolated_db.events.on('query',context=>{console.log(request.request_id,context.name)})returnnewUserService(isolated_db).list().then(users=>reply(users))}

In the above exampleisolated_db is the same instance of TinyPg except with an overridden events property anddispose method to remove all listeners. TheUserService can create other services and pass its reference toisolated_db to other services. In doing so, you can track all database queries executed as the result of every API request.

Hooks enable user defined functions to be called before (or after depending on the hook) TinyPg functions are executed. Most hooks allow user defined context to be passed throughout the hook lifecycle. Different hooks defined in the sameTinyHooks object maintain a shared context across each hook call. The following hooks are currently supported:

• preSql
  • Called at the start ofTinyPg.sql before the db call is performed.
  • (tiny_ctx: TinyCallContext, name: string, params: TinyPgParams) => HookResult<[string, TinyPgParams]>
  • Context for the givenHookSet is set to thectx field of the object returned
• preRawQuery
  • Called at the start ofTinyPg.query before the db call is performed.
  • (tiny_ctx: TinyCallContext, query: string, params: TinyPgParams) => HookResult<[string, TinyPgParams]>
  • Context for the givenHookSet is set to thectx field of the object returned
• onQuery
  • Called when thePg.PoolClient is obtained (start of db call). Thequery event is also emitted at this point.
  • (ctx: any, query_begin_context: QueryBeginContext) => any
  • Context for the givenHookSet is set to the return value
• onSubmit
  • Called when the query is [submitted]. Thesubmit event is emitted at this point. SeePg Query.
  • (ctx: any, query_submit_context: QuerySubmitContext) => any
  • Context for the givenHookSet is set to the return value
• onResult
  • Called when the query promise is resolved or rejected. Theresult event is emitted at this point.
  • (ctx: any, query_complete_context: QueryCompleteContext) => any
  • Context for the givenHookSet is set to the return value
• preTransaction
  • Called at the start ofTinyPg.transaction.
  • (transaction_id: string) => any
  • Transaction context for the givenHookSet is set to the return value
• onBegin
  • Called immediately after theBEGIN promise is resolved to begin the transaction.
  • (transaction_ctx: any, transaction_id: string) => any
  • Transaction context for the givenHookSet is set to the return value
• onCommit
  • Called immediately after theCOMMIT promise is resolved to commit the transaction.
  • (transaction_ctx: any, transaction_id: string) => any
  • Transaction context for the givenHookSet is set to the return value
• onRollback
  • Called immediately after theROLLBACK promise is resolved to abort the transaction.
  • (transaction_ctx: any, transaction_id: string, error: Error) => any
  • Transaction context for the givenHookSet is set to the return value

Note: aHookResult follows the form:

interface HookResult<T> {   args: T   ctx: any}

There is only one rule about using hooks:HOOKS MUST BE SYNCHRONOUS

Hooks can be created via thehooks field on theTinyPgOptions passed to the constructor OR by callingwithHooks(hooks: TinyHooks) on an instance ofTinyPg. An exampleTinyHooks object can be found below. Notice how well hooks play with tracing tools such asStackDriver trace.

{preSql:(tiny_context,file_name,params)=>{consttracer=TraceAgent.get()constspan=tracer.createChildSpan({name:`${file_name}_sql`,})_.forEach(buildSqlLabels(tiny_context,params),label=>{span.addLabel(label.label_key,label.label_value)})return{ctx:{sql_span:span,},args:[file_name,params],}},onResult:(ctx:{sql_span:TraceAgent.PluginTypes.Span},query_complete_context)=>{consttracer=TraceAgent.get()if(tracer.isRealSpan(ctx.sql_span)){_.forEach(buildResultLabels(query_complete_context),label=>{ctx.sql_span.addLabel(label.label_key,label.label_value)})ctx.sql_span.endSpan()}returnctx},preTransaction:transaction_id=>{consttracer=TraceAgent.get()constspan=tracer.createChildSpan({name:'tinypg_transaction',})span.addLabel('transaction_id',transaction_id)return{transaction_span:span,}},onCommit:(transaction_ctx:{transaction_span:TraceAgent.PluginTypes.Span},_transaction_id)=>{consttracer=TraceAgent.get()if(tracer.isRealSpan(transaction_ctx.transaction_span)){transaction_ctx.transaction_span.endSpan()}returntransaction_ctx},onRollback:(transaction_ctx:{transaction_span:TraceAgent.PluginTypes.Span},_transaction_id,error)=>{consttracer=TraceAgent.get()if(tracer.isRealSpan(transaction_ctx.transaction_span)){transaction_ctx.transaction_span.addLabel('error',error)transaction_ctx.transaction_span.endSpan()}returntransaction_ctx},}

If you're using TypeScript in your project (which I highly recommend) you can get an extra level of validation and editor integration by using the TinyPg VS Code plugin. This plugin can statically analyze (why I suggest using object literals) your code to ensure you've referenced sql files that exist and have provided all required parameters.

See Example Project on Github

TinyPg VS Code Plugin

TSLint Rules

• root_dir: string[] - a list of directories. All directories must be specified using the full path.
• connection_string: string - The database connection string in URL format. e.g. postgres://user:password@host:port/database?options=query
• error_transformer: Function - Allows transforming all errors from TinyPg to your domain.
• capture_stack_trace: boolean - Opt-in to capturing stack trace to give a better indication of what function in your domain caused an error.
• tls_options - TLS options passed to the underlying socket.
• pool_options: (Seenode-pg-pool - only difference is casing)
• hooks: TinyHooks - TinyHooks object seeHooks for details about hooks.

consterror_transformer=(error)=>{constparseErrorByCode=()=>{constpg_error=error.queryContext.errorconstcode=pg_error.codeswitch(code){case'22P02':// Invalid text representationreturnnewE.InvalidArgumentError(error.message)case'23502':// Constraint errorreturnnewE.InvalidArgumentError(`Invalid Argument:${pg_error.column}`)case'23503':// Foreign key violationreturnnewE.ForeignKeyViolationError('Foreign Key Violation',pg_error)case'23505':// unique violationcase'23P01':// exclusion constraint violationreturnnewE.ConflictError('Data Conflict Error',pg_error)case'23514':// Check ViolationreturnnewE.InvalidArgumentError(`Invalid Argument:${error.message}`)default:returnnewE.UnknownPostgresError(error.message)}}letnew_errorif(error.queryContext&&error.queryContext.error&&error.queryContext.error.code){new_error=parseErrorByCode()}else{new_error=newE.UnknownPostgresError(error.message)}new_error.stack=error.stackreturnnew_error}

SeePg Error Codes Documentation

• raw_sql: string - The SQL query to execute.
• params: Object (optional) - parameters for the query.

• name: string - The key of the sql file. This is the path to the file substituting. for path delimiter. e.g.users.create

Select a SQL file that has formattable parts. Seenode-pg-format for format strings. This is useful when needing to build dynamic queries.

• name: string - The key of the sql file. This is the path to the file substituting. for path delimiter. e.g.users.create

database/users/retrieve.sql

SELECT*FROM usersWHERE last_name= :last_nameORDER BY-- Custom ordering  %s  user_idDESC;

Usage in code

db.formattable('users.retrieve')  .format('last_name ASC,')  .query({ last_name: 'Andaverde' })

Resulting Query

SELECT*FROM usersWHERE last_name= :last_nameORDER BY-- Custom ordering  last_nameASC,  user_idDESC;

Starts a database transaction and ensures all queries executed against the provided TinyPg instance use the same client.

• tx_fn: (db: TinyPg) => Promise - Provides db to perform transacted queries.

Returns a new instance of TinyPg with the givenTinyHooks added to the end of the instance's hook collection. SeeHooks for more details about each hook.

interface TinyHooks {   preSql?: (tiny_ctx: TinyCallContext, name: string, params: TinyPgParams) => HookResult<[string, TinyPgParams]>   preRawQuery?: (tiny_ctx: TinyCallContext, query: string, params: TinyPgParams) => HookResult<[string, TinyPgParams]>   onQuery?: (ctx: any, query_begin_context: QueryBeginContext) => any   onSubmit?: (ctx: any, query_submit_context: QuerySubmitContext) => any   onResult?: (ctx: any, query_complete_context: QueryCompleteContext) => any   preTransaction?: (transaction_id: string) => any   onBegin?: (transaction_ctx: any, transaction_id: string) => any   onCommit?: (transaction_ctx: any, transaction_id: string) => any   onRollback?: (transaction_ctx: any, transaction_id: string, error: Error) => any}

Provides an isolated event emitter so thatquery,submit, andresult events (in that order) can be monitored for all queries related to the new TinyPg instance.

Shuts down the postgres client pool.

You should have a local development Postgres server running. This server must allow connections from thepostgres user without password. If this isn't the behavior your want change the connection string insrc/test/helper.ts.

npm installnpmtest