Movatterモバイル変換


[0]ホーム

URL:


Skip to content
DEV Community
Log in Create account

DEV Community

Dynamically generating SQL queries using Node.js

Gajus Kuizinas on August 29, 2019

Ever since I have released Slonik (PostgreSQL client for Node.js) and written a controversial Stop using Knex.js article (tl;dr; query builders are...
pic
Create template

Templates let you quickly answer FAQs or store snippets for re-use.

Dismiss
CollapseExpand
 
ivan_paqmind profile image
Ivan Kleshnin
Experienced software engineer, mentor and entrepreneur. 12+ years in webdev. Generalist. Proponent of minimalism and 改善 (kaizen).
  • Location
    Poland
  • Education
    Computer Systems and Networks
  • Joined
• Edited on• Edited

Is it ok to usesql.raw to create a fully dynamicORDER BY query?

letfield="createdAt"letdir="DESC"// generated like field.startsWith("+") ? "ASC" : "DESC"letorderToken=sql`  ORDER BY${sql.identifier(field)}${sql.raw(dir)}'`letq=sql`  SELECT * FROM "post"${orderToken}  LIMIT 3`
Enter fullscreen modeExit fullscreen mode
CollapseExpand
 
gajus profile image
Gajus Kuizinas
Open-source engineer interested in JavaScript, PostgreSQL and DevOps. Follow me on Twitter for outbursts about startups & engineering.
• Edited on• Edited

You don't need to:sql tags can be nested.

letfield='createdAt';letdir=field.startsWith('+')?sql`ASC`:sql`DESC`;letorderToken=sql`  ORDER BY${sql.identifier([field])}${dir}'`;letq=sql`  SELECT * FROM "post"${orderToken}  LIMIT 3`;
CollapseExpand
 
ivan_paqmind profile image
Ivan Kleshnin
Experienced software engineer, mentor and entrepreneur. 12+ years in webdev. Generalist. Proponent of minimalism and 改善 (kaizen).
  • Location
    Poland
  • Education
    Computer Systems and Networks
  • Joined
• Edited on• Edited

Thanks. What about optionalORDER BY? ForWHERE you started withTRUE so the absence of conditions results inWHERE (true) which is syntactically correct and has no performance implications. I can't find anything than can be used to order by default:

ORDER BY ??? default ???
Enter fullscreen modeExit fullscreen mode

Another attempt with?: breaks the placeholder ordering in query:

SELECT ${makeProjection(pick)} FROM ${sql.identifier([table])}WHERE ${makeWhere(filter)}${sort ? sql`ORDER BY ${makeOrder(sort)}` : ``} -- doesn't workLIMIT ${makeLimit(limit)}OFFSET ${makeOffset(offset)}
Enter fullscreen modeExit fullscreen mode
syntax error at or near "$1"
Enter fullscreen modeExit fullscreen mode
Thread Thread
 
gajus profile image
Gajus Kuizinas
Open-source engineer interested in JavaScript, PostgreSQL and DevOps. Follow me on Twitter for outbursts about startups & engineering.
${sort ? sql`ORDER BY ${makeOrder(sort)}` : sql``}
Enter fullscreen modeExit fullscreen mode
Thread Thread
 
ivan_paqmind profile image
Ivan Kleshnin
Experienced software engineer, mentor and entrepreneur. 12+ years in webdev. Generalist. Proponent of minimalism and 改善 (kaizen).
  • Location
    Poland
  • Education
    Computer Systems and Networks
  • Joined

Causes

InvalidInputError: Unexpected SQL input. Query cannot be empty.
Thread Thread
 
gajus profile image
Gajus Kuizinas
Open-source engineer interested in JavaScript, PostgreSQL and DevOps. Follow me on Twitter for outbursts about startups & engineering.
sql`--`
Thread Thread
 
gajus profile image
Gajus Kuizinas
Open-source engineer interested in JavaScript, PostgreSQL and DevOps. Follow me on Twitter for outbursts about startups & engineering.

I cannot recall the original reason for this constraint. Will likely remove this constraint. Trackgithub.com/gajus/slonik/issues/93.

CollapseExpand
 
ivan_paqmind profile image
Ivan Kleshnin
Experienced software engineer, mentor and entrepreneur. 12+ years in webdev. Generalist. Proponent of minimalism and 改善 (kaizen).
  • Location
    Poland
  • Education
    Computer Systems and Networks
  • Joined
• Edited on• Edited

Another question. WhyvalueList and other similar functions support only primitive values?

Is TypeScript a reason? The thing I like aboutpg driver is that, unlike MySQL, I canSELECT orINSERT a date, a boolean, an object, etc. value an it will do "the right thing" without an extra hassle.

Maybe it's not entirely safe or somehow "not a good practice" – I dunno.
For now, the API feels limiting for no obvious reason.

CollapseExpand
 
gajus profile image
Gajus Kuizinas
Open-source engineer interested in JavaScript, PostgreSQL and DevOps. Follow me on Twitter for outbursts about startups & engineering.

Value token can be any primitive that is shared between pg and JavaScript (string, integer, float, boolean, null).

If you have a requirement for automatic coalescing of other object types (such as Date), I suggest raising an issue. I cannot think of other types that would be relatively safe to cast, though.

CollapseExpand
 
ivan_paqmind profile image
Ivan Kleshnin
Experienced software engineer, mentor and entrepreneur. 12+ years in webdev. Generalist. Proponent of minimalism and 改善 (kaizen).
  • Location
    Poland
  • Education
    Computer Systems and Networks
  • Joined

JS Object withJSON.stringify?

Thread Thread
 
gajus profile image
Gajus Kuizinas
Open-source engineer interested in JavaScript, PostgreSQL and DevOps. Follow me on Twitter for outbursts about startups & engineering.

Too many edges cases where an object could be passed accidentally inserting potentially sensitive data to the database.

There issql.json() for that, though.

CollapseExpand
 
cvh23 profile image
cvh23
  • Joined

Slonik's approach is very interesting. Although it's maybe not directly related to Slonik, but because we don't use an ORM:What would be the best way for transforming a SELECT query result of rows into an array of nested objects? For instance if we are joining multiple tables and have a master-detail-relationship. The web client usually expects some kind of nested JSON structure.

CollapseExpand
 
gajus profile image
Gajus Kuizinas
Open-source engineer interested in JavaScript, PostgreSQL and DevOps. Follow me on Twitter for outbursts about startups & engineering.

You can return JSON from PostgreSQL query.

CollapseExpand
 
cvh23 profile image
cvh23
  • Joined

So your recommendation is to do everything in PostgreSQL with its JSON functions? So we already have the ready-to-use object or array of objects as query result?

Thread Thread
 
gajus profile image
Gajus Kuizinas
Open-source engineer interested in JavaScript, PostgreSQL and DevOps. Follow me on Twitter for outbursts about startups & engineering.

Not at all. I think it is entirely unnecessary. Keep the queries simple. If you need to nest objects, either use frameworks that automate nesting (e.g. GraphQL), or write simple iteration routines to amend the data structures.

Thread Thread
 
cvh23 profile image
cvh23
  • Joined

Sounds good, thank you! Do you know of any application which uses Slonik + GraphQL and is open source, so that I can study this approach?

CollapseExpand
 
5422m4n profile image
Sven Kanoldt
polyglot software engineer | #rustlang enthusiast | co-organizer at @RustMunich | maintainer of #cargogenerate | creator of @t_rec_rs and stegano-rs
  • Location
    Munich
  • Education
    MSc in Computer Science
  • Joined

Seems thatsql.valueList does not exist anymore. Also the link above providedContinue reading sql.array vs sql.valueList. does not lead to the right section of the README in the repo.

Was this feature removed lately?

CollapseExpand
 
paulovieira profile image
Paulo Vieira
  • Location
    Lisbon, Portugal
  • Joined

Hello Gajus,

I've been experimenting with Slonik for the first time today. Seems interesting, I'm very much aligned with the philosophy of "just write plain old sql". This article was useful, especially the final example. Thanks.

CollapseExpand
 
ivan_paqmind profile image
Ivan Kleshnin
Experienced software engineer, mentor and entrepreneur. 12+ years in webdev. Generalist. Proponent of minimalism and 改善 (kaizen).
  • Location
    Poland
  • Education
    Computer Systems and Networks
  • Joined
• Edited on• Edited

What's the difference betweentuple andvalueList? The only one I see is thattuple adds( and) parens around its values... The implementation and types are almost identical tovalueList, to the point I wonder why not to express one in terms of another or just ditchtuple as unnecessary. Most probably I miss something – that's just my first impression.

CollapseExpand
 
gajus profile image
Gajus Kuizinas
Open-source engineer interested in JavaScript, PostgreSQL and DevOps. Follow me on Twitter for outbursts about startups & engineering.

Eventuallysql.valueList,sql.rawList,sql.tupleList,sql.identifierList andsql.booleanExpression will be removed in favour of a single methodsql.expressionList.

github.com/gajus/slonik/issues/91

The original intention for separation was to force semantic resemblance and type strictness specific to the code fragment being generated. However, since then types have evolved for all of these helpers to allow a lot broader spectrum of allowed values.

CollapseExpand
 
baerrach profile image
Barrie Treloar
  • Joined

sql.booleanExpression was removed in favour ofsql.join but the article has not been updated.

Seefeat: remove multiple methods in favor of sql.join

The doc link is nowgithub.com/gajus/slonik#slonik-que...

CollapseExpand
 
sajedulkarim profile image
Sajedul karim
Software Engineer
  • Joined

I am fetching issue where I have to generate dynamic ORDER BY clause for multiple columns.
Here is my sample query:

SELECT *FROM userWHERE gender = 'male'  ORDER BY created_at ASC, updated_at DESCLIMIT 10 OFFSET 0;
Enter fullscreen modeExit fullscreen mode

My Try:

const orderBy: string = 'ORDER BY created_at ASC, updated_at DESC';SELECT *FROM userWHERE gender = 'male'  ${orderBy}LIMIT 10 OFFSET 0;
Enter fullscreen modeExit fullscreen mode

Here, if I pass ORDER BY clause as ${orderBy}, then it take it as a value. It shows error.

The generated sql is like below:

{  sql: "\n  SELECT *\nFROM user\nWHERE gender = 'male'\n  $1\nLIMIT 10 OFFSET 0;\n",  type: 'SLONIK_TOKEN_SQL',  values: [ 'ORDER BY created_at ASC, updated_at DESC' ]}
Enter fullscreen modeExit fullscreen mode

Please help me.

CollapseExpand
 
craigmichaelmartin profile image
craig martin
  • Joined

Hey Garjus - this is the first time I'm coming across your articles/project. Looks really interesting! I look foward to digging in :) I would love to hear your thoughts ongithub.com/craigmichaelmartin/pure... - apure ORM (no query builder dimension) which allows you to write regular native SQL and receive back properly nested/structured nestedpure (not db-aware stateful) business objects.

CollapseExpand
 
ackvf profile image
Vítězslav Ackermann Ferko
React.js developer since 2015
  • Location
    Czech Republic
  • Joined

Where have all the sql builder functions go and how to replace them?
comparisonPredicate, valueList, etc.

I am trying to construct multiple WHERE statements dynamically, but I get all sorts of errors. The thing is, some of them might be undefined and thus ommited, but then sql builder yells that it's not a valid value

CollapseExpand
 
shijiezhou profile image
Shijie Zhou
Living life openly. This year I set the following goals. Below you can find the progress on each.
  • Location
    Salt Lake City
  • Education
    University of California, San Diego | B.A.
  • Work
    Software Engineer at MMSG
  • Joined

I think you should look into graphic QL

CollapseExpand
 
gajus profile image
Gajus Kuizinas
Open-source engineer interested in JavaScript, PostgreSQL and DevOps. Follow me on Twitter for outbursts about startups & engineering.

What about GraphQL?

Are you sure you want to hide this comment? It will become hidden in your post, but will still be visible via the comment'spermalink.

For further actions, you may consider blocking this person and/orreporting abuse

DEV Community

We're a place where coders share, stay up-to-date and grow their careers.

Log in Create account

[8]ページ先頭

©2009-2025 Movatter.jp