- Notifications
You must be signed in to change notification settings - Fork516
Description
Motivation
Postgres itself has great type inference when executing prepared statements without types, e.g.:
PREPARE coercion_testASSELECT $1+2;EXECUTE coercion_test('123');# 125, despite using a string literal
Currently, it's unclear how to leverage that type inference when using therust-postgres
packages. The ability to leverage that type inference would be useful for supporting Postgres types that don't yet have a robustToSql
implementation (e.g.MONEY
) or for skipping the entire Rust type conversion step where query parameters might not be known ahead of time, or that originate via protocols that can't express the same complex types as Rust (e.g JSON, as mentioned in#668).
Prior Art
Similar libraries attempt to leverage this inference as a fallback case where types are not explicitly provided.libpq
'sPQexecParams
includesparamTypes[]
, which, if left empty or uses a0
in place of anOid
for a param, will infer "a data type for the parameter symbol in the same way it would do for an untyped literal string".node-postgres
appears to do a similar thing: every type that isn'tnull
or aBuffer
is written directly into the message without any attempt at differentiating between types (as far as I can tell).
Previous attempts
Trying to implement this myself in userland without messing around with anyrust-postgres
internals, I tried the following:
using
prepare_typed
with every param explicitly typed toType::TEXT
, as suggested in[Tokio postgres] [Question] getting parameters from JSON and casting them to valid types in Rust #668. But this overrides Postgres's inference, rather than handling coercion of string literals into inferred types. Specifically:let statement = client.prepare_typed("SELECT $1 + 2",&[Type::TEXT]).await?;client.query(&statement,&[&"123"]).await?;
...results in the following error:
db error: ERROR: operator does not exist: text + integerHINT: No operator matches the given name and argument types. You might need to add explicit type casts.
using an empty array of Types or using a slice of
Type::UNKNOWN
inprepare_typed
, or usingprepare
orquery
(which callsprepare
for all non-Statement
s, it looks like) directly results in a conversion error, e.g.error serializing parameter 0: cannot convert between the Rust type `&&&str` and the Postgres type `int4`
using a
RawValue
that represents a nullable "string literal" and implementsToSql
such that type-checking byrust-postgres
is completely bypassed:pubstructRawValue<'a>{raw:&'aOption<&'a[u8]>}impl<'a>ToSqlforRawValue<'a>{fnto_sql(&self,type_:&Type,out:&mutBytesMut) ->Result<IsNull,Error>{matchself.raw{Some(raw) =>{ out.put_slice(raw);// or e.g. <&str as ToSql>::to_sql()Ok(IsNull::No)},None =>Ok(IsNull::Yes)}}fnaccepts(_:&Type) ->bool{true}fnto_sql_checked(&self,type_:&Type,out:&mutBytesMut) ->Result<IsNull,Error>{// YOLO/bypass those type checksself.to_sql(type_, out)}}
...which, predictably, results in various message encoding errors, depending on the specifics of how one attempts to write to the output buffer directly (usually
insufficient data left in message
orincorrect binary data format in bind parameter 1
). This 👆 is unsurprising, since my understanding is that a) there's no way to prepare an untyped statement, b) preparation is unavoidable as part of the current implementation ofToStatement
forstr
and c) the current encoding of messages requires alignment of types between those inferred fromprepare
and those written to the output buffer by aToSql
implementation (although my understanding of that last point is shaky at best... still navigating my way through the codebase to figure out why/where that association is enforced in the protocol).
Next Steps
I'm not an expert on the Postgres message protocol or the internals of this codebase by any means, but here are a few approaches that I thought of to handle this case:
- implement a
Type
forOid
0
(or some equivalent pseudo-type that can be matched on later) so that users can explicitly delegate inference of types duringprepare_typed
- expose an untyped query method on
Client
(maybequery_untyped?
) - cop the
node-postgres
M.O. and delegate type inference down to Postgres by default (<- this is probably not the right approach, since it would probably be a breaking change, but worth mentioning)
It's also entirely possible that it's already simple enough to handle this case usingToSql
alone, and I'm either misunderstanding how this library works or the Postgres message protocol itself! In which case, it would be nice to document or provide an example for this kind of inference pass-through, as there have already been a few issues in a similar vein (e.g.#668,#722,#377).
Thanks in advance for your help! If it turns out that I'm not misunderstanding the capabilities of Postgres andrust-postgres
and you'd like to implement this kind of inference as a crate-side feature, then I'd be happy to help out with a PR (with a bit of guidance along the way).