Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit8e65d9f

Browse files
committed
Fix unique key checks in JSON object constructors
When building a JSON object, the code builds a hash table of keys, toallow checking if the keys are unique. The uniqueness check and addingthe new key happens in json_unique_check_key(), but this assumes thepointer to the key remains valid.Unfortunately, two places passed pointers to keys in a buffer, whilealso appending more data (additional key/value pairs) to the buffer.With enough data the buffer is resized by enlargeStringInfo(), whichcalls repalloc(), invalidating the earlier key pointers.Due to this the uniqueness check may fail with both false negatives andfalse positives, producing JSON objects with duplicate keys or failingto produce a perfectly valid JSON object.This affects multiple functions that enforce uniqueness of keys, allintroduced in PG16 with the new SQL/JSON:- json_object_agg_unique / jsonb_object_agg_unique- json_object / jsonb_objectaggExisting regression tests did not detect the issue, simply because theinitial buffer size is 1024 and the objects were small enough not torequire the repalloc.With a sufficiently large object, AddressSanitizer reported the accessto invalid memory immediately. So would valgrind, of course.Fixed by copying the key into the hash table memory context, and addingregression tests with enough data to repalloc the buffer. Backpatch to16, where the functions were introduced.Reported by Alexander Lakhin. Investigation and initial fix by JunwangZhao, with various improvements and tests by me.Reported-by: Alexander LakhinAuthor: Junwang Zhao, Tomas VondraBackpatch-through: 16Discussion:https://postgr.es/m/18598-3279ed972a2347c7@postgresql.orgDiscussion:https://postgr.es/m/CAEG8a3JjH0ReJF2_O7-8LuEbO69BxPhYeXs95_x7+H9AMWF1gw@mail.gmail.com
1 parent06c2850 commit8e65d9f

File tree

5 files changed

+32
-3
lines changed

5 files changed

+32
-3
lines changed

‎src/backend/utils/adt/json.c

Lines changed: 17 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1181,7 +1181,14 @@ json_object_agg_transfn_worker(FunctionCallInfo fcinfo,
11811181

11821182
if (unique_keys)
11831183
{
1184-
constchar*key=&out->data[key_offset];
1184+
/*
1185+
* Copy the key first, instead of pointing into the buffer. It will be
1186+
* added to the hash table, but the buffer may get reallocated as
1187+
* we're appending more data to it. That would invalidate pointers to
1188+
* keys in the current buffer.
1189+
*/
1190+
constchar*key=MemoryContextStrdup(aggcontext,
1191+
&out->data[key_offset]);
11851192

11861193
if (!json_unique_check_key(&state->unique_check.check,key,0))
11871194
ereport(ERROR,
@@ -1343,8 +1350,15 @@ json_build_object_worker(int nargs, Datum *args, bool *nulls, Oid *types,
13431350

13441351
if (unique_keys)
13451352
{
1346-
/* check key uniqueness after key appending */
1347-
constchar*key=&out->data[key_offset];
1353+
/*
1354+
* check key uniqueness after key appending
1355+
*
1356+
* Copy the key first, instead of pointing into the buffer. It
1357+
* will be added to the hash table, but the buffer may get
1358+
* reallocated as we're appending more data to it. That would
1359+
* invalidate pointers to keys in the current buffer.
1360+
*/
1361+
constchar*key=pstrdup(&out->data[key_offset]);
13481362

13491363
if (!json_unique_check_key(&unique_check.check,key,0))
13501364
ereport(ERROR,

‎src/test/regress/expected/json.out

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2282,6 +2282,9 @@ select json_object('{a,b,"","d e f"}','{1,2,3,"a b c"}');
22822282
{"a" : "1", "b" : "2", "" : "3", "d e f" : "a b c"}
22832283
(1 row)
22842284

2285+
-- json_object_agg_unique requires unique keys
2286+
select json_object_agg_unique(mod(i,100), i) from generate_series(0, 199) i;
2287+
ERROR: duplicate JSON object key value: "0"
22852288
-- json_to_record and json_to_recordset
22862289
select * from json_to_record('{"a":1,"b":"foo","c":"bar"}')
22872290
as x(a int, b text, d text);

‎src/test/regress/expected/sqljson.out

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,6 +247,8 @@ SELECT JSON_OBJECT('a': '1', 'b': NULL, 'c': 2 ABSENT ON NULL);
247247
{"a" : "1", "c" : 2}
248248
(1 row)
249249

250+
SELECT JSON_OBJECT(1: 1, '2': NULL, '3': 1, repeat('x', 1000): 1, 2: repeat('a', 100) WITH UNIQUE);
251+
ERROR: duplicate JSON object key value: "2"
250252
SELECT JSON_OBJECT(1: 1, '1': NULL WITH UNIQUE);
251253
ERROR: duplicate JSON object key value: "1"
252254
SELECT JSON_OBJECT(1: 1, '1': NULL ABSENT ON NULL WITH UNIQUE);
@@ -624,6 +626,9 @@ ERROR: duplicate JSON object key value
624626
SELECT JSON_OBJECTAGG(k: v ABSENT ON NULL WITH UNIQUE KEYS RETURNING jsonb)
625627
FROM (VALUES (1, 1), (1, NULL), (2, 2)) foo(k, v);
626628
ERROR: duplicate JSON object key value
629+
SELECT JSON_OBJECTAGG(mod(i,100): (i)::text FORMAT JSON WITH UNIQUE)
630+
FROM generate_series(0, 199) i;
631+
ERROR: duplicate JSON object key value: "0"
627632
-- Test JSON_OBJECT deparsing
628633
EXPLAIN (VERBOSE, COSTS OFF)
629634
SELECT JSON_OBJECT('foo' : '1' FORMAT JSON, 'bar' : 'baz' RETURNING json);

‎src/test/regress/sql/json.sql

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -748,6 +748,8 @@ select json_object('{a,b,NULL,"d e f"}','{1,2,3,"a b c"}');
748748

749749
select json_object('{a,b,"","d e f"}','{1,2,3,"a b c"}');
750750

751+
-- json_object_agg_unique requires unique keys
752+
select json_object_agg_unique(mod(i,100), i)from generate_series(0,199) i;
751753

752754
-- json_to_record and json_to_recordset
753755

‎src/test/regress/sql/sqljson.sql

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,8 @@ SELECT JSON_OBJECT('a': '1', 'b': NULL, 'c': 2);
7474
SELECT JSON_OBJECT('a':'1','b':NULL,'c':2NULLONNULL);
7575
SELECT JSON_OBJECT('a':'1','b':NULL,'c':2 ABSENTONNULL);
7676

77+
SELECT JSON_OBJECT(1:1,'2':NULL,'3':1, repeat('x',1000):1,2: repeat('a',100) WITH UNIQUE);
78+
7779
SELECT JSON_OBJECT(1:1,'1':NULL WITH UNIQUE);
7880
SELECT JSON_OBJECT(1:1,'1':NULL ABSENTONNULL WITH UNIQUE);
7981
SELECT JSON_OBJECT(1:1,'1':NULLNULLONNULL WITH UNIQUE RETURNING jsonb);
@@ -216,6 +218,9 @@ FROM (VALUES (1, 1), (1, NULL), (2, 2)) foo(k, v);
216218
SELECT JSON_OBJECTAGG(k: v ABSENTONNULL WITH UNIQUE KEYS RETURNING jsonb)
217219
FROM (VALUES (1,1), (1,NULL), (2,2)) foo(k, v);
218220

221+
SELECT JSON_OBJECTAGG(mod(i,100): (i)::text FORMAT JSON WITH UNIQUE)
222+
FROM generate_series(0,199) i;
223+
219224
-- Test JSON_OBJECT deparsing
220225
EXPLAIN (VERBOSE, COSTS OFF)
221226
SELECT JSON_OBJECT('foo' :'1' FORMAT JSON,'bar' :'baz' RETURNING json);

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp