Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit42fa4b6

Browse files
committed
Assorted minor cleanups in the test_json_parser module
Per gripes from Michael PaquierDiscussion:https://postgr.es/m/ZhTQ6_w1vwOhqTQI@paquier.xyzAlong the way, also clean up a handful of typos in3311ea8 andea7b4e9, found by Alexander Lakhin, and a couple of stylisticsnafus noted by Daniel Westermann and Daniel Gustafsson.
1 parentb8a7bfa commit42fa4b6

File tree

6 files changed

+43
-29
lines changed

6 files changed

+43
-29
lines changed

‎src/backend/backup/basebackup_incremental.c

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@
3434
#defineBLOCKS_PER_READ512
3535

3636
/*
37-
*we expectthe find the last lines of the manifest, including the checksum,
37+
*We expectto find the last lines of the manifest, including the checksum,
3838
* in the last MIN_CHUNK bytes of the manifest. We trigger an incremental
3939
* parse step if we are about to overflow MAX_CHUNK bytes.
4040
*/
@@ -88,8 +88,8 @@ struct IncrementalBackupInfo
8888
* Files extracted from the backup manifest.
8989
*
9090
* We don't really need this information, because we use WAL summaries to
91-
* figure what's changed. It would be unsafe to just rely on the list of
92-
* files that existed before, because it's possible for a file to be
91+
* figureoutwhat's changed. It would be unsafe to just rely on the list
92+
*offiles that existed before, because it's possible for a file to be
9393
* removed and a new one created with the same name and different
9494
* contents. In such cases, the whole file must still be sent. We can tell
9595
* from the WAL summaries whether that happened, but not from the file

‎src/common/jsonapi.c

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -356,7 +356,7 @@ makeJsonLexContextCstringLen(JsonLexContext *lex, char *json,
356356
* need explicit stacks for predictions, field names and null indicators, but
357357
* we don't need the input, that will be handed in bit by bit to the
358358
* parse routine. We also need an accumulator for partial tokens in case
359-
* the boundary between chunkshappns to fall in the middle of a token.
359+
* the boundary between chunkshappens to fall in the middle of a token.
360360
*/
361361
#defineJS_STACK_CHUNK_SIZE 64
362362
#defineJS_MAX_PROD_LEN 10/* more than we need */
@@ -1414,9 +1414,9 @@ json_lex(JsonLexContext *lex)
14141414
}
14151415

14161416
/*
1417-
* Add any remainingalpha_numeric chars. This takes care of the
1417+
* Add any remainingalphanumeric chars. This takes care of the
14181418
* {null, false, true} literals as well as any trailing
1419-
*alpha-numeric junk on non-string tokens.
1419+
*alphanumeric junk on non-string tokens.
14201420
*/
14211421
for (inti=added;i<lex->input_length;i++)
14221422
{

‎src/common/parse_manifest.c

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -806,7 +806,7 @@ json_manifest_finalize_wal_range(JsonManifestParseState *parse)
806806
* the rest of the file.
807807
*
808808
* For an incremental parse, this will just be called on the last chunk of the
809-
* manifest, and the cryptohash contextpaswed in. For a non-incremental
809+
* manifest, and the cryptohash contextpassed in. For a non-incremental
810810
* parse incr_ctx will be NULL.
811811
*/
812812
staticvoid

‎src/test/modules/test_json_parser/README

Lines changed: 9 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -4,22 +4,21 @@ Module `test_json_parser`
44
This module contains two programs for testing the json parsers.
55

66
- `test_json_parser_incremental` is for testing the incremental parser, It
7-
reads in a file and pases it in very small chunks (60 bytes at a time) to
8-
the incremental parser. It's not meant to be a speed test but to test the
9-
accuracy of the incremental parser. It takes one argument: the name of the
10-
input file.
7+
reads in a file and passes it in very small chunks (default is 60 bytes at a
8+
time) to the incremental parser. It's not meant to be a speed test but to
9+
test the accuracy of the incremental parser. There are two option arguments,
10+
"-c nn" specifies an alternative chunk size, and "-s" specifies using
11+
semantic routines. The semantic routines re-output the json, although not in
12+
a very pretty form. The required non-option argument is the input file name.
1113
- `test_json_parser_perf` is for speed testing both the standard
1214
recursive descent parser and the non-recursive incremental
1315
parser. If given the `-i` flag it uses the non-recursive parser,
14-
otherwise thestardard parser. The remaining flags are the number of
16+
otherwise thestandard parser. The remaining flags are the number of
1517
parsing iterations and the file containing the input. Even when
1618
using the non-recursive parser, the input is passed to the parser in a
1719
single chunk. The results are thus comparable to those of the
1820
standard parser.
1921

20-
The easiest way to use these is to run `make check` and `make speed-check`
21-
22-
The sample input file is a small extract from a list of `delicious`
22+
The sample input file is a small, sanitized extract from a list of `delicious`
2323
bookmarks taken some years ago, all wrapped in a single json
24-
array. 10,000 iterations of parsing this file gives a reasonable
25-
benchmark, and that is what the `speed-check` target does.
24+
array.

‎src/test/modules/test_json_parser/test_json_parser_incremental.c

Lines changed: 21 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -3,16 +3,17 @@
33
* test_json_parser_incremental.c
44
* Test program for incremental JSON parser
55
*
6-
* Copyright (c)2023, PostgreSQL Global Development Group
6+
* Copyright (c)2024, PostgreSQL Global Development Group
77
*
88
* IDENTIFICATION
99
* src/test/modules/test_json_parser/test_json_parser_incremental.c
1010
*
11-
* Thisprogam tests incremental parsing of json. The input is fed into
11+
* Thisprogram tests incremental parsing of json. The input is fed into
1212
* the parser in very small chunks. In practice you would normally use
1313
* much larger chunks, but doing this makes it more likely that the
14-
* full range of incement handling, especially in the lexer, is exercised.
15-
* If the "-c SIZE" option is provided, that chunk size is used instead.
14+
* full range of increment handling, especially in the lexer, is exercised.
15+
* If the "-c SIZE" option is provided, that chunk size is used instead
16+
* of the default of 60.
1617
*
1718
* The argument specifies the file containing the JSON input.
1819
*
@@ -31,6 +32,9 @@
3132
#include"mb/pg_wchar.h"
3233
#include"pg_getopt.h"
3334

35+
#defineBUFSIZE 6000
36+
#defineDEFAULT_CHUNK_SIZE 60
37+
3438
typedefstructDoState
3539
{
3640
JsonLexContext*lex;
@@ -67,14 +71,13 @@ JsonSemAction sem = {
6771
int
6872
main(intargc,char**argv)
6973
{
70-
/* max delicious line length is less than this */
71-
charbuff[6001];
74+
charbuff[BUFSIZE];
7275
FILE*json_file;
7376
JsonParseErrorTyperesult;
7477
JsonLexContextlex;
7578
StringInfoDatajson;
7679
intn_read;
77-
size_tchunk_size=60;
80+
size_tchunk_size=DEFAULT_CHUNK_SIZE;
7881
structstatstatbuf;
7982
off_tbytes_left;
8083
JsonSemAction*testsem=&nullSemAction;
@@ -88,6 +91,11 @@ main(int argc, char **argv)
8891
{
8992
case'c':/* chunksize */
9093
sscanf(optarg,"%zu",&chunk_size);
94+
if (chunk_size>BUFSIZE)
95+
{
96+
fprintf(stderr,"chunk size cannot exceed %d\n",BUFSIZE);
97+
exit(1);
98+
}
9199
break;
92100
case's':/* do semantic processing */
93101
testsem=&sem;
@@ -121,6 +129,12 @@ main(int argc, char **argv)
121129
{
122130
n_read=fread(buff,1,chunk_size,json_file);
123131
appendBinaryStringInfo(&json,buff,n_read);
132+
133+
/*
134+
* Append some trailing junk to the buffer passed to the parser. This
135+
* helps us ensure that the parser does the right thing even if the
136+
* chunk isn't terminated with a '\0'.
137+
*/
124138
appendStringInfoString(&json,"1+23 trailing junk");
125139
bytes_left-=n_read;
126140
if (bytes_left>0)

‎src/test/modules/test_json_parser/test_json_parser_perf.c

Lines changed: 6 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,14 +1,14 @@
11
/*-------------------------------------------------------------------------
22
*
33
* test_json_parser_perf.c
4-
*Performancet est program for both flavors of the JSON parser
4+
*Performance test program for both flavors of the JSON parser
55
*
6-
* Copyright (c)2023, PostgreSQL Global Development Group
6+
* Copyright (c)2024, PostgreSQL Global Development Group
77
*
88
* IDENTIFICATION
99
* src/test/modules/test_json_parser/test_json_parser_perf.c
1010
*
11-
* Thisprogam tests either the standard (recursive descent) JSON parser
11+
* Thisprogram tests either the standard (recursive descent) JSON parser
1212
* or the incremental (table driven) parser, but without breaking the input
1313
* into chunks in the latter case. Thus it can be used to compare the pure
1414
* parsing speed of the two parsers. If the "-i" option is used, then the
@@ -28,11 +28,12 @@
2828
#include<stdio.h>
2929
#include<string.h>
3030

31+
#defineBUFSIZE 6000
32+
3133
int
3234
main(intargc,char**argv)
3335
{
34-
/* max delicious line length is less than this */
35-
charbuff[6001];
36+
charbuff[BUFSIZE];
3637
FILE*json_file;
3738
JsonParseErrorTyperesult;
3839
JsonLexContext*lex;

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp