Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commite1be1df

Browse files
committed
Add --sampling-rate option to pgbench.
This allows logging only some fraction of transactions, greatly reducingthe amount of log generated.Tomas Vondra, reviewed by Robert Haas and Jeff Janes.
1 parent7ae1815 commite1be1df

File tree

2 files changed

+67
-11
lines changed

2 files changed

+67
-11
lines changed

‎contrib/pgbench/pgbench.c

Lines changed: 43 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,11 @@ intforeign_keys = 0;
129129
*/
130130
intunlogged_tables=0;
131131

132+
/*
133+
* log sampling rate (1.0 = log everything, 0.0 = option not given)
134+
*/
135+
doublesample_rate=0.0;
136+
132137
/*
133138
* tablespace selection
134139
*/
@@ -370,6 +375,8 @@ usage(void)
370375
" -f FILENAME read transaction script from FILENAME\n"
371376
" -j NUM number of threads (default: 1)\n"
372377
" -l write transaction times to log file\n"
378+
" --sampling-rate NUM\n"
379+
" fraction of transactions to log (e.g. 0.01 for 1%% sample)\n"
373380
" -M simple|extended|prepared\n"
374381
" protocol for submitting queries to server (default: simple)\n"
375382
" -n do not run VACUUM before tests\n"
@@ -883,21 +890,30 @@ doCustom(TState *thread, CState *st, instr_time *conn_time, FILE *logfile)
883890
instr_timediff;
884891
doubleusec;
885892

886-
INSTR_TIME_SET_CURRENT(now);
887-
diff=now;
888-
INSTR_TIME_SUBTRACT(diff,st->txn_begin);
889-
usec= (double)INSTR_TIME_GET_MICROSEC(diff);
893+
/*
894+
* write the log entry if this row belongs to the random sample,
895+
* or no sampling rate was given which means log everything.
896+
*/
897+
if (sample_rate==0.0||
898+
pg_erand48(thread->random_state) <=sample_rate)
899+
{
900+
901+
INSTR_TIME_SET_CURRENT(now);
902+
diff=now;
903+
INSTR_TIME_SUBTRACT(diff,st->txn_begin);
904+
usec= (double)INSTR_TIME_GET_MICROSEC(diff);
890905

891906
#ifndefWIN32
892-
/* This is more than we really ought to know about instr_time */
893-
fprintf(logfile,"%d %d %.0f %d %ld %ld\n",
894-
st->id,st->cnt,usec,st->use_file,
895-
(long)now.tv_sec, (long)now.tv_usec);
907+
/* This is more than we really ought to know about instr_time */
908+
fprintf(logfile,"%d %d %.0f %d %ld %ld\n",
909+
st->id,st->cnt,usec,st->use_file,
910+
(long)now.tv_sec, (long)now.tv_usec);
896911
#else
897-
/* On Windows, instr_time doesn't provide a timestamp anyway */
898-
fprintf(logfile,"%d %d %.0f %d 0 0\n",
899-
st->id,st->cnt,usec,st->use_file);
912+
/* On Windows, instr_time doesn't provide a timestamp anyway */
913+
fprintf(logfile,"%d %d %.0f %d 0 0\n",
914+
st->id,st->cnt,usec,st->use_file);
900915
#endif
916+
}
901917
}
902918

903919
if (commands[st->state]->type==SQL_COMMAND)
@@ -1926,6 +1942,7 @@ main(int argc, char **argv)
19261942
{"index-tablespace",required_argument,NULL,3},
19271943
{"tablespace",required_argument,NULL,2},
19281944
{"unlogged-tables",no_argument,&unlogged_tables,1},
1945+
{"sampling-rate",required_argument,NULL,4},
19291946
{NULL,0,NULL,0}
19301947
};
19311948

@@ -2131,6 +2148,14 @@ main(int argc, char **argv)
21312148
case3:/* index-tablespace */
21322149
index_tablespace=optarg;
21332150
break;
2151+
case4:
2152+
sample_rate=atof(optarg);
2153+
if (sample_rate <=0.0||sample_rate>1.0)
2154+
{
2155+
fprintf(stderr,"invalid sampling rate: %f\n",sample_rate);
2156+
exit(1);
2157+
}
2158+
break;
21342159
default:
21352160
fprintf(stderr,_("Try \"%s --help\" for more information.\n"),progname);
21362161
exit(1);
@@ -2166,6 +2191,13 @@ main(int argc, char **argv)
21662191
exit(1);
21672192
}
21682193

2194+
/* --sampling-rate may be used only with -l */
2195+
if (sample_rate>0.0&& !use_log)
2196+
{
2197+
fprintf(stderr,"log sampling rate is allowed only when logging transactions (-l) \n");
2198+
exit(1);
2199+
}
2200+
21692201
/*
21702202
* is_latencies only works with multiple threads in thread-based
21712203
* implementations, not fork-based ones, because it supposes that the

‎doc/src/sgml/pgbench.sgml

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -316,6 +316,24 @@ pgbench <optional> <replaceable>options</> </optional> <replaceable>dbname</>
316316
</listitem>
317317
</varlistentry>
318318

319+
<varlistentry>
320+
<term><option>--sampling-rate</option> <replaceable>rate</></term>
321+
<listitem>
322+
<para>
323+
Sampling rate, used when writing data into the log, to reduce the
324+
amount of log generated. If this option is given, only the specified
325+
fraction of transactions are logged. 1.0 means all transactions will
326+
be logged, 0.05 means only 5% of the transactions will be logged.
327+
</para>
328+
<para>
329+
Remember to take the sampling rate into account when processing the
330+
log file. For example, when computing tps values, you need to multiply
331+
the numbers accordingly (e.g. with 0.01 sample rate, you'll only get
332+
1/100 of the actual tps).
333+
</para>
334+
</listitem>
335+
</varlistentry>
336+
319337
<varlistentry>
320338
<term><option>-M</option> <replaceable>querymode</></term>
321339
<listitem>
@@ -750,6 +768,12 @@ END;
750768
0 201 2513 0 1175850569 608
751769
0 202 2038 0 1175850569 2663
752770
</screen></para>
771+
772+
<para>
773+
When running a long test on hardware that can handle a lot of transactions,
774+
the log files can become very large. The <option>--sampling-rate</> option
775+
can be used to log only a random sample of transactions.
776+
</para>
753777
</refsect2>
754778

755779
<refsect2>

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp