Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit43620e3

Browse files
committed
Add function to log the memory contexts of specified backend process.
Commit3e98c0b added pg_backend_memory_contexts view to displaythe memory contexts of the backend process. However its target processis limited to the backend that is accessing to the view. So this isnot so convenient when investigating the local memory bloat of otherbackend process. To improve this situation, this commit addspg_log_backend_memory_contexts() function that requests to logthe memory contexts of the specified backend process.This information can be also collected by callingMemoryContextStats(TopMemoryContext) via a debugger. Butthis technique cannot be used in some environments because no debuggeris available there. So, pg_log_backend_memory_contexts() allows us tosee the memory contexts of specified backend more easily.Only superusers are allowed to request to log the memory contextsbecause allowing any users to issue this request at an unbounded ratewould cause lots of log messages and which can lead to denial of service.On receipt of the request, at the next CHECK_FOR_INTERRUPTS(),the target backend logs its memory contexts at LOG_SERVER_ONLY level,so that these memory contexts will appear in the server log but notbe sent to the client. It logs one message per memory context.Because if it buffers all memory contexts into StringInfo to log themas one message, which may require the buffer to be enlarged very muchand lead to OOM error since there can be a large number of memorycontexts in a backend.When a backend process is consuming huge memory, logging all itsmemory contexts might overrun available disk space. To prevent this,now this patch limits the number of child contexts to log per parentto 100. As with MemoryContextStats(), it supposes that practical caseswhere the log gets long will typically be huge numbers of siblingsunder the same parent context; while the additional debugging valuefrom seeing details about individual siblings beyond 100 will not be large.There was another proposed patch to add the function to returnthe memory contexts of specified backend as the result sets,instead of logging them, in the discussion. However that patch isnot included in this commit because it had several issues to address.Thanks to Tatsuhito Kasahara, Andres Freund, Tom Lane, Tomas Vondra,Michael Paquier, Kyotaro Horiguchi and Zhihong Yu for the discussion.Bump catalog version.Author: Atsushi TorikoshiReviewed-by: Kyotaro Horiguchi, Zhihong Yu, Fujii MasaoDiscussion:https://postgr.es/m/0271f440ac77f2a4180e0e56ebd944d1@oss.nttdata.com
1 parent5a71964 commit43620e3

File tree

17 files changed

+320
-48
lines changed

17 files changed

+320
-48
lines changed

‎doc/src/sgml/func.sgml

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24913,6 +24913,26 @@ SELECT collation for ('foo' COLLATE "de_DE");
2491324913
</para></entry>
2491424914
</row>
2491524915

24916+
<row>
24917+
<entry role="func_table_entry"><para role="func_signature">
24918+
<indexterm>
24919+
<primary>pg_log_backend_memory_contexts</primary>
24920+
</indexterm>
24921+
<function>pg_log_backend_memory_contexts</function> ( <parameter>pid</parameter> <type>integer</type> )
24922+
<returnvalue>boolean</returnvalue>
24923+
</para>
24924+
<para>
24925+
Requests to log the memory contexts whose backend process has
24926+
the specified process ID. These memory contexts will be logged at
24927+
<literal>LOG</literal> message level. They will appear in
24928+
the server log based on the log configuration set
24929+
(See <xref linkend="runtime-config-logging"/> for more information),
24930+
but will not be sent to the client whatever the setting of
24931+
<xref linkend="guc-client-min-messages"/>.
24932+
Only superusers can request to log the memory contexts.
24933+
</para></entry>
24934+
</row>
24935+
2491624936
<row>
2491724937
<entry role="func_table_entry"><para role="func_signature">
2491824938
<indexterm>
@@ -24983,6 +25003,38 @@ SELECT collation for ('foo' COLLATE "de_DE");
2498325003
<structname>pg_stat_activity</structname> view.
2498425004
</para>
2498525005

25006+
<para>
25007+
<function>pg_log_backend_memory_contexts</function> can be used
25008+
to log the memory contexts of the backend process. For example,
25009+
<programlisting>
25010+
postgres=# SELECT pg_log_backend_memory_contexts(pg_backend_pid());
25011+
pg_log_backend_memory_contexts
25012+
--------------------------------
25013+
t
25014+
(1 row)
25015+
</programlisting>
25016+
One message for each memory context will be logged. For example:
25017+
<screen>
25018+
LOG: logging memory contexts of PID 10377
25019+
STATEMENT: SELECT pg_log_backend_memory_contexts(pg_backend_pid());
25020+
LOG: level: 0; TopMemoryContext: 80800 total in 6 blocks; 14432 free (5 chunks); 66368 used
25021+
LOG: level: 1; pgstat TabStatusArray lookup hash table: 8192 total in 1 blocks; 1408 free (0 chunks); 6784 used
25022+
LOG: level: 1; TopTransactionContext: 8192 total in 1 blocks; 7720 free (1 chunks); 472 used
25023+
LOG: level: 1; RowDescriptionContext: 8192 total in 1 blocks; 6880 free (0 chunks); 1312 used
25024+
LOG: level: 1; MessageContext: 16384 total in 2 blocks; 5152 free (0 chunks); 11232 used
25025+
LOG: level: 1; Operator class cache: 8192 total in 1 blocks; 512 free (0 chunks); 7680 used
25026+
LOG: level: 1; smgr relation table: 16384 total in 2 blocks; 4544 free (3 chunks); 11840 used
25027+
LOG: level: 1; TransactionAbortContext: 32768 total in 1 blocks; 32504 free (0 chunks); 264 used
25028+
...
25029+
LOG: level: 1; ErrorContext: 8192 total in 1 blocks; 7928 free (3 chunks); 264 used
25030+
LOG: Grand total: 1651920 bytes in 201 blocks; 622360 free (88 chunks); 1029560 used
25031+
</screen>
25032+
For more than 100 child contexts under the same parent one,
25033+
100 child contexts and a summary of the remaining ones will be logged.
25034+
Note that frequent calls to this function could incur significant overhead,
25035+
because it may generate a large number of log messages.
25036+
</para>
25037+
2498625038
</sect2>
2498725039

2498825040
<sect2 id="functions-admin-backup">

‎src/backend/storage/ipc/procsignal.c

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@
3030
#include"storage/shmem.h"
3131
#include"storage/sinval.h"
3232
#include"tcop/tcopprot.h"
33+
#include"utils/memutils.h"
3334

3435
/*
3536
* The SIGUSR1 signal is multiplexed to support signaling multiple event
@@ -657,6 +658,9 @@ procsignal_sigusr1_handler(SIGNAL_ARGS)
657658
if (CheckProcSignal(PROCSIG_BARRIER))
658659
HandleProcSignalBarrierInterrupt();
659660

661+
if (CheckProcSignal(PROCSIG_LOG_MEMORY_CONTEXT))
662+
HandleLogMemoryContextInterrupt();
663+
660664
if (CheckProcSignal(PROCSIG_RECOVERY_CONFLICT_DATABASE))
661665
RecoveryConflictInterrupt(PROCSIG_RECOVERY_CONFLICT_DATABASE);
662666

‎src/backend/tcop/postgres.c

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3327,6 +3327,9 @@ ProcessInterrupts(void)
33273327

33283328
if (ParallelMessagePending)
33293329
HandleParallelMessages();
3330+
3331+
if (LogMemoryContextPending)
3332+
ProcessLogMemoryContextInterrupt();
33303333
}
33313334

33323335

‎src/backend/utils/adt/mcxtfuncs.c

Lines changed: 59 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,8 @@
1818
#include"funcapi.h"
1919
#include"miscadmin.h"
2020
#include"mb/pg_wchar.h"
21+
#include"storage/proc.h"
22+
#include"storage/procarray.h"
2123
#include"utils/builtins.h"
2224

2325
/* ----------
@@ -61,7 +63,7 @@ PutMemoryContextsStatsTupleStore(Tuplestorestate *tupstore,
6163

6264
/* Examine the context itself */
6365
memset(&stat,0,sizeof(stat));
64-
(*context->methods->stats) (context,NULL, (void*)&level,&stat);
66+
(*context->methods->stats) (context,NULL, (void*)&level,&stat, true);
6567

6668
memset(values,0,sizeof(values));
6769
memset(nulls,0,sizeof(nulls));
@@ -155,3 +157,59 @@ pg_get_backend_memory_contexts(PG_FUNCTION_ARGS)
155157

156158
return (Datum)0;
157159
}
160+
161+
/*
162+
* pg_log_backend_memory_contexts
163+
*Signal a backend process to log its memory contexts.
164+
*
165+
* Only superusers are allowed to signal to log the memory contexts
166+
* because allowing any users to issue this request at an unbounded
167+
* rate would cause lots of log messages and which can lead to
168+
* denial of service.
169+
*
170+
* On receipt of this signal, a backend sets the flag in the signal
171+
* handler, which causes the next CHECK_FOR_INTERRUPTS() to log the
172+
* memory contexts.
173+
*/
174+
Datum
175+
pg_log_backend_memory_contexts(PG_FUNCTION_ARGS)
176+
{
177+
intpid=PG_GETARG_INT32(0);
178+
PGPROC*proc=BackendPidGetProc(pid);
179+
180+
/*
181+
* BackendPidGetProc returns NULL if the pid isn't valid; but by the time
182+
* we reach kill(), a process for which we get a valid proc here might
183+
* have terminated on its own. There's no way to acquire a lock on an
184+
* arbitrary process to prevent that. But since this mechanism is usually
185+
* used to debug a backend running and consuming lots of memory, that it
186+
* might end on its own first and its memory contexts are not logged is
187+
* not a problem.
188+
*/
189+
if (proc==NULL)
190+
{
191+
/*
192+
* This is just a warning so a loop-through-resultset will not abort
193+
* if one backend terminated on its own during the run.
194+
*/
195+
ereport(WARNING,
196+
(errmsg("PID %d is not a PostgreSQL server process",pid)));
197+
PG_RETURN_BOOL(false);
198+
}
199+
200+
/* Only allow superusers to log memory contexts. */
201+
if (!superuser())
202+
ereport(ERROR,
203+
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
204+
errmsg("must be a superuser to log memory contexts")));
205+
206+
if (SendProcSignal(pid,PROCSIG_LOG_MEMORY_CONTEXT,proc->backendId)<0)
207+
{
208+
/* Again, just a warning to allow loops */
209+
ereport(WARNING,
210+
(errmsg("could not send signal to process %d: %m",pid)));
211+
PG_RETURN_BOOL(false);
212+
}
213+
214+
PG_RETURN_BOOL(true);
215+
}

‎src/backend/utils/init/globals.c

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ volatile sig_atomic_t ClientConnectionLost = false;
3535
volatilesig_atomic_tIdleInTransactionSessionTimeoutPending= false;
3636
volatilesig_atomic_tIdleSessionTimeoutPending= false;
3737
volatilesig_atomic_tProcSignalBarrierPending= false;
38+
volatilesig_atomic_tLogMemoryContextPending= false;
3839
volatileuint32InterruptHoldoffCount=0;
3940
volatileuint32QueryCancelHoldoffCount=0;
4041
volatileuint32CritSectionCount=0;

‎src/backend/utils/mmgr/aset.c

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -272,7 +272,8 @@ static Size AllocSetGetChunkSpace(MemoryContext context, void *pointer);
272272
staticboolAllocSetIsEmpty(MemoryContextcontext);
273273
staticvoidAllocSetStats(MemoryContextcontext,
274274
MemoryStatsPrintFuncprintfunc,void*passthru,
275-
MemoryContextCounters*totals);
275+
MemoryContextCounters*totals,
276+
boolprint_to_stderr);
276277

277278
#ifdefMEMORY_CONTEXT_CHECKING
278279
staticvoidAllocSetCheck(MemoryContextcontext);
@@ -1336,11 +1337,12 @@ AllocSetIsEmpty(MemoryContext context)
13361337
* printfunc: if not NULL, pass a human-readable stats string to this.
13371338
* passthru: pass this pointer through to printfunc.
13381339
* totals: if not NULL, add stats about this context into *totals.
1340+
* print_to_stderr: print stats to stderr if true, elog otherwise.
13391341
*/
13401342
staticvoid
13411343
AllocSetStats(MemoryContextcontext,
13421344
MemoryStatsPrintFuncprintfunc,void*passthru,
1343-
MemoryContextCounters*totals)
1345+
MemoryContextCounters*totals,boolprint_to_stderr)
13441346
{
13451347
AllocSetset= (AllocSet)context;
13461348
Sizenblocks=0;
@@ -1379,7 +1381,7 @@ AllocSetStats(MemoryContext context,
13791381
"%zu total in %zd blocks; %zu free (%zd chunks); %zu used",
13801382
totalspace,nblocks,freespace,freechunks,
13811383
totalspace-freespace);
1382-
printfunc(context,passthru,stats_string);
1384+
printfunc(context,passthru,stats_string,print_to_stderr);
13831385
}
13841386

13851387
if (totals)

‎src/backend/utils/mmgr/generation.c

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -155,7 +155,8 @@ static Size GenerationGetChunkSpace(MemoryContext context, void *pointer);
155155
staticboolGenerationIsEmpty(MemoryContextcontext);
156156
staticvoidGenerationStats(MemoryContextcontext,
157157
MemoryStatsPrintFuncprintfunc,void*passthru,
158-
MemoryContextCounters*totals);
158+
MemoryContextCounters*totals,
159+
boolprint_to_stderr);
159160

160161
#ifdefMEMORY_CONTEXT_CHECKING
161162
staticvoidGenerationCheck(MemoryContextcontext);
@@ -665,14 +666,15 @@ GenerationIsEmpty(MemoryContext context)
665666
* printfunc: if not NULL, pass a human-readable stats string to this.
666667
* passthru: pass this pointer through to printfunc.
667668
* totals: if not NULL, add stats about this context into *totals.
669+
* print_to_stderr: print stats to stderr if true, elog otherwise.
668670
*
669671
* XXX freespace only accounts for empty space at the end of the block, not
670672
* space of freed chunks (which is unknown).
671673
*/
672674
staticvoid
673675
GenerationStats(MemoryContextcontext,
674676
MemoryStatsPrintFuncprintfunc,void*passthru,
675-
MemoryContextCounters*totals)
677+
MemoryContextCounters*totals,boolprint_to_stderr)
676678
{
677679
GenerationContext*set= (GenerationContext*)context;
678680
Sizenblocks=0;
@@ -704,7 +706,7 @@ GenerationStats(MemoryContext context,
704706
"%zu total in %zd blocks (%zd chunks); %zu free (%zd chunks); %zu used",
705707
totalspace,nblocks,nchunks,freespace,
706708
nfreechunks,totalspace-freespace);
707-
printfunc(context,passthru,stats_string);
709+
printfunc(context,passthru,stats_string,print_to_stderr);
708710
}
709711

710712
if (totals)

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp