Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit8b2bcf3

Browse files
Introduce the dynamic shared memory registry.
Presently, the most straightforward way for a shared library to useshared memory is to request it at server startup via ashmem_request_hook, which requires specifying the library inshared_preload_libraries. Alternatively, the library can create adynamic shared memory (DSM) segment, but absent a shared locationto store the segment's handle, other backends cannot use it. Thiscommit introduces a registry for DSM segments so that these otherbackends can look up existing segments with a library-specifiedstring. This allows libraries to easily use shared memory withoutneeding to request it at server startup.The registry is accessed via the new GetNamedDSMSegment() function.This function handles allocating the segment and initializing itvia a provided callback. If another backend already created andinitialized the segment, it simply attaches the segment.GetNamedDSMSegment() locks the registry appropriately to ensurethat only one backend initializes the segment and that all otherbackends just attach it.The registry itself is comprised of a dshash table that stores theDSM segment handles keyed by a library-specified string.Reviewed-by: Michael Paquier, Andrei Lepikhov, Nikita Malakhov, Robert Haas, Bharath Rupireddy, Zhang Mingli, Amul SulDiscussion:https://postgr.es/m/20231205034647.GA2705267%40nathanxps13
1 parent964152c commit8b2bcf3

File tree

21 files changed

+455
-3
lines changed

21 files changed

+455
-3
lines changed

‎doc/src/sgml/xfunc.sgml

Lines changed: 46 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3460,6 +3460,45 @@ LWLockRelease(AddinShmemInitLock);
34603460
the <productname>PostgreSQL</productname> source tree.
34613461
</para>
34623462
</sect3>
3463+
3464+
<sect3 id="xfunc-shared-addin-after-startup">
3465+
<title>Requesting Shared Memory After Startup</title>
3466+
3467+
<para>
3468+
There is another, more flexible method of reserving shared memory that
3469+
can be done after server startup and outside a
3470+
<literal>shmem_request_hook</literal>. To do so, each backend that will
3471+
use the shared memory should obtain a pointer to it by calling:
3472+
<programlisting>
3473+
void *GetNamedDSMSegment(const char *name, size_t size,
3474+
void (*init_callback) (void *ptr),
3475+
bool *found)
3476+
</programlisting>
3477+
If a dynamic shared memory segment with the given name does not yet
3478+
exist, this function will allocate it and initialize it with the provided
3479+
<function>init_callback</function> callback function. If the segment has
3480+
already been allocated and initialized by another backend, this function
3481+
simply attaches the existing dynamic shared memory segment to the current
3482+
backend.
3483+
</para>
3484+
3485+
<para>
3486+
Unlike shared memory reserved at server startup, there is no need to
3487+
acquire <function>AddinShmemInitLock</function> or otherwise take action
3488+
to avoid race conditions when reserving shared memory with
3489+
<function>GetNamedDSMSegment</function>. This function ensures that only
3490+
one backend allocates and initializes the segment and that all other
3491+
backends receive a pointer to the fully allocated and initialized
3492+
segment.
3493+
</para>
3494+
3495+
<para>
3496+
A complete usage example of <function>GetNamedDSMSegment</function> can
3497+
be found in
3498+
<filename>src/test/modules/test_dsm_registry/test_dsm_registry.c</filename>
3499+
in the <productname>PostgreSQL</productname> source tree.
3500+
</para>
3501+
</sect3>
34633502
</sect2>
34643503

34653504
<sect2 id="xfunc-addin-lwlocks">
@@ -3469,8 +3508,9 @@ LWLockRelease(AddinShmemInitLock);
34693508
<title>Requesting LWLocks at Startup</title>
34703509

34713510
<para>
3472-
Add-ins can reserve LWLocks on server startup. As with shared memory,
3473-
the add-in's shared library must be preloaded by specifying it in
3511+
Add-ins can reserve LWLocks on server startup. As with shared memory
3512+
reserved at server startup, the add-in's shared library must be preloaded
3513+
by specifying it in
34743514
<xref linkend="guc-shared-preload-libraries"/><indexterm><primary>shared_preload_libraries</primary></indexterm>,
34753515
and the shared library should register a
34763516
<literal>shmem_request_hook</literal> in its
@@ -3508,7 +3548,10 @@ void LWLockInitialize(LWLock *lock, int tranche_id)
35083548
process allocates a new <literal>tranche_id</literal> and initializes
35093549
each new LWLock. One way to do this is to only call these functions in
35103550
your shared memory initialization code with the
3511-
<function>AddinShmemInitLock</function> held exclusively.
3551+
<function>AddinShmemInitLock</function> held exclusively. If using
3552+
<function>GetNamedDSMSegment</function>, calling these functions in the
3553+
<function>init_callback</function> callback function is sufficient to
3554+
avoid race conditions.
35123555
</para>
35133556

35143557
<para>

‎src/backend/storage/ipc/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ OBJS = \
1212
barrier.o\
1313
dsm.o\
1414
dsm_impl.o\
15+
dsm_registry.o\
1516
ipc.o\
1617
ipci.o\
1718
latch.o\
Lines changed: 198 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,198 @@
1+
/*-------------------------------------------------------------------------
2+
*
3+
* dsm_registry.c
4+
* Functions for interfacing with the dynamic shared memory registry.
5+
*
6+
* This provides a way for libraries to use shared memory without needing
7+
* to request it at startup time via a shmem_request_hook. The registry
8+
* stores dynamic shared memory (DSM) segment handles keyed by a
9+
* library-specified string.
10+
*
11+
* The registry is accessed by calling GetNamedDSMSegment(). If a segment
12+
* with the provided name does not yet exist, it is created and initialized
13+
* with the provided init_callback callback function. Otherwise,
14+
* GetNamedDSMSegment() simply ensures that the segment is attached to the
15+
* current backend. This function guarantees that only one backend
16+
* initializes the segment and that all other backends just attach it.
17+
*
18+
* Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
19+
* Portions Copyright (c) 1994, Regents of the University of California
20+
*
21+
* IDENTIFICATION
22+
* src/backend/storage/ipc/dsm_registry.c
23+
*
24+
*-------------------------------------------------------------------------
25+
*/
26+
27+
#include"postgres.h"
28+
29+
#include"lib/dshash.h"
30+
#include"storage/dsm_registry.h"
31+
#include"storage/lwlock.h"
32+
#include"storage/shmem.h"
33+
#include"utils/memutils.h"
34+
35+
typedefstructDSMRegistryCtxStruct
36+
{
37+
dsa_handledsah;
38+
dshash_table_handledshh;
39+
}DSMRegistryCtxStruct;
40+
41+
staticDSMRegistryCtxStruct*DSMRegistryCtx;
42+
43+
typedefstructDSMRegistryEntry
44+
{
45+
charname[64];
46+
dsm_handlehandle;
47+
size_tsize;
48+
}DSMRegistryEntry;
49+
50+
staticconstdshash_parametersdsh_params= {
51+
offsetof(DSMRegistryEntry,handle),
52+
sizeof(DSMRegistryEntry),
53+
dshash_memcmp,
54+
dshash_memhash,
55+
LWTRANCHE_DSM_REGISTRY_HASH
56+
};
57+
58+
staticdsa_area*dsm_registry_dsa;
59+
staticdshash_table*dsm_registry_table;
60+
61+
Size
62+
DSMRegistryShmemSize(void)
63+
{
64+
returnMAXALIGN(sizeof(DSMRegistryCtxStruct));
65+
}
66+
67+
void
68+
DSMRegistryShmemInit(void)
69+
{
70+
boolfound;
71+
72+
DSMRegistryCtx= (DSMRegistryCtxStruct*)
73+
ShmemInitStruct("DSM Registry Data",
74+
DSMRegistryShmemSize(),
75+
&found);
76+
77+
if (!found)
78+
{
79+
DSMRegistryCtx->dsah=DSA_HANDLE_INVALID;
80+
DSMRegistryCtx->dshh=DSHASH_HANDLE_INVALID;
81+
}
82+
}
83+
84+
/*
85+
* Initialize or attach to the dynamic shared hash table that stores the DSM
86+
* registry entries, if not already done. This must be called before accessing
87+
* the table.
88+
*/
89+
staticvoid
90+
init_dsm_registry(void)
91+
{
92+
/* Quick exit if we already did this. */
93+
if (dsm_registry_table)
94+
return;
95+
96+
/* Otherwise, use a lock to ensure only one process creates the table. */
97+
LWLockAcquire(DSMRegistryLock,LW_EXCLUSIVE);
98+
99+
if (DSMRegistryCtx->dshh==DSHASH_HANDLE_INVALID)
100+
{
101+
/* Initialize dynamic shared hash table for registry. */
102+
dsm_registry_dsa=dsa_create(LWTRANCHE_DSM_REGISTRY_DSA);
103+
dsa_pin(dsm_registry_dsa);
104+
dsa_pin_mapping(dsm_registry_dsa);
105+
dsm_registry_table=dshash_create(dsm_registry_dsa,&dsh_params,NULL);
106+
107+
/* Store handles in shared memory for other backends to use. */
108+
DSMRegistryCtx->dsah=dsa_get_handle(dsm_registry_dsa);
109+
DSMRegistryCtx->dshh=dshash_get_hash_table_handle(dsm_registry_table);
110+
}
111+
else
112+
{
113+
/* Attach to existing dynamic shared hash table. */
114+
dsm_registry_dsa=dsa_attach(DSMRegistryCtx->dsah);
115+
dsa_pin_mapping(dsm_registry_dsa);
116+
dsm_registry_table=dshash_attach(dsm_registry_dsa,&dsh_params,
117+
DSMRegistryCtx->dshh,NULL);
118+
}
119+
120+
LWLockRelease(DSMRegistryLock);
121+
}
122+
123+
/*
124+
* Initialize or attach a named DSM segment.
125+
*
126+
* This routine returns the address of the segment. init_callback is called to
127+
* initialize the segment when it is first created.
128+
*/
129+
void*
130+
GetNamedDSMSegment(constchar*name,size_tsize,
131+
void (*init_callback) (void*ptr),bool*found)
132+
{
133+
DSMRegistryEntry*entry;
134+
MemoryContextoldcontext;
135+
charname_padded[offsetof(DSMRegistryEntry,handle)]= {0};
136+
void*ret;
137+
138+
Assert(found);
139+
140+
if (!name||*name=='\0')
141+
ereport(ERROR,
142+
(errmsg("DSM segment name cannot be empty")));
143+
144+
if (strlen(name) >= offsetof(DSMRegistryEntry,handle))
145+
ereport(ERROR,
146+
(errmsg("DSM segment name too long")));
147+
148+
if (size==0)
149+
ereport(ERROR,
150+
(errmsg("DSM segment size must be nonzero")));
151+
152+
/* Be sure any local memory allocated by DSM/DSA routines is persistent. */
153+
oldcontext=MemoryContextSwitchTo(TopMemoryContext);
154+
155+
/* Connect to the registry. */
156+
init_dsm_registry();
157+
158+
strcpy(name_padded,name);
159+
entry=dshash_find_or_insert(dsm_registry_table,name_padded,found);
160+
if (!(*found))
161+
{
162+
/* Initialize the segment. */
163+
dsm_segment*seg=dsm_create(size,0);
164+
165+
dsm_pin_segment(seg);
166+
dsm_pin_mapping(seg);
167+
entry->handle=dsm_segment_handle(seg);
168+
entry->size=size;
169+
ret=dsm_segment_address(seg);
170+
171+
if (init_callback)
172+
(*init_callback) (ret);
173+
}
174+
elseif (entry->size!=size)
175+
{
176+
ereport(ERROR,
177+
(errmsg("requested DSM segment size does not match size of "
178+
"existing segment")));
179+
}
180+
elseif (!dsm_find_mapping(entry->handle))
181+
{
182+
/* Attach to existing segment. */
183+
dsm_segment*seg=dsm_attach(entry->handle);
184+
185+
dsm_pin_mapping(seg);
186+
ret=dsm_segment_address(seg);
187+
}
188+
else
189+
{
190+
/* Return address of an already-attached segment. */
191+
ret=dsm_segment_address(dsm_find_mapping(entry->handle));
192+
}
193+
194+
dshash_release_lock(dsm_registry_table,entry);
195+
MemoryContextSwitchTo(oldcontext);
196+
197+
returnret;
198+
}

‎src/backend/storage/ipc/ipci.c

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,6 +40,7 @@
4040
#include"replication/walsender.h"
4141
#include"storage/bufmgr.h"
4242
#include"storage/dsm.h"
43+
#include"storage/dsm_registry.h"
4344
#include"storage/ipc.h"
4445
#include"storage/pg_shmem.h"
4546
#include"storage/pmsignal.h"
@@ -115,6 +116,7 @@ CalculateShmemSize(int *num_semaphores)
115116
size=add_size(size,hash_estimate_size(SHMEM_INDEX_SIZE,
116117
sizeof(ShmemIndexEnt)));
117118
size=add_size(size,dsm_estimate_size());
119+
size=add_size(size,DSMRegistryShmemSize());
118120
size=add_size(size,BufferShmemSize());
119121
size=add_size(size,LockShmemSize());
120122
size=add_size(size,PredicateLockShmemSize());
@@ -289,6 +291,7 @@ CreateOrAttachShmemStructs(void)
289291
InitShmemIndex();
290292

291293
dsm_shmem_init();
294+
DSMRegistryShmemInit();
292295

293296
/*
294297
* Set up xlog, clog, and buffers

‎src/backend/storage/ipc/meson.build

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ backend_sources += files(
44
'barrier.c',
55
'dsm.c',
66
'dsm_impl.c',
7+
'dsm_registry.c',
78
'ipc.c',
89
'ipci.c',
910
'latch.c',

‎src/backend/storage/lmgr/lwlock.c

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -190,6 +190,10 @@ static const char *const BuiltinTrancheNames[] = {
190190
"LogicalRepLauncherDSA",
191191
/* LWTRANCHE_LAUNCHER_HASH: */
192192
"LogicalRepLauncherHash",
193+
/* LWTRANCHE_DSM_REGISTRY_DSA: */
194+
"DSMRegistryDSA",
195+
/* LWTRANCHE_DSM_REGISTRY_HASH: */
196+
"DSMRegistryHash",
193197
};
194198

195199
StaticAssertDecl(lengthof(BuiltinTrancheNames)==

‎src/backend/storage/lmgr/lwlocknames.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,3 +55,4 @@ WrapLimitsVacuumLock46
5555
NotifyQueueTailLock47
5656
WaitEventExtensionLock48
5757
WALSummarizerLock49
58+
DSMRegistryLock50

‎src/backend/utils/activity/wait_event_names.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -329,6 +329,7 @@ WrapLimitsVacuum"Waiting to update limits on transaction id and multixact consu
329329
NotifyQueueTail"Waiting to update limit on <command>NOTIFY</command> message storage."
330330
WaitEventExtension"Waiting to read or update custom wait events information for extensions."
331331
WALSummarizer"Waiting to read or update WAL summarization state."
332+
DSMRegistry"Waiting to read or update the dynamic shared memory registry."
332333

333334
#
334335
# END OF PREDEFINED LWLOCKS (DO NOT CHANGE THIS LINE)
@@ -367,6 +368,8 @@ PgStatsHash"Waiting for stats shared memory hash table access."
367368
PgStatsData"Waiting for shared memory stats data access."
368369
LogicalRepLauncherDSA"Waiting to access logical replication launcher's dynamic shared memory allocator."
369370
LogicalRepLauncherHash"Waiting to access logical replication launcher's shared hash table."
371+
DSMRegistryDSA"Waiting to access dynamic shared memory registry's dynamic shared memory allocator."
372+
DSMRegistryHash"Waiting to access dynamic shared memory registry's shared hash table."
370373

371374
#
372375
# Wait Events - Lock

‎src/include/storage/dsm_registry.h

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
/*-------------------------------------------------------------------------
2+
*
3+
* dsm_registry.h
4+
* Functions for interfacing with the dynamic shared memory registry.
5+
*
6+
* Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
7+
* Portions Copyright (c) 1994, Regents of the University of California
8+
*
9+
* src/include/storage/dsm_registry.h
10+
*
11+
*-------------------------------------------------------------------------
12+
*/
13+
#ifndefDSM_REGISTRY_H
14+
#defineDSM_REGISTRY_H
15+
16+
externvoid*GetNamedDSMSegment(constchar*name,size_tsize,
17+
void (*init_callback) (void*ptr),
18+
bool*found);
19+
20+
externSizeDSMRegistryShmemSize(void);
21+
externvoidDSMRegistryShmemInit(void);
22+
23+
#endif/* DSM_REGISTRY_H */

‎src/include/storage/lwlock.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -207,6 +207,8 @@ typedef enum BuiltinTrancheIds
207207
LWTRANCHE_PGSTATS_DATA,
208208
LWTRANCHE_LAUNCHER_DSA,
209209
LWTRANCHE_LAUNCHER_HASH,
210+
LWTRANCHE_DSM_REGISTRY_DSA,
211+
LWTRANCHE_DSM_REGISTRY_HASH,
210212
LWTRANCHE_FIRST_USER_DEFINED,
211213
}BuiltinTrancheIds;
212214

‎src/test/modules/Makefile

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ SUBDIRS = \
1818
test_custom_rmgrs\
1919
test_ddl_deparse\
2020
test_dsa\
21+
test_dsm_registry\
2122
test_extensions\
2223
test_ginpostinglist\
2324
test_integerset\

‎src/test/modules/meson.build

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ subdir('test_copy_callbacks')
1515
subdir('test_custom_rmgrs')
1616
subdir('test_ddl_deparse')
1717
subdir('test_dsa')
18+
subdir('test_dsm_registry')
1819
subdir('test_extensions')
1920
subdir('test_ginpostinglist')
2021
subdir('test_integerset')
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# Generated subdirectories
2+
/log/
3+
/results/
4+
/tmp_check/

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp