Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit43c1c4f

Browse files
committed
Introduce GUC shared_memory_size_in_huge_pages
This runtime-computed GUC shows the number of huge pages requiredfor the server's main shared memory area, taking advantage of thework done in 0c39c29 and 0bd305e. This is useful for users to estimatethe amount of huge pages required for a server as it becomes possible todo an estimation without having to start the server and potentiallyallocate a large chunk of shared memory.The number of huge pages is calculated based on the existing GUChuge_page_size if set, or by using the system's default by looking at/proc/meminfo on Linux. There is nothing new here as this commit reusesthe existing calculation methods, and just exposes this informationdirectly to the user. The routine calculating the huge page size isrefactored to limit the number of files with platform-specific flags.This new GUC's name was the most popular choice based on the discussiondone. This is only supported on Linux.I have taken the time to test the change on Linux, Windows and MacOS,though for the last two ones large pages are not supported. The firstone calculates correctly the number of pages depending on the existingGUC huge_page_size or the system's default.Thanks to Andres Freund, Robert Haas, Kyotaro Horiguchi, Tom Lane,Justin Pryzby (and anybody forgotten here) for the discussion.Author: Nathan BossartDiscussion:https://postgr.es/m/F2772387-CE0F-46BF-B5F1-CC55516EB885@amazon.com
1 parent5e6716c commit43c1c4f

File tree

8 files changed

+106
-29
lines changed

8 files changed

+106
-29
lines changed

‎doc/src/sgml/config.sgml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10289,6 +10289,27 @@ dynamic_library_path = 'C:\tools\postgresql;H:\my_project\lib;$libdir'
1028910289
</listitem>
1029010290
</varlistentry>
1029110291

10292+
<varlistentry id="guc-shared-memory-size-in-huge-pages" xreflabel="shared_memory_size_in_huge_pages">
10293+
<term><varname>shared_memory_size_in_huge_pages</varname> (<type>integer</type>)
10294+
<indexterm>
10295+
<primary><varname>shared_memory_size_in_huge_pages</varname> configuration parameter</primary>
10296+
</indexterm>
10297+
</term>
10298+
<listitem>
10299+
<para>
10300+
Reports the number of huge pages that are needed for the main shared
10301+
memory area based on the specified <xref linkend="guc-huge-page-size"/>.
10302+
If huge pages are not supported, this will be <literal>-1</literal>.
10303+
</para>
10304+
<para>
10305+
This setting is supported only on <productname>Linux</productname>. It
10306+
is always set to <literal>-1</literal> on other platforms. For more
10307+
details about using huge pages on <productname>Linux</productname>, see
10308+
<xref linkend="linux-huge-pages"/>.
10309+
</para>
10310+
</listitem>
10311+
</varlistentry>
10312+
1029210313
<varlistentry id="guc-ssl-library" xreflabel="ssl_library">
1029310314
<term><varname>ssl_library</varname> (<type>string</type>)
1029410315
<indexterm>

‎doc/src/sgml/ref/postgres-ref.sgml

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -143,7 +143,8 @@ PostgreSQL documentation
143143
<para>
144144
This can be used on a running server for most parameters. However,
145145
the server must be shut down for some runtime-computed parameters
146-
(e.g., <xref linkend="guc-shared-memory-size"/> and
146+
(e.g., <xref linkend="guc-shared-memory-size"/>,
147+
<xref linkend="guc-shared-memory-size-in-huge-pages"/>, and
147148
<xref linkend="guc-wal-segment-size"/>).
148149
</para>
149150

‎doc/src/sgml/runtime.sgml

Lines changed: 12 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1442,31 +1442,28 @@ export PG_OOM_ADJUST_VALUE=0
14421442
with <varname>CONFIG_HUGETLBFS=y</varname> and
14431443
<varname>CONFIG_HUGETLB_PAGE=y</varname>. You will also have to configure
14441444
the operating system to provide enough huge pages of the desired size.
1445-
To estimate the number of huge pages needed, start
1446-
<productname>PostgreSQL</productname> without huge pages enabled and check
1447-
the postmaster's anonymous shared memory segment size, as well as the
1448-
system's default and supported huge page sizes, using the
1449-
<filename>/proc</filename> and <filename>/sys</filename> file systems.
1445+
To determine the number of huge pages needed, use the
1446+
<command>postgres</command> command to see the value of
1447+
<xref linkend="guc-shared-memory-size-in-huge-pages"/>. Note that the
1448+
server must be shut down to view this runtime-computed parameter.
14501449
This might look like:
14511450
<programlisting>
1452-
$ <userinput>head -1 $PGDATA/postmaster.pid</userinput>
1453-
4170
1454-
$ <userinput>pmap 4170 | awk '/rw-s/ &amp;&amp; /zero/ {print $2}'</userinput>
1455-
6490428K
1451+
$ <userinput>postgres -D $PGDATA -C shared_memory_size_in_huge_pages</userinput>
1452+
3170
14561453
$ <userinput>grep ^Hugepagesize /proc/meminfo</userinput>
14571454
Hugepagesize: 2048 kB
14581455
$ <userinput>ls /sys/kernel/mm/hugepages</userinput>
14591456
hugepages-1048576kB hugepages-2048kB
14601457
</programlisting>
14611458

14621459
In this example the default is 2MB, but you can also explicitly request
1463-
either 2MB or 1GB with <xref linkend="guc-huge-page-size"/>.
1460+
either 2MB or 1GB with <xref linkend="guc-huge-page-size"/> to adapt
1461+
the number of pages calculated by
1462+
<varname>shared_memory_size_in_huge_pages</varname>.
14641463

1465-
Assuming <literal>2MB</literal> huge pages,
1466-
<literal>6490428</literal> / <literal>2048</literal> gives approximately
1467-
<literal>3169.154</literal>, so in this example we need at
1468-
least <literal>3170</literal> huge pages. A larger setting would be
1469-
appropriate if other programs on the machine also need huge pages.
1464+
While we need at least <literal>3170</literal> huge pages in this example,
1465+
a larger setting would be appropriate if other programs on the machine
1466+
also need huge pages.
14701467
We can set this with:
14711468
<programlisting>
14721469
# <userinput>sysctl -w vm.nr_hugepages=3170</userinput>

‎src/backend/port/sysv_shmem.c

Lines changed: 30 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -456,8 +456,6 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
456456
returnshmStat.shm_nattch==0 ?SHMSTATE_UNATTACHED :SHMSTATE_ATTACHED;
457457
}
458458

459-
#ifdefMAP_HUGETLB
460-
461459
/*
462460
* Identify the huge page size to use, and compute the related mmap flags.
463461
*
@@ -475,13 +473,19 @@ PGSharedMemoryAttach(IpcMemoryId shmId,
475473
* hugepage sizes, we might want to think about more invasive strategies,
476474
* such as increasing shared_buffers to absorb the extra space.
477475
*
478-
* Returns the (real, assumed or config provided) page size into *hugepagesize,
479-
* and the hugepage-related mmap flags to use into *mmap_flags.
476+
* Returns the (real, assumed or config provided) page size into
477+
* *hugepagesize, and the hugepage-related mmap flags to use into
478+
* *mmap_flags if requested by the caller. If huge pages are not supported,
479+
* *hugepagesize and *mmap_flags are set to 0.
480480
*/
481-
staticvoid
481+
void
482482
GetHugePageSize(Size*hugepagesize,int*mmap_flags)
483483
{
484+
#ifdefMAP_HUGETLB
485+
484486
Sizedefault_hugepagesize=0;
487+
Sizehugepagesize_local=0;
488+
intmmap_flags_local=0;
485489

486490
/*
487491
* System-dependent code to find out the default huge page size.
@@ -519,12 +523,12 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
519523
if (huge_page_size!=0)
520524
{
521525
/* If huge page size is requested explicitly, use that. */
522-
*hugepagesize= (Size)huge_page_size*1024;
526+
hugepagesize_local= (Size)huge_page_size*1024;
523527
}
524528
elseif (default_hugepagesize!=0)
525529
{
526530
/* Otherwise use the system default, if we have it. */
527-
*hugepagesize=default_hugepagesize;
531+
hugepagesize_local=default_hugepagesize;
528532
}
529533
else
530534
{
@@ -536,26 +540,39 @@ GetHugePageSize(Size *hugepagesize, int *mmap_flags)
536540
* writing, there are no reports of any non-Linux systems being picky
537541
* about that.
538542
*/
539-
*hugepagesize=2*1024*1024;
543+
hugepagesize_local=2*1024*1024;
540544
}
541545

542-
*mmap_flags=MAP_HUGETLB;
546+
mmap_flags_local=MAP_HUGETLB;
543547

544548
/*
545549
* On recent enough Linux, also include the explicit page size, if
546550
* necessary.
547551
*/
548552
#if defined(MAP_HUGE_MASK)&& defined(MAP_HUGE_SHIFT)
549-
if (*hugepagesize!=default_hugepagesize)
553+
if (hugepagesize_local!=default_hugepagesize)
550554
{
551-
intshift=pg_ceil_log2_64(*hugepagesize);
555+
intshift=pg_ceil_log2_64(hugepagesize_local);
552556

553-
*mmap_flags |= (shift&MAP_HUGE_MASK) <<MAP_HUGE_SHIFT;
557+
mmap_flags_local |= (shift&MAP_HUGE_MASK) <<MAP_HUGE_SHIFT;
554558
}
555559
#endif
556-
}
560+
561+
/* assign the results found */
562+
if (mmap_flags)
563+
*mmap_flags=mmap_flags_local;
564+
if (hugepagesize)
565+
*hugepagesize=hugepagesize_local;
566+
567+
#else
568+
569+
if (hugepagesize)
570+
*hugepagesize=0;
571+
if (mmap_flags)
572+
*mmap_flags=0;
557573

558574
#endif/* MAP_HUGETLB */
575+
}
559576

560577
/*
561578
* Creates an anonymous mmap()ed shared memory segment.

‎src/backend/port/win32_shmem.c

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -605,3 +605,17 @@ pgwin32_ReserveSharedMemoryRegion(HANDLE hChild)
605605

606606
return true;
607607
}
608+
609+
/*
610+
* This function is provided for consistency with sysv_shmem.c and does not
611+
* provide any useful information for Windows. To obtain the large page size,
612+
* use GetLargePageMinimum() instead.
613+
*/
614+
void
615+
GetHugePageSize(Size*hugepagesize,int*mmap_flags)
616+
{
617+
if (hugepagesize)
618+
*hugepagesize=0;
619+
if (mmap_flags)
620+
*mmap_flags=0;
621+
}

‎src/backend/storage/ipc/ipci.c

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -326,6 +326,7 @@ InitializeShmemGUCs(void)
326326
charbuf[64];
327327
Sizesize_b;
328328
Sizesize_mb;
329+
Sizehp_size;
329330

330331
/*
331332
* Calculate the shared memory size and round up to the nearest megabyte.
@@ -334,4 +335,17 @@ InitializeShmemGUCs(void)
334335
size_mb=add_size(size_b, (1024*1024)-1) / (1024*1024);
335336
sprintf(buf,"%zu",size_mb);
336337
SetConfigOption("shared_memory_size",buf,PGC_INTERNAL,PGC_S_OVERRIDE);
338+
339+
/*
340+
* Calculate the number of huge pages required.
341+
*/
342+
GetHugePageSize(&hp_size,NULL);
343+
if (hp_size!=0)
344+
{
345+
Sizehp_required;
346+
347+
hp_required=add_size(size_b /hp_size,1);
348+
sprintf(buf,"%zu",hp_required);
349+
SetConfigOption("shared_memory_size_in_huge_pages",buf,PGC_INTERNAL,PGC_S_OVERRIDE);
350+
}
337351
}

‎src/backend/utils/misc/guc.c

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -665,6 +665,7 @@ static intmax_identifier_length;
665665
staticintblock_size;
666666
staticintsegment_size;
667667
staticintshared_memory_size_mb;
668+
staticintshared_memory_size_in_huge_pages;
668669
staticintwal_block_size;
669670
staticbooldata_checksums;
670671
staticboolinteger_datetimes;
@@ -2349,6 +2350,17 @@ static struct config_int ConfigureNamesInt[] =
23492350
NULL,NULL,NULL
23502351
},
23512352

2353+
{
2354+
{"shared_memory_size_in_huge_pages",PGC_INTERNAL,PRESET_OPTIONS,
2355+
gettext_noop("Shows the number of huge pages needed for the main shared memory area."),
2356+
gettext_noop("-1 indicates that the value could not be determined."),
2357+
GUC_NOT_IN_SAMPLE |GUC_DISALLOW_IN_FILE |GUC_RUNTIME_COMPUTED
2358+
},
2359+
&shared_memory_size_in_huge_pages,
2360+
-1,-1,INT_MAX,
2361+
NULL,NULL,NULL
2362+
},
2363+
23522364
{
23532365
{"temp_buffers",PGC_USERSET,RESOURCES_MEM,
23542366
gettext_noop("Sets the maximum number of temporary buffers used by each session."),

‎src/include/storage/pg_shmem.h

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,5 +87,6 @@ extern PGShmemHeader *PGSharedMemoryCreate(Size size,
8787
PGShmemHeader**shim);
8888
externboolPGSharedMemoryIsInUse(unsigned longid1,unsigned longid2);
8989
externvoidPGSharedMemoryDetach(void);
90+
externvoidGetHugePageSize(Size*hugepagesize,int*mmap_flags);
9091

9192
#endif/* PG_SHMEM_H */

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp