Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit675c945

Browse files
committed
Move temporary file cleanup to before_shmem_exit().
As reported by a few OSX buildfarm animals there exist at least one path wheretemporary files exist during AtProcExit_Files() processing. As temporary filecleanup causes pgstat reporting, the assertions added inee3f8d3 causedfailures.This is not an OSX specific issue, we were just lucky that timing on OSXreliably triggered the problem. The known way to cause this is a FATAL errorduring perform_base_backup() with a MANIFEST used - adding an elog(FATAL)after InitializeBackupManifest() reliably reproduces the problem in isolation.The problem is that the temporary file created in InitializeBackupManifest()is not cleaned up via resource owner cleanup as WalSndResourceCleanup()currently is only used for non-FATAL errors. That then allows to reachAtProcExit_Files() with existing temporary files, causing the assertionfailure.To fix this problem, move temporary file cleanup to a before_shmem_exit() hookand add assertions ensuring that no temporary files are created before / aftertemporary file management has been initialized / shut down. The cleanest wayto do so seems to be to split fd.c initialization into two, one for plain fileaccess and one for temporary file access.Right now there's no need to perform further fd.c cleanup during process exit,so I just renamed AtProcExit_Files() to BeforeShmemExit_Files(). Alternativelywe could perform another pass through the files to check that no temporaryfiles exist, but the added assertions seem to provide enough protectionagainst that.It might turn out that the assertions added inee3f8d3 will cause too muchnoise - in that case we'll have to downgrade them to a WARNING, at leasttemporarily.This commit is not necessarily the best approach to address this issue, but itshould resolve the buildfarm failures. We can revise later.Author: Andres Freund <andres@anarazel.de>Discussion:https://postgr.es/m/20210807190131.2bm24acbebl4wl6i@alap3.anarazel.de
1 parent256909c commit675c945

File tree

3 files changed

+65
-8
lines changed

3 files changed

+65
-8
lines changed

‎src/backend/storage/file/fd.c‎

Lines changed: 51 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -231,6 +231,11 @@ static bool have_xact_temporary_files = false;
231231
*/
232232
staticuint64temporary_files_size=0;
233233

234+
/* Temporary file access initialized and not yet shut down? */
235+
#ifdefUSE_ASSERT_CHECKING
236+
staticbooltemporary_files_allowed= false;
237+
#endif
238+
234239
/*
235240
* List of OS handles opened with AllocateFile, AllocateDir and
236241
* OpenTransientFile.
@@ -327,7 +332,7 @@ static File OpenTemporaryFileInTablespace(Oid tblspcOid, bool rejectError);
327332
staticboolreserveAllocatedDesc(void);
328333
staticintFreeDesc(AllocateDesc*desc);
329334

330-
staticvoidAtProcExit_Files(intcode,Datumarg);
335+
staticvoidBeforeShmemExit_Files(intcode,Datumarg);
331336
staticvoidCleanupTempFiles(boolisCommit,boolisProcExit);
332337
staticvoidRemovePgTempRelationFiles(constchar*tsdirname);
333338
staticvoidRemovePgTempRelationFilesInDbspace(constchar*dbspacedirname);
@@ -868,6 +873,9 @@ durable_rename_excl(const char *oldfile, const char *newfile, int elevel)
868873
*
869874
* This is called during either normal or standalone backend start.
870875
* It is *not* called in the postmaster.
876+
*
877+
* Note that this does not initialize temporary file access, that is
878+
* separately initialized via InitTemporaryFileAccess().
871879
*/
872880
void
873881
InitFileAccess(void)
@@ -885,9 +893,35 @@ InitFileAccess(void)
885893
VfdCache->fd=VFD_CLOSED;
886894

887895
SizeVfdCache=1;
896+
}
897+
898+
/*
899+
* InitTemporaryFileAccess --- initialize temporary file access during startup
900+
*
901+
* This is called during either normal or standalone backend start.
902+
* It is *not* called in the postmaster.
903+
*
904+
* This is separate from InitFileAccess() because temporary file cleanup can
905+
* cause pgstat reporting. As pgstat is shut down during before_shmem_exit(),
906+
* our reporting has to happen before that. Low level file access should be
907+
* available for longer, hence the separate initialization / shutdown of
908+
* temporary file handling.
909+
*/
910+
void
911+
InitTemporaryFileAccess(void)
912+
{
913+
Assert(SizeVfdCache!=0);/* InitFileAccess() needs to have run*/
914+
Assert(!temporary_files_allowed);/* call me only once */
915+
916+
/*
917+
* Register before-shmem-exit hook to ensure temp files are dropped while
918+
* we can still report stats.
919+
*/
920+
before_shmem_exit(BeforeShmemExit_Files,0);
888921

889-
/* register proc-exit hook to ensure temp files are dropped at exit */
890-
on_proc_exit(AtProcExit_Files,0);
922+
#ifdefUSE_ASSERT_CHECKING
923+
temporary_files_allowed= true;
924+
#endif
891925
}
892926

893927
/*
@@ -1670,6 +1704,8 @@ OpenTemporaryFile(bool interXact)
16701704
{
16711705
Filefile=0;
16721706

1707+
Assert(temporary_files_allowed);/* check temp file access is up */
1708+
16731709
/*
16741710
* Make sure the current resource owner has space for this File before we
16751711
* open it, if we'll be registering it below.
@@ -1805,6 +1841,8 @@ PathNameCreateTemporaryFile(const char *path, bool error_on_failure)
18051841
{
18061842
Filefile;
18071843

1844+
Assert(temporary_files_allowed);/* check temp file access is up */
1845+
18081846
ResourceOwnerEnlargeFiles(CurrentResourceOwner);
18091847

18101848
/*
@@ -1843,6 +1881,8 @@ PathNameOpenTemporaryFile(const char *path, int mode)
18431881
{
18441882
Filefile;
18451883

1884+
Assert(temporary_files_allowed);/* check temp file access is up */
1885+
18461886
ResourceOwnerEnlargeFiles(CurrentResourceOwner);
18471887

18481888
file=PathNameOpenFile(path,mode |PG_BINARY);
@@ -3004,15 +3044,20 @@ AtEOXact_Files(bool isCommit)
30043044
}
30053045

30063046
/*
3007-
*AtProcExit_Files
3047+
*BeforeShmemExit_Files
30083048
*
3009-
*on_proc_exit hook to clean up temp files during backend shutdown.
3049+
*before_shmem_access hook to clean up temp files during backend shutdown.
30103050
* Here, we want to clean up *all* temp files including interXact ones.
30113051
*/
30123052
staticvoid
3013-
AtProcExit_Files(intcode,Datumarg)
3053+
BeforeShmemExit_Files(intcode,Datumarg)
30143054
{
30153055
CleanupTempFiles(false, true);
3056+
3057+
/* prevent further temp files from being created */
3058+
#ifdefUSE_ASSERT_CHECKING
3059+
temporary_files_allowed= false;
3060+
#endif
30163061
}
30173062

30183063
/*

‎src/backend/utils/init/postinit.c‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -517,6 +517,12 @@ BaseInit(void)
517517
*/
518518
DebugFileOpen();
519519

520+
/*
521+
* Initialize file access. Done early so other subsystems can access
522+
* files.
523+
*/
524+
InitFileAccess();
525+
520526
/*
521527
* Initialize statistics reporting. This needs to happen early to ensure
522528
* that pgstat's shutdown callback runs after the shutdown callbacks of
@@ -525,11 +531,16 @@ BaseInit(void)
525531
*/
526532
pgstat_initialize();
527533

528-
/* Do local initialization of file, storage and buffer managers */
529-
InitFileAccess();
534+
/* Do local initialization of storage and buffer managers */
530535
InitSync();
531536
smgrinit();
532537
InitBufferPoolAccess();
538+
539+
/*
540+
* Initialize temporary file access after pgstat, so that the temorary
541+
* file shutdown hook can report temporary file statistics.
542+
*/
543+
InitTemporaryFileAccess();
533544
}
534545

535546

‎src/include/storage/fd.h‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -158,6 +158,7 @@ extern intMakePGDirectory(const char *directoryName);
158158

159159
/* Miscellaneous support routines */
160160
externvoidInitFileAccess(void);
161+
externvoidInitTemporaryFileAccess(void);
161162
externvoidset_max_safe_fds(void);
162163
externvoidcloseAllVfds(void);
163164
externvoidSetTempTablespaces(Oid*tableSpaces,intnumSpaces);

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp