Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit9c9b619

Browse files
committed
Remove the CheckpointStartLock in favor of having backends show whether they
are in their commit critical sections via flags in the ProcArray. Checkpointcan watch the ProcArray to determine when it's safe to proceed. This isa considerably better solution to the original problem of race conditionsbetween checkpoint and transaction commit: it speeds up commit, since there'sone less lock to fool with, and it prevents the problem of checkpoint beingdelayed indefinitely when there's a constant flow of commits. Heikki, withsome kibitzing from Tom.
1 parentfb4279e commit9c9b619

File tree

8 files changed

+182
-53
lines changed

8 files changed

+182
-53
lines changed

‎src/backend/access/transam/twophase.c

Lines changed: 13 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
* Portions Copyright (c) 1994, Regents of the University of California
88
*
99
* IDENTIFICATION
10-
*$PostgreSQL: pgsql/src/backend/access/transam/twophase.c,v 1.28 2007/02/13 19:39:42 tgl Exp $
10+
*$PostgreSQL: pgsql/src/backend/access/transam/twophase.c,v 1.29 2007/04/03 16:34:35 tgl Exp $
1111
*
1212
* NOTES
1313
*Each global transaction is associated with a global transaction
@@ -279,6 +279,7 @@ MarkAsPreparing(TransactionId xid, const char *gid,
279279
gxact->proc.pid=0;
280280
gxact->proc.databaseId=databaseid;
281281
gxact->proc.roleId=owner;
282+
gxact->proc.inCommit= false;
282283
gxact->proc.inVacuum= false;
283284
gxact->proc.isAutovacuum= false;
284285
gxact->proc.lwWaiting= false;
@@ -934,18 +935,18 @@ EndPrepare(GlobalTransaction gxact)
934935
* odds of a PANIC actually occurring should be very tiny given that we
935936
* were able to write the bogus CRC above.
936937
*
937-
* We have tolock out checkpoint start here, too; otherwise a checkpoint
938+
* We have toset inCommit here, too; otherwise a checkpoint
938939
* starting immediately after the WAL record is inserted could complete
939940
* without fsync'ing our state file. (This is essentially the same kind
940941
* of race condition as the COMMIT-to-clog-write case that
941-
* RecordTransactionCommit usesCheckpointStartLock for; see notes there.)
942+
* RecordTransactionCommit usesinCommit for; see notes there.)
942943
*
943944
* We save the PREPARE record's location in the gxact for later use by
944945
* CheckPointTwoPhase.
945946
*/
946947
START_CRIT_SECTION();
947948

948-
LWLockAcquire(CheckpointStartLock,LW_SHARED);
949+
MyProc->inCommit= true;
949950

950951
gxact->prepare_lsn=XLogInsert(RM_XACT_ID,XLOG_XACT_PREPARE,
951952
records.head);
@@ -982,10 +983,11 @@ EndPrepare(GlobalTransaction gxact)
982983
MarkAsPrepared(gxact);
983984

984985
/*
985-
* Now we can release the checkpoint start lock: a checkpoint starting
986-
* after this will certainly see the gxact as a candidate for fsyncing.
986+
* Now we can mark ourselves as out of the commit critical section:
987+
* a checkpoint starting after this will certainly see the gxact as a
988+
* candidate for fsyncing.
987989
*/
988-
LWLockRelease(CheckpointStartLock);
990+
MyProc->inCommit= false;
989991

990992
END_CRIT_SECTION();
991993

@@ -1649,7 +1651,7 @@ RecoverPreparedTransactions(void)
16491651
*RecordTransactionCommitPrepared
16501652
*
16511653
* This is basically the same as RecordTransactionCommit: in particular,
1652-
* we musttake theCheckpointStartLock to avoid a race condition.
1654+
* we mustset theinCommit flag to avoid a race condition.
16531655
*
16541656
* We know the transaction made at least one XLOG entry (its PREPARE),
16551657
* so it is never possible to optimize out the commit record.
@@ -1669,7 +1671,7 @@ RecordTransactionCommitPrepared(TransactionId xid,
16691671
START_CRIT_SECTION();
16701672

16711673
/* See notes in RecordTransactionCommit */
1672-
LWLockAcquire(CheckpointStartLock,LW_SHARED);
1674+
MyProc->inCommit= true;
16731675

16741676
/* Emit the XLOG commit record */
16751677
xlrec.xid=xid;
@@ -1713,8 +1715,8 @@ RecordTransactionCommitPrepared(TransactionId xid,
17131715
/* to avoid race conditions, the parent must commit first */
17141716
TransactionIdCommitTree(nchildren,children);
17151717

1716-
/* Checkpointis allowed again */
1717-
LWLockRelease(CheckpointStartLock);
1718+
/* Checkpointcan proceed now */
1719+
MyProc->inCommit= false;
17181720

17191721
END_CRIT_SECTION();
17201722
}

‎src/backend/access/transam/xact.c

Lines changed: 25 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
*
1111
*
1212
* IDENTIFICATION
13-
* $PostgreSQL: pgsql/src/backend/access/transam/xact.c,v 1.238 2007/03/22 19:55:04 tgl Exp $
13+
* $PostgreSQL: pgsql/src/backend/access/transam/xact.c,v 1.239 2007/04/03 16:34:35 tgl Exp $
1414
*
1515
*-------------------------------------------------------------------------
1616
*/
@@ -716,35 +716,37 @@ RecordTransactionCommit(void)
716716

717717
START_CRIT_SECTION();
718718

719-
/*
720-
* If our transaction made any transaction-controlled XLOG entries, we
721-
* need to lock out checkpoint start between writing our XLOG record
722-
* and updating pg_clog. Otherwise it is possible for the checkpoint
723-
* to set REDO after the XLOG record but fail to flush the pg_clog
724-
* update to disk, leading to loss of the transaction commit if we
725-
* crash a little later. Slightly klugy fix for problem discovered
726-
* 2004-08-10.
727-
*
728-
* (If it made no transaction-controlled XLOG entries, its XID appears
729-
* nowhere in permanent storage, so no one else will ever care if it
730-
* committed; so it doesn't matter if we lose the commit flag.)
731-
*
732-
* Note we only need a shared lock.
733-
*/
734-
madeTCentries= (MyLastRecPtr.xrecoff!=0);
735-
if (madeTCentries)
736-
LWLockAcquire(CheckpointStartLock,LW_SHARED);
737-
738719
/*
739720
* We only need to log the commit in XLOG if the transaction made any
740721
* transaction-controlled XLOG entries or will delete files.
741722
*/
723+
madeTCentries= (MyLastRecPtr.xrecoff!=0);
742724
if (madeTCentries||nrels>0)
743725
{
744726
XLogRecDatardata[3];
745727
intlastrdata=0;
746728
xl_xact_commitxlrec;
747729

730+
/*
731+
* Mark ourselves as within our "commit critical section". This
732+
* forces any concurrent checkpoint to wait until we've updated
733+
* pg_clog. Without this, it is possible for the checkpoint to
734+
* set REDO after the XLOG record but fail to flush the pg_clog
735+
* update to disk, leading to loss of the transaction commit if
736+
* the system crashes a little later.
737+
*
738+
* Note: we could, but don't bother to, set this flag in
739+
* RecordTransactionAbort. That's because loss of a transaction
740+
* abort is noncritical; the presumption would be that it aborted,
741+
* anyway.
742+
*
743+
* It's safe to change the inCommit flag of our own backend
744+
* without holding the ProcArrayLock, since we're the only one
745+
* modifying it. This makes checkpoint's determination of which
746+
* xacts are inCommit a bit fuzzy, but it doesn't matter.
747+
*/
748+
MyProc->inCommit= true;
749+
748750
xlrec.xtime=time(NULL);
749751
xlrec.nrels=nrels;
750752
xlrec.nsubxacts=nchildren;
@@ -825,9 +827,8 @@ RecordTransactionCommit(void)
825827
TransactionIdCommitTree(nchildren,children);
826828
}
827829

828-
/* Unlock checkpoint lock if we acquired it */
829-
if (madeTCentries)
830-
LWLockRelease(CheckpointStartLock);
830+
/* Checkpoint can proceed now */
831+
MyProc->inCommit= false;
831832

832833
END_CRIT_SECTION();
833834
}
@@ -1961,6 +1962,7 @@ AbortTransaction(void)
19611962
MyProc->xid=InvalidTransactionId;
19621963
MyProc->xmin=InvalidTransactionId;
19631964
MyProc->inVacuum= false;/* must be cleared with xid/xmin */
1965+
MyProc->inCommit= false;/* be sure this gets cleared */
19641966

19651967
/* Clear the subtransaction-XID cache too while holding the lock */
19661968
MyProc->subxids.nxids=0;

‎src/backend/access/transam/xlog.c

Lines changed: 40 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
* Portions Copyright (c) 1996-2007, PostgreSQL Global Development Group
88
* Portions Copyright (c) 1994, Regents of the University of California
99
*
10-
* $PostgreSQL: pgsql/src/backend/access/transam/xlog.c,v 1.266 2007/04/0304:14:26 tgl Exp $
10+
* $PostgreSQL: pgsql/src/backend/access/transam/xlog.c,v 1.267 2007/04/0316:34:35 tgl Exp $
1111
*
1212
*-------------------------------------------------------------------------
1313
*/
@@ -5366,6 +5366,8 @@ CreateCheckPoint(bool shutdown, bool force)
53665366
intnsegsadded=0;
53675367
intnsegsremoved=0;
53685368
intnsegsrecycled=0;
5369+
TransactionId*inCommitXids;
5370+
intnInCommit;
53695371

53705372
/*
53715373
* Acquire CheckpointLock to ensure only one checkpoint happens at a time.
@@ -5392,14 +5394,9 @@ CreateCheckPoint(bool shutdown, bool force)
53925394
checkPoint.time=time(NULL);
53935395

53945396
/*
5395-
* We must hold CheckpointStartLock while determining the checkpoint REDO
5396-
* pointer. This ensures that any concurrent transaction commits will be
5397-
* either not yet logged, or logged and recorded in pg_clog. See notes in
5398-
* RecordTransactionCommit().
5397+
* We must hold WALInsertLock while examining insert state to determine
5398+
* the checkpoint REDO pointer.
53995399
*/
5400-
LWLockAcquire(CheckpointStartLock,LW_EXCLUSIVE);
5401-
5402-
/* And we need WALInsertLock too */
54035400
LWLockAcquire(WALInsertLock,LW_EXCLUSIVE);
54045401

54055402
/*
@@ -5431,7 +5428,6 @@ CreateCheckPoint(bool shutdown, bool force)
54315428
ControlFile->checkPointCopy.redo.xrecoff)
54325429
{
54335430
LWLockRelease(WALInsertLock);
5434-
LWLockRelease(CheckpointStartLock);
54355431
LWLockRelease(CheckpointLock);
54365432
END_CRIT_SECTION();
54375433
return;
@@ -5476,17 +5472,48 @@ CreateCheckPoint(bool shutdown, bool force)
54765472
}
54775473

54785474
/*
5479-
* Now we can release insert lock and checkpoint start lock, allowing
5480-
*other xacts to proceed evenwhile we are flushing disk buffers.
5475+
* Now we can releaseWALinsert lock, allowing other xacts to proceed
5476+
* while we are flushing disk buffers.
54815477
*/
54825478
LWLockRelease(WALInsertLock);
54835479

5484-
LWLockRelease(CheckpointStartLock);
5485-
54865480
if (!shutdown)
54875481
ereport(DEBUG2,
54885482
(errmsg("checkpoint starting")));
54895483

5484+
/*
5485+
* Before flushing data, we must wait for any transactions that are
5486+
* currently in their commit critical sections. If an xact inserted its
5487+
* commit record into XLOG just before the REDO point, then a crash
5488+
* restart from the REDO point would not replay that record, which means
5489+
* that our flushing had better include the xact's update of pg_clog. So
5490+
* we wait till he's out of his commit critical section before proceeding.
5491+
* See notes in RecordTransactionCommit().
5492+
*
5493+
* Because we've already released WALInsertLock, this test is a bit fuzzy:
5494+
* it is possible that we will wait for xacts we didn't really need to
5495+
* wait for. But the delay should be short and it seems better to make
5496+
* checkpoint take a bit longer than to hold locks longer than necessary.
5497+
* (In fact, the whole reason we have this issue is that xact.c does
5498+
* commit record XLOG insertion and clog update as two separate steps
5499+
* protected by different locks, but again that seems best on grounds
5500+
* of minimizing lock contention.)
5501+
*
5502+
* A transaction that has not yet set inCommit when we look cannot be
5503+
* at risk, since he's not inserted his commit record yet; and one that's
5504+
* already cleared it is not at risk either, since he's done fixing clog
5505+
* and we will correctly flush the update below. So we cannot miss any
5506+
* xacts we need to wait for.
5507+
*/
5508+
nInCommit=GetTransactionsInCommit(&inCommitXids);
5509+
if (nInCommit>0)
5510+
{
5511+
do {
5512+
pg_usleep(10000L);/* wait for 10 msec */
5513+
}while (HaveTransactionsInCommit(inCommitXids,nInCommit));
5514+
}
5515+
pfree(inCommitXids);
5516+
54905517
/*
54915518
* Get the other info we need for the checkpoint record.
54925519
*/

‎src/backend/storage/ipc/procarray.c

Lines changed: 93 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@
2323
*
2424
*
2525
* IDENTIFICATION
26-
* $PostgreSQL: pgsql/src/backend/storage/ipc/procarray.c,v 1.23 2007/03/25 19:45:14 tgl Exp $
26+
* $PostgreSQL: pgsql/src/backend/storage/ipc/procarray.c,v 1.24 2007/04/03 16:34:36 tgl Exp $
2727
*
2828
*-------------------------------------------------------------------------
2929
*/
@@ -739,6 +739,98 @@ DatabaseCancelAutovacuumActivity(Oid databaseId, bool ignoreMyself)
739739
return (num!=0);
740740
}
741741

742+
/*
743+
* GetTransactionsInCommit -- Get the XIDs of transactions that are committing
744+
*
745+
* Constructs an array of XIDs of transactions that are currently in commit
746+
* critical sections, as shown by having inCommit set in their PGPROC entries.
747+
*
748+
* *xids_p is set to a palloc'd array that should be freed by the caller.
749+
* The return value is the number of valid entries.
750+
*
751+
* Note that because backends set or clear inCommit without holding any lock,
752+
* the result is somewhat indeterminate, but we don't really care. Even in
753+
* a multiprocessor with delayed writes to shared memory, it should be certain
754+
* that setting of inCommit will propagate to shared memory when the backend
755+
* takes the WALInsertLock, so we cannot fail to see an xact as inCommit if
756+
* it's already inserted its commit record. Whether it takes a little while
757+
* for clearing of inCommit to propagate is unimportant for correctness.
758+
*/
759+
int
760+
GetTransactionsInCommit(TransactionId**xids_p)
761+
{
762+
ProcArrayStruct*arrayP=procArray;
763+
TransactionId*xids;
764+
intnxids;
765+
intindex;
766+
767+
xids= (TransactionId*)palloc(arrayP->maxProcs*sizeof(TransactionId));
768+
nxids=0;
769+
770+
LWLockAcquire(ProcArrayLock,LW_SHARED);
771+
772+
for (index=0;index<arrayP->numProcs;index++)
773+
{
774+
PGPROC*proc=arrayP->procs[index];
775+
/* Fetch xid just once - see GetNewTransactionId */
776+
TransactionIdpxid=proc->xid;
777+
778+
if (proc->inCommit&&TransactionIdIsValid(pxid))
779+
xids[nxids++]=pxid;
780+
}
781+
782+
LWLockRelease(ProcArrayLock);
783+
784+
*xids_p=xids;
785+
returnnxids;
786+
}
787+
788+
/*
789+
* HaveTransactionsInCommit -- Are any of the specified XIDs in commit?
790+
*
791+
* This is used with the results of GetTransactionsInCommit to see if any
792+
* of the specified XIDs are still in their commit critical sections.
793+
*
794+
* Note: this is O(N^2) in the number of xacts that are/were in commit, but
795+
* those numbers should be small enough for it not to be a problem.
796+
*/
797+
bool
798+
HaveTransactionsInCommit(TransactionId*xids,intnxids)
799+
{
800+
boolresult= false;
801+
ProcArrayStruct*arrayP=procArray;
802+
intindex;
803+
804+
LWLockAcquire(ProcArrayLock,LW_SHARED);
805+
806+
for (index=0;index<arrayP->numProcs;index++)
807+
{
808+
PGPROC*proc=arrayP->procs[index];
809+
/* Fetch xid just once - see GetNewTransactionId */
810+
TransactionIdpxid=proc->xid;
811+
812+
if (proc->inCommit&&TransactionIdIsValid(pxid))
813+
{
814+
inti;
815+
816+
for (i=0;i<nxids;i++)
817+
{
818+
if (xids[i]==pxid)
819+
{
820+
result= true;
821+
break;
822+
}
823+
}
824+
if (result)
825+
break;
826+
}
827+
}
828+
829+
LWLockRelease(ProcArrayLock);
830+
831+
returnresult;
832+
}
833+
742834
/*
743835
* BackendPidGetProc -- get a backend's PGPROC given its PID
744836
*

‎src/backend/storage/lmgr/proc.c

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
*
99
*
1010
* IDENTIFICATION
11-
* $PostgreSQL: pgsql/src/backend/storage/lmgr/proc.c,v 1.186 2007/03/07 13:35:03 alvherre Exp $
11+
* $PostgreSQL: pgsql/src/backend/storage/lmgr/proc.c,v 1.187 2007/04/03 16:34:36 tgl Exp $
1212
*
1313
*-------------------------------------------------------------------------
1414
*/
@@ -259,6 +259,7 @@ InitProcess(void)
259259
/* databaseId and roleId will be filled in later */
260260
MyProc->databaseId=InvalidOid;
261261
MyProc->roleId=InvalidOid;
262+
MyProc->inCommit= false;
262263
MyProc->inVacuum= false;
263264
MyProc->isAutovacuum=IsAutoVacuumWorkerProcess();
264265
MyProc->lwWaiting= false;
@@ -392,6 +393,7 @@ InitAuxiliaryProcess(void)
392393
MyProc->xmin=InvalidTransactionId;
393394
MyProc->databaseId=InvalidOid;
394395
MyProc->roleId=InvalidOid;
396+
MyProc->inCommit= false;
395397
MyProc->inVacuum= false;
396398
MyProc->isAutovacuum=IsAutoVacuumLauncherProcess();/* is this needed? */
397399
MyProc->lwWaiting= false;

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp