NotificationsYou must be signed in to change notification settings
Fork6
Star31

Commit9cd00c4

committed

Checkpoint sorting and balancing.

Up to now checkpoints were written in the order they're in theBufferDescriptors. That's nearly random in a lot of cases, whichperforms badly on rotating media, but even on SSDs it causes slowdowns.To avoid that, sort checkpoints before writing them out. We currentlysort by tablespace, relfilenode, fork and block number.One of the major reasons that previously wasn't done, was fear ofimbalance between tablespaces. To address that balance writes betweentablespaces.The other prime concern was that the relatively large allocation to sortthe buffers in might fail, preventing checkpoints from happening. Thuspre-allocate the required memory in shared memory, at server startup.This particularly makes it more efficient to have checkpoint flushingenabled, because that'll often result in a lot of writes that can becoalesced into one flush.Discussion: alpine.DEB.2.10.1506011320000.28433@stoAuthor: Fabien Coelho and Andres Freund

1 parent428b1d6 commit9cd00c4Copy full SHA for 9cd00c4

File tree

5 files changed

+277

-44

lines changed

src
- backend/storage/buffer
- include/storage
  - buf_internals.h
- tools/pgindent
  - typedefs.list

5 files changed

+277

-44

lines changed

`‎src/backend/storage/buffer/README`

Lines changed: 0 additions & 5 deletions

Original file line number	Diff line number	Diff line change
`@@ -267,11 +267,6 @@ only needs to take the lock long enough to read the variable value, not`
`267`	`267`	`while scanning the buffers. (This is a very substantial improvement in`
`268`	`268`	`the contention cost of the writer compared to PG 8.0.)`
`269`	`269`
`270`		`-During a checkpoint, the writer's strategy must be to write every dirty`
`271`		`-buffer (pinned or not!). We may as well make it start this scan from`
`272`		`-nextVictimBuffer, however, so that the first-to-be-written pages are the`
`273`		`-ones that backends might otherwise have to write for themselves soon.`
`274`		`-`
`275`	`270`	`The background writer takes shared content lock on a buffer while writing it`
`276`	`271`	`out (and anyone else who flushes buffer contents to disk must do so too).`
`277`	`272`	`This ensures that the page image transferred to disk is reasonably consistent.`

`‎src/backend/storage/buffer/buf_init.c`

Lines changed: 19 additions & 3 deletions

Original file line number	Diff line number	Diff line change
`@@ -24,6 +24,7 @@ LWLockMinimallyPadded *BufferIOLWLockArray = NULL;`
`24`	`24`	`LWLockTrancheBufferIOLWLockTranche;`
`25`	`25`	`LWLockTrancheBufferContentLWLockTranche;`
`26`	`26`	`WritebackContextBackendWritebackContext;`
	`27`	`+CkptSortItem*CkptBufferIds;`
`27`	`28`
`28`	`29`
`29`	`30`	`/*`
`@@ -70,7 +71,8 @@ InitBufferPool(void)`
`70`	`71`	`{`
`71`	`72`	`boolfoundBufs,`
`72`	`73`	`foundDescs,`
`73`		`-foundIOLocks;`
	`74`	`+foundIOLocks,`
	`75`	`+foundBufCkpt;`
`74`	`76`
`75`	`77`	`/* Align descriptors to a cacheline boundary. */`
`76`	`78`	`BufferDescriptors= (BufferDescPadded*)`
`@@ -104,10 +106,21 @@ InitBufferPool(void)`
`104`	`106`	`LWLockRegisterTranche(LWTRANCHE_BUFFER_CONTENT,`
`105`	`107`	`&BufferContentLWLockTranche);`
`106`	`108`
`107`		`-if (foundDescs\|\|foundBufs\|\|foundIOLocks)`
	`109`	`+/*`
	`110`	`+ * The array used to sort to-be-checkpointed buffer ids is located in`
	`111`	`+ * shared memory, to avoid having to allocate significant amounts of`
	`112`	`+ * memory at runtime. As that'd be in the middle of a checkpoint, or when`
	`113`	`+ * the checkpointer is restarted, memory allocation failures would be`
	`114`	`+ * painful.`
	`115`	`+ */`
	`116`	`+CkptBufferIds= (CkptSortItem*)`
	`117`	`+ShmemInitStruct("Checkpoint BufferIds",`
	`118`	`+NBuffers*sizeof(CkptSortItem),&foundBufCkpt);`
	`119`	`+`
	`120`	`+if (foundDescs\|\|foundBufs\|\|foundIOLocks\|\|foundBufCkpt)`
`108`	`121`	`{`
`109`	`122`	`/* should find all of these, or none of them */`
`110`		`-Assert(foundDescs&&foundBufs&&foundIOLocks);`
	`123`	`+Assert(foundDescs&&foundBufs&&foundIOLocks&&foundBufCkpt);`
`111`	`124`	`/* note: this path is only taken in EXEC_BACKEND case */`
`112`	`125`	`}`
`113`	`126`	`else`
`@@ -190,5 +203,8 @@ BufferShmemSize(void)`
`190`	`203`	`/* to allow aligning the above */`
`191`	`204`	`size=add_size(size,PG_CACHE_LINE_SIZE);`
`192`	`205`
	`206`	`+/* size of checkpoint sort array in bufmgr.c */`
	`207`	`+size=add_size(size,mul_size(NBuffers,sizeof(CkptSortItem)));`
	`208`	`+`
`193`	`209`	`returnsize;`
`194`	`210`	`}`

0 commit comments

Comments

(0)

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Commit9cd00c4

File tree

5 files changed

5 files changed

`‎src/backend/storage/buffer/README`

`‎src/backend/storage/buffer/buf_init.c`

0 commit comments