Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitd5b31cc

Browse files
committed
Fix an O(N^2) performance issue for sessions modifying many relations.
AtEOXact_RelationCache() scanned the entire relation cache at the end ofany transaction that created a new relation or assigned a new relfilenode.Thus, clients such as pg_restore had an O(N^2) performance problem thatwould start to be noticeable after creating 10000 or so tables. Sincetypically only a small number of relcache entries need any cleanup, wecan fix this by keeping a small list of their OIDs and doing hash_searchesfor them. We fall back to the full-table scan if the list overflows.Ideally, the maximum list length would be set at the point where Nhash_searches would cost just less than the full-table scan. Some quickexperimentation says that point might be around 50-100; I (tgl)conservatively set MAX_EOXACT_LIST = 32. For the case that we're worriedabout here, which is short single-statement transactions, it's unlikelythere would ever be more than about a dozen list entries anyway; so it'sprobably not worth being too tense about the value.We could avoid the hash_searches by instead keeping the target relcacheentries linked into a list, but that would be noticeably more complicatedand bug-prone because of the need to maintain such a list in the face ofrelcache entry drops. Since a relcache entry can only need such cleanupafter a somewhat-heavyweight filesystem operation, trying to save ahash_search per cleanup doesn't seem very useful anyway --- it's the scanover all the not-needing-cleanup entries that we wish to avoid here.Jeff Janes, reviewed and tweaked a bit by Tom Lane
1 parent0a2da52 commitd5b31cc

File tree

1 file changed

+128
-46
lines changed

1 file changed

+128
-46
lines changed

‎src/backend/utils/cache/relcache.c

Lines changed: 128 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -137,9 +137,27 @@ static long relcacheInvalsReceived = 0L;
137137
staticList*initFileRelationIds=NIL;
138138

139139
/*
140-
* This flag lets us optimize away work in AtEO(Sub)Xact_RelationCache().
140+
* eoxact_list[] stores the OIDs of relations that (might) need AtEOXact
141+
* cleanup work. This list intentionally has limited size; if it overflows,
142+
* we fall back to scanning the whole hashtable. There is no value in a very
143+
* large list because (1) at some point, a hash_seq_search scan is faster than
144+
* retail lookups, and (2) the value of this is to reduce EOXact work for
145+
* short transactions, which can't have dirtied all that many tables anyway.
146+
* EOXactListAdd() does not bother to prevent duplicate list entries, so the
147+
* cleanup processing must be idempotent.
141148
*/
142-
staticboolneed_eoxact_work= false;
149+
#defineMAX_EOXACT_LIST 32
150+
staticOideoxact_list[MAX_EOXACT_LIST];
151+
staticinteoxact_list_len=0;
152+
staticbooleoxact_list_overflowed= false;
153+
154+
#defineEOXactListAdd(rel) \
155+
do { \
156+
if (eoxact_list_len < MAX_EOXACT_LIST) \
157+
eoxact_list[eoxact_list_len++] = (rel)->rd_id; \
158+
else \
159+
eoxact_list_overflowed = true; \
160+
} while (0)
143161

144162

145163
/*
@@ -204,6 +222,9 @@ static void RelationClearRelation(Relation relation, bool rebuild);
204222

205223
staticvoidRelationReloadIndexInfo(Relationrelation);
206224
staticvoidRelationFlushRelation(Relationrelation);
225+
staticvoidAtEOXact_cleanup(Relationrelation,boolisCommit);
226+
staticvoidAtEOSubXact_cleanup(Relationrelation,boolisCommit,
227+
SubTransactionIdmySubid,SubTransactionIdparentSubid);
207228
staticboolload_relcache_init_file(boolshared);
208229
staticvoidwrite_relcache_init_file(boolshared);
209230
staticvoidwrite_item(constvoid*data,Sizelen,FILE*fp);
@@ -2275,38 +2296,69 @@ AtEOXact_RelationCache(bool isCommit)
22752296
{
22762297
HASH_SEQ_STATUSstatus;
22772298
RelIdCacheEnt*idhentry;
2299+
inti;
22782300

22792301
/*
2280-
* To speed up transaction exit, we want to avoid scanning the relcache
2281-
* unless there is actually something for this routine to do. Other than
2282-
* the debug-only Assert checks, most transactions don't create any work
2283-
* for us to do here, so we keep a static flag that gets set if there is
2284-
* anything to do.(Currently, this means either a relation is created in
2285-
* the current xact, or one is given a new relfilenode, or an index list
2286-
* is forced.)For simplicity, the flag remains set till end of top-level
2287-
* transaction, even though we could clear it at subtransaction end in
2288-
* some cases.
2289-
*/
2290-
if (!need_eoxact_work
2291-
#ifdefUSE_ASSERT_CHECKING
2292-
&& !assert_enabled
2293-
#endif
2294-
)
2295-
return;
2296-
2297-
hash_seq_init(&status,RelationIdCache);
2298-
2299-
while ((idhentry= (RelIdCacheEnt*)hash_seq_search(&status))!=NULL)
2302+
* Unless the eoxact_list[] overflowed, we only need to examine the rels
2303+
* listed in it. Otherwise fall back on a hash_seq_search scan.
2304+
*
2305+
* For simplicity, eoxact_list[] entries are not deleted till end of
2306+
* top-level transaction, even though we could remove them at
2307+
* subtransaction end in some cases, or remove relations from the list if
2308+
* they are cleared for other reasons. Therefore we should expect the
2309+
* case that list entries are not found in the hashtable; if not, there's
2310+
* nothing to do for them.
2311+
*/
2312+
if (eoxact_list_overflowed)
23002313
{
2301-
Relationrelation=idhentry->reldesc;
2314+
hash_seq_init(&status,RelationIdCache);
2315+
while ((idhentry= (RelIdCacheEnt*)hash_seq_search(&status))!=NULL)
2316+
{
2317+
AtEOXact_cleanup(idhentry->reldesc,isCommit);
2318+
}
2319+
}
2320+
else
2321+
{
2322+
for (i=0;i<eoxact_list_len;i++)
2323+
{
2324+
idhentry= (RelIdCacheEnt*)hash_search(RelationIdCache,
2325+
(void*)&eoxact_list[i],
2326+
HASH_FIND,
2327+
NULL);
2328+
if (idhentry!=NULL)
2329+
AtEOXact_cleanup(idhentry->reldesc,isCommit);
2330+
}
2331+
}
23022332

2333+
/* Now we're out of the transaction and can clear the list */
2334+
eoxact_list_len=0;
2335+
eoxact_list_overflowed= false;
2336+
}
2337+
2338+
/*
2339+
* AtEOXact_cleanup
2340+
*
2341+
*Clean up a single rel at main-transaction commit or abort
2342+
*
2343+
* NB: this processing must be idempotent, because EOXactListAdd() doesn't
2344+
* bother to prevent duplicate entries in eoxact_list[].
2345+
*/
2346+
staticvoid
2347+
AtEOXact_cleanup(Relationrelation,boolisCommit)
2348+
{
23032349
/*
23042350
* The relcache entry's ref count should be back to its normal
23052351
* not-in-a-transaction state: 0 unless it's nailed in cache.
23062352
*
23072353
* In bootstrap mode, this is NOT true, so don't check it --- the
23082354
* bootstrap code expects relations to stay open across start/commit
23092355
* transaction calls. (That seems bogus, but it's not worth fixing.)
2356+
*
2357+
* Note: ideally this check would be applied to every relcache entry,
2358+
* not just those that have eoxact work to do.But it's not worth
2359+
* forcing a scan of the whole relcache just for this.(Moreover,
2360+
* doing so would mean that assert-enabled testing never tests the
2361+
* hash_search code path above, which seems a bad idea.)
23102362
*/
23112363
#ifdefUSE_ASSERT_CHECKING
23122364
if (!IsBootstrapProcessingMode())
@@ -2335,7 +2387,7 @@ AtEOXact_RelationCache(bool isCommit)
23352387
else
23362388
{
23372389
RelationClearRelation(relation, false);
2338-
continue;
2390+
return;
23392391
}
23402392
}
23412393

@@ -2354,10 +2406,6 @@ AtEOXact_RelationCache(bool isCommit)
23542406
relation->rd_oidindex=InvalidOid;
23552407
relation->rd_indexvalid=0;
23562408
}
2357-
}
2358-
2359-
/* Once done with the transaction, we can reset need_eoxact_work */
2360-
need_eoxact_work= false;
23612409
}
23622410

23632411
/*
@@ -2373,20 +2421,51 @@ AtEOSubXact_RelationCache(bool isCommit, SubTransactionId mySubid,
23732421
{
23742422
HASH_SEQ_STATUSstatus;
23752423
RelIdCacheEnt*idhentry;
2424+
inti;
23762425

23772426
/*
2378-
* Skip the relcache scan if nothing to do --- see notes for
2379-
* AtEOXact_RelationCache.
2427+
* Unless the eoxact_list[] overflowed, we only need to examine the rels
2428+
* listed in it. Otherwise fall back on a hash_seq_search scan. Same
2429+
* logic as in AtEOXact_RelationCache.
23802430
*/
2381-
if (!need_eoxact_work)
2382-
return;
2383-
2384-
hash_seq_init(&status,RelationIdCache);
2385-
2386-
while ((idhentry= (RelIdCacheEnt*)hash_seq_search(&status))!=NULL)
2431+
if (eoxact_list_overflowed)
23872432
{
2388-
Relationrelation=idhentry->reldesc;
2433+
hash_seq_init(&status,RelationIdCache);
2434+
while ((idhentry= (RelIdCacheEnt*)hash_seq_search(&status))!=NULL)
2435+
{
2436+
AtEOSubXact_cleanup(idhentry->reldesc,isCommit,
2437+
mySubid,parentSubid);
2438+
}
2439+
}
2440+
else
2441+
{
2442+
for (i=0;i<eoxact_list_len;i++)
2443+
{
2444+
idhentry= (RelIdCacheEnt*)hash_search(RelationIdCache,
2445+
(void*)&eoxact_list[i],
2446+
HASH_FIND,
2447+
NULL);
2448+
if (idhentry!=NULL)
2449+
AtEOSubXact_cleanup(idhentry->reldesc,isCommit,
2450+
mySubid,parentSubid);
2451+
}
2452+
}
23892453

2454+
/* Don't reset the list; we still need more cleanup later */
2455+
}
2456+
2457+
/*
2458+
* AtEOSubXact_cleanup
2459+
*
2460+
*Clean up a single rel at subtransaction commit or abort
2461+
*
2462+
* NB: this processing must be idempotent, because EOXactListAdd() doesn't
2463+
* bother to prevent duplicate entries in eoxact_list[].
2464+
*/
2465+
staticvoid
2466+
AtEOSubXact_cleanup(Relationrelation,boolisCommit,
2467+
SubTransactionIdmySubid,SubTransactionIdparentSubid)
2468+
{
23902469
/*
23912470
* Is it a relation created in the current subtransaction?
23922471
*
@@ -2400,7 +2479,7 @@ AtEOSubXact_RelationCache(bool isCommit, SubTransactionId mySubid,
24002479
else
24012480
{
24022481
RelationClearRelation(relation, false);
2403-
continue;
2482+
return;
24042483
}
24052484
}
24062485

@@ -2426,7 +2505,6 @@ AtEOSubXact_RelationCache(bool isCommit, SubTransactionId mySubid,
24262505
relation->rd_oidindex=InvalidOid;
24272506
relation->rd_indexvalid=0;
24282507
}
2429-
}
24302508
}
24312509

24322510

@@ -2516,9 +2594,6 @@ RelationBuildLocalRelation(const char *relname,
25162594
rel->rd_createSubid=GetCurrentSubTransactionId();
25172595
rel->rd_newRelfilenodeSubid=InvalidSubTransactionId;
25182596

2519-
/* must flag that we have rels created in this transaction */
2520-
need_eoxact_work= true;
2521-
25222597
/*
25232598
* create a new tuple descriptor from the one passed in. We do this
25242599
* partly to copy it into the cache context, and partly because the new
@@ -2609,6 +2684,12 @@ RelationBuildLocalRelation(const char *relname,
26092684
*/
26102685
RelationCacheInsert(rel);
26112686

2687+
/*
2688+
* Flag relation as needing eoxact cleanup (to clear rd_createSubid).
2689+
* We can't do this before storing relid in it.
2690+
*/
2691+
EOXactListAdd(rel);
2692+
26122693
/*
26132694
* done building relcache entry.
26142695
*/
@@ -2732,8 +2813,9 @@ RelationSetNewRelfilenode(Relation relation, TransactionId freezeXid)
27322813
* operations on the rel in the same transaction.
27332814
*/
27342815
relation->rd_newRelfilenodeSubid=GetCurrentSubTransactionId();
2735-
/* ... and now we have eoxact cleanup work to do */
2736-
need_eoxact_work= true;
2816+
2817+
/* Flag relation as needing eoxact cleanup (to remove the hint) */
2818+
EOXactListAdd(relation);
27372819
}
27382820

27392821

@@ -3513,8 +3595,8 @@ RelationSetIndexList(Relation relation, List *indexIds, Oid oidIndex)
35133595
relation->rd_indexlist=indexIds;
35143596
relation->rd_oidindex=oidIndex;
35153597
relation->rd_indexvalid=2;/* mark list as forced */
3516-
/*must flag that we have a forced indexlist */
3517-
need_eoxact_work= true;
3598+
/*Flag relation as needing eoxact cleanup (to reset thelist) */
3599+
EOXactListAdd(relation);
35183600
}
35193601

35203602
/*

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp