Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commitb24e6ca

Browse files
committed
Fix race condition with toast table access from a stale syscache entry.
If a tuple in a syscache contains an out-of-line toasted field, and wetry to fetch that field shortly after some other transaction has committedan update or deletion of the tuple, there is a race condition: vacuumcould come along and remove the toast tuples before we can fetch them.This leads to transient failures like "missing chunk number 0 for toastvalue NNNNN in pg_toast_2619", as seen in recent reports from AndrewHammond and Tim Uckun.The design idea of syscache is that access to stale syscache entriesshould be prevented by relation-level locks, but that fails for at leasttwo cases where toasted fields are possible: ANALYZE updates pg_statisticrows without locking out sessions that might want to plan queries on thesame table, and CREATE OR REPLACE FUNCTION updates pg_proc rows withoutany meaningful lock at all.The least risky fix seems to be an idea that Heikki suggested when wewere dealing with a related problem back in August: forcibly detoast anyout-of-line fields before putting a tuple into syscache in the first place.This avoids the problem because at the time we fetch the parent tuple fromthe catalog, we should be holding an MVCC snapshot that will preventremoval of the toast tuples, even if the parent tuple is outdatedimmediately after we fetch it. (Note: I'm not convinced that thisstatement holds true at every instant where we could be fetching a syscacheentry at all, but it does appear to hold true at the times where we couldfetch an entry that could have a toasted field. We will need to be a bitwary of adding toast tables to low-level catalogs that don't have themalready.) An additional benefit is that subsequent uses of the syscacheentry should be faster, since they won't have to detoast the field.Back-patch to all supported versions. The problem is significantly harderto reproduce in pre-9.0 releases, because of their willingness to flushevery entry in a syscache whenever the underlying catalog is vacuumed(cf CatalogCacheFlushRelation); but there is still a window for trouble.
1 parent5b297de commitb24e6ca

File tree

3 files changed

+105
-1
lines changed

3 files changed

+105
-1
lines changed

‎src/backend/access/heap/tuptoaster.c

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -809,6 +809,84 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup)
809809
}
810810

811811

812+
/* ----------
813+
* toast_flatten_tuple -
814+
*
815+
*"Flatten" a tuple to contain no out-of-line toasted fields.
816+
*(This does not eliminate compressed or short-header datums.)
817+
* ----------
818+
*/
819+
HeapTuple
820+
toast_flatten_tuple(HeapTupletup,TupleDesctupleDesc)
821+
{
822+
HeapTuplenew_tuple;
823+
Form_pg_attribute*att=tupleDesc->attrs;
824+
intnumAttrs=tupleDesc->natts;
825+
inti;
826+
Datumtoast_values[MaxTupleAttributeNumber];
827+
booltoast_isnull[MaxTupleAttributeNumber];
828+
booltoast_free[MaxTupleAttributeNumber];
829+
830+
/*
831+
* Break down the tuple into fields.
832+
*/
833+
Assert(numAttrs <=MaxTupleAttributeNumber);
834+
heap_deform_tuple(tup,tupleDesc,toast_values,toast_isnull);
835+
836+
memset(toast_free,0,numAttrs*sizeof(bool));
837+
838+
for (i=0;i<numAttrs;i++)
839+
{
840+
/*
841+
* Look at non-null varlena attributes
842+
*/
843+
if (!toast_isnull[i]&&att[i]->attlen==-1)
844+
{
845+
varattrib*new_value;
846+
847+
new_value= (varattrib*)DatumGetPointer(toast_values[i]);
848+
if (VARATT_IS_EXTERNAL(new_value))
849+
{
850+
new_value=toast_fetch_datum(new_value);
851+
toast_values[i]=PointerGetDatum(new_value);
852+
toast_free[i]= true;
853+
}
854+
}
855+
}
856+
857+
/*
858+
* Form the reconfigured tuple.
859+
*/
860+
new_tuple=heap_form_tuple(tupleDesc,toast_values,toast_isnull);
861+
862+
/*
863+
* Be sure to copy the tuple's OID and identity fields. We also make a
864+
* point of copying visibility info, just in case anybody looks at those
865+
* fields in a syscache entry.
866+
*/
867+
if (tupleDesc->tdhasoid)
868+
HeapTupleSetOid(new_tuple,HeapTupleGetOid(tup));
869+
870+
new_tuple->t_self=tup->t_self;
871+
new_tuple->t_tableOid=tup->t_tableOid;
872+
873+
new_tuple->t_data->t_choice=tup->t_data->t_choice;
874+
new_tuple->t_data->t_ctid=tup->t_data->t_ctid;
875+
new_tuple->t_data->t_infomask &= ~HEAP_XACT_MASK;
876+
new_tuple->t_data->t_infomask |=
877+
tup->t_data->t_infomask&HEAP_XACT_MASK;
878+
879+
/*
880+
* Free allocated temp values
881+
*/
882+
for (i=0;i<numAttrs;i++)
883+
if (toast_free[i])
884+
pfree(DatumGetPointer(toast_values[i]));
885+
886+
returnnew_tuple;
887+
}
888+
889+
812890
/* ----------
813891
* toast_flatten_tuple_attribute -
814892
*

‎src/backend/utils/cache/catcache.c

Lines changed: 18 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@
1717
#include"access/genam.h"
1818
#include"access/hash.h"
1919
#include"access/heapam.h"
20+
#include"access/tuptoaster.h"
2021
#include"access/valid.h"
2122
#include"catalog/pg_operator.h"
2223
#include"catalog/pg_type.h"
@@ -1634,16 +1635,32 @@ CatalogCacheCreateEntry(CatCache *cache, HeapTuple ntp,
16341635
uint32hashValue,IndexhashIndex,boolnegative)
16351636
{
16361637
CatCTup*ct;
1638+
HeapTupledtp;
16371639
MemoryContextoldcxt;
16381640

1641+
/*
1642+
* If there are any out-of-line toasted fields in the tuple, expand them
1643+
* in-line. This saves cycles during later use of the catcache entry,
1644+
* and also protects us against the possibility of the toast tuples being
1645+
* freed before we attempt to fetch them, in case of something using a
1646+
* slightly stale catcache entry.
1647+
*/
1648+
if (HeapTupleHasExternal(ntp))
1649+
dtp=toast_flatten_tuple(ntp,cache->cc_tupdesc);
1650+
else
1651+
dtp=ntp;
1652+
16391653
/*
16401654
* Allocate CatCTup header in cache memory, and copy the tuple there too.
16411655
*/
16421656
oldcxt=MemoryContextSwitchTo(CacheMemoryContext);
16431657
ct= (CatCTup*)palloc(sizeof(CatCTup));
1644-
heap_copytuple_with_tuple(ntp,&ct->tuple);
1658+
heap_copytuple_with_tuple(dtp,&ct->tuple);
16451659
MemoryContextSwitchTo(oldcxt);
16461660

1661+
if (dtp!=ntp)
1662+
heap_freetuple(dtp);
1663+
16471664
/*
16481665
* Finish initializing the CatCTup header, and add it to the cache's
16491666
* linked list and counts.

‎src/include/access/tuptoaster.h

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,15 @@ extern varattrib *heap_tuple_untoast_attr_slice(varattrib *attr,
112112
int32sliceoffset,
113113
int32slicelength);
114114

115+
/* ----------
116+
* toast_flatten_tuple -
117+
*
118+
*"Flatten" a tuple to contain no out-of-line toasted fields.
119+
*(This does not eliminate compressed or short-header datums.)
120+
* ----------
121+
*/
122+
externHeapTupletoast_flatten_tuple(HeapTupletup,TupleDesctupleDesc);
123+
115124
/* ----------
116125
* toast_flatten_tuple_attribute -
117126
*

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp