Movatterモバイル変換

[0]ホーム
URL:

Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

 Sign up
Appearance settings

Commit0319443

Browse files
committed
I attach a version of my toast-slicing patch, against current CVS
(current as of a few hours ago.)This patch:1. Adds PG_GETARG_xxx_P_SLICE() macros and associated support routines.2. Adds routines in src/backend/access/tuptoaster.c for fetching onlynecessary chunks of a toasted value. (Modelled on latest changes toassume chunks are returned in order).3. Amends text_substr and bytea_substr to use new methods. It nowhandles multibyte cases -and should still lead to a performanceimprovement in the multibyte case where the substring is near thebeginning of the string.4. Added new command: ALTER TABLE tabname ALTER COLUMN colname SETSTORAGE {PLAIN | EXTERNAL | EXTENDED | MAIN} to parser and documented inalter-table.sgml. (NB I used ColId as the item type for the storagemode string, rather than a new production - I hope this makes sense!).All this does is sets attstorage for the specified column.4. AlterTableAlterColumnStatistics is now AlterTableAlterColumnFlags andhandles both statistics and storage (it uses the subtype code todistinguish). The previous version of my patch also re-arranged othercode in backend/commands/command.c but I have dropped that from thispatch.(I plan to return to it separately).5. Documented new macros (and also the PG_GETARG_xxx_P_COPY macros) inxfunc.sgml. ref/alter_table.sgml also contains documentation for ALTERCOLUMN SET STORAGE.John Gray
1 parent276fc7c commit0319443

File tree

13 files changed

+498
-79
lines changed

13 files changed

+498
-79
lines changed

‎doc/src/sgml/ref/alter_table.sgml

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!--
2-
$Header: /cvsroot/pgsql/doc/src/sgml/ref/alter_table.sgml,v 1.38 2002/02/17 13:29:00 momjian Exp $
2+
$Header: /cvsroot/pgsql/doc/src/sgml/ref/alter_table.sgml,v 1.39 2002/03/05 05:33:04 momjian Exp $
33
PostgreSQL documentation
44
-->
55

@@ -30,6 +30,8 @@ ALTER TABLE [ ONLY ] <replaceable class="PARAMETER">table</replaceable> [ * ]
3030
class="PARAMETER">value</replaceable> | DROP DEFAULT }
3131
ALTER TABLE [ ONLY ] <replaceable class="PARAMETER">table</replaceable> [ * ]
3232
ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> SET STATISTICS <replaceable class="PARAMETER">integer</replaceable>
33+
ALTER TABLE [ ONLY ] <replaceable class="PARAMETER">table</replaceable> [ * ]
34+
ALTER [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> SET STORAGE {PLAIN | EXTERNAL | EXTENDED | MAIN}
3335
ALTER TABLE [ ONLY ] <replaceable class="PARAMETER">table</replaceable> [ * ]
3436
RENAME [ COLUMN ] <replaceable class="PARAMETER">column</replaceable> TO <replaceable
3537
class="PARAMETER">newcolumn</replaceable>
@@ -169,6 +171,17 @@ ALTER TABLE <replaceable class="PARAMETER">table</replaceable>
169171
The <literal>ALTER COLUMN SET STATISTICS</literal> form allows you to
170172
set the statistics-gathering target for subsequent
171173
<xref linkend="sql-analyze" endterm="sql-analyze-title"> operations.
174+
The <literal>ALTER COLUMN SET STORAGE</literal> form allows the
175+
column storage mode to be set. This controls whether this column is
176+
held inline or in a supplementary table, and whether the data
177+
should be compressed or not. <literal>PLAIN</literal> must be used
178+
for fixed-length values such as <literal>INTEGER</literal> and is
179+
inline, uncompressed. <literal>MAIN</literal> is for inline,
180+
compressible data. <literal>EXTERNAL</literal> is for external,
181+
uncompressed data and <literal>EXTENDED</literal> is for external,
182+
compressed data. The use of <literal>EXTERNAL</literal> will make
183+
substring operations on a column faster, at the penalty of
184+
increased storage space.
172185
The <literal>RENAME</literal> clause causes the name of a table,
173186
column, index, or sequence to change without changing any of the
174187
data. The data will remain of the same type and size after the

‎doc/src/sgml/xfunc.sgml

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
<!--
2-
$Header: /cvsroot/pgsql/doc/src/sgml/xfunc.sgml,v 1.47 2002/01/20 22:19:56 petere Exp $
2+
$Header: /cvsroot/pgsql/doc/src/sgml/xfunc.sgml,v 1.48 2002/03/05 05:33:00 momjian Exp $
33
-->
44

55
<chapter id="xfunc">
@@ -1296,6 +1296,35 @@ concat_text(PG_FUNCTION_ARGS)
12961296
this works in both strict and nonstrict functions.
12971297
</para>
12981298

1299+
<para>
1300+
Other options provided in the new-style interface are two
1301+
variants of the
1302+
<function>PG_GETARG_<replaceable>xxx</replaceable>()</function>
1303+
macros. The first of these,
1304+
<function>PG_GETARG_<replaceable>xxx</replaceable>_COPY()</function>
1305+
guarantees to return a copy of the specified parameter which is
1306+
safe for writing into. (The normal macros will sometimes return a
1307+
pointer to the value which must not be written to. Using the
1308+
<function>PG_GETARG_<replaceable>xxx</replaceable>_COPY()</function>
1309+
macros guarantees a writable result.)
1310+
</para>
1311+
1312+
<para>
1313+
The second variant consists of the
1314+
<function>PG_GETARG_<replaceable>xxx</replaceable>_SLICE()</function>
1315+
macros which take three parameters. The first is the number of the
1316+
parameter (as above). The second and third are the offset and
1317+
length of the segment to be returned. Offsets are counted from
1318+
zero, and a negative length requests that the remainder of the
1319+
value be returned. These routines provide more efficient access to
1320+
parts of large values in the case where they have storage type
1321+
"external". (The storage type of a column can be specified using
1322+
<command>ALTER TABLE <repaceable>tablename</replaceable> ALTER
1323+
COLUMN <replaceable>colname</replaceable> SET STORAGE
1324+
<replaceable>storagetype</replaceable>. Storage type is one of
1325+
plain, external, extended or main.)
1326+
</para>
1327+
12991328
<para>
13001329
The version-1 function call conventions make it possible to
13011330
return <quote>set</quote> results and implement trigger functions and

‎src/backend/access/heap/tuptoaster.c

Lines changed: 272 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
*
99
*
1010
* IDENTIFICATION
11-
* $Header: /cvsroot/pgsql/src/backend/access/heap/tuptoaster.c,v 1.27 2002/01/16 20:29:01 tgl Exp $
11+
* $Header: /cvsroot/pgsql/src/backend/access/heap/tuptoaster.c,v 1.28 2002/03/05 05:33:06 momjian Exp $
1212
*
1313
*
1414
* INTERFACE ROUTINES
@@ -47,6 +47,8 @@ static void toast_insert_or_update(Relation rel, HeapTuple newtup,
4747
HeapTupleoldtup);
4848
staticDatumtoast_save_datum(Relationrel,Datumvalue);
4949
staticvarattrib*toast_fetch_datum(varattrib*attr);
50+
staticvarattrib*toast_fetch_datum_slice(varattrib*attr,
51+
int32sliceoffset,int32length);
5052

5153

5254
/* ----------
@@ -162,6 +164,80 @@ heap_tuple_untoast_attr(varattrib *attr)
162164
}
163165

164166

167+
/* ----------
168+
* heap_tuple_untoast_attr_slice -
169+
*
170+
* Public entry point to get back part of a toasted value
171+
* from compression or external storage.
172+
* ----------
173+
*/
174+
varattrib*
175+
heap_tuple_untoast_attr_slice(varattrib*attr,int32sliceoffset,int32slicelength)
176+
{
177+
varattrib*preslice;
178+
varattrib*result;
179+
int32attrsize;
180+
181+
if (VARATT_IS_COMPRESSED(attr))
182+
{
183+
varattrib*tmp;
184+
185+
if (VARATT_IS_EXTERNAL(attr))
186+
{
187+
tmp=toast_fetch_datum(attr);
188+
}
189+
else
190+
{
191+
tmp=attr;/* compressed in main tuple */
192+
}
193+
194+
preslice= (varattrib*)palloc(attr->va_content.va_external.va_rawsize
195+
+VARHDRSZ);
196+
VARATT_SIZEP(preslice)=attr->va_content.va_external.va_rawsize+VARHDRSZ;
197+
pglz_decompress((PGLZ_Header*)tmp,VARATT_DATA(preslice));
198+
199+
if (tmp!=attr)
200+
pfree(tmp);
201+
}
202+
else
203+
{
204+
/* Plain value */
205+
if (VARATT_IS_EXTERNAL(attr))
206+
{
207+
/* fast path */
208+
return (toast_fetch_datum_slice(attr,sliceoffset,slicelength));
209+
}
210+
else
211+
{
212+
preslice=attr;
213+
}
214+
}
215+
216+
/* slicing of datum for compressed cases and plain value */
217+
218+
attrsize=VARSIZE(preslice)-VARHDRSZ;
219+
if (sliceoffset >=attrsize)
220+
{
221+
sliceoffset=0;
222+
slicelength=0;
223+
}
224+
225+
if (((sliceoffset+slicelength)>attrsize)||slicelength<0)
226+
{
227+
slicelength=attrsize-sliceoffset;
228+
}
229+
230+
result= (varattrib*)palloc(slicelength+VARHDRSZ);
231+
VARATT_SIZEP(result)=slicelength+VARHDRSZ;
232+
233+
memcpy(VARDATA(result),VARDATA(preslice)+sliceoffset,slicelength);
234+
235+
if (preslice!=attr)pfree(preslice);
236+
237+
returnresult;
238+
}
239+
240+
165241
/* ----------
166242
* toast_raw_datum_size -
167243
*
@@ -981,7 +1057,7 @@ toast_fetch_datum(varattrib *attr)
9811057
VARATT_SIZEP(result) |=VARATT_FLAG_COMPRESSED;
9821058

9831059
/*
984-
* Open the toast relation andit's index
1060+
* Open the toast relation andits index
9851061
*/
9861062
toastrel=heap_open(attr->va_content.va_external.va_toastrelid,
9871063
AccessShareLock);
@@ -1081,4 +1157,198 @@ toast_fetch_datum(varattrib *attr)
10811157
returnresult;
10821158
}
10831159

1160+
/* ----------
1161+
* toast_fetch_datum_slice -
1162+
*
1163+
*Reconstruct a segment of a varattrib from the chunks saved
1164+
*in the toast relation
1165+
* ----------
1166+
*/
1167+
staticvarattrib*
1168+
toast_fetch_datum_slice(varattrib*attr,int32sliceoffset,int32length)
1169+
{
1170+
Relationtoastrel;
1171+
Relationtoastidx;
1172+
ScanKeyDatatoastkey[3];
1173+
IndexScanDesctoastscan;
1174+
HeapTupleDatatoasttup;
1175+
HeapTuplettup;
1176+
TupleDesctoasttupDesc;
1177+
RetrieveIndexResultindexRes;
1178+
Bufferbuffer;
1179+
1180+
varattrib*result;
1181+
int32attrsize;
1182+
int32nscankeys;
1183+
int32residx;
1184+
int32nextidx;
1185+
intnumchunks;
1186+
intstartchunk;
1187+
intendchunk;
1188+
int32startoffset;
1189+
int32endoffset;
1190+
inttotalchunks;
1191+
Pointerchunk;
1192+
boolisnull;
1193+
int32chunksize;
1194+
int32chcpystrt;
1195+
int32chcpyend;
1196+
1197+
attrsize=attr->va_content.va_external.va_extsize;
1198+
totalchunks= ((attrsize-1) /TOAST_MAX_CHUNK_SIZE)+1;
1199+
1200+
if (sliceoffset >=attrsize)
1201+
{
1202+
sliceoffset=0;
1203+
length=0;
1204+
}
1205+
1206+
if (((sliceoffset+length)>attrsize)||length<0)
1207+
{
1208+
length=attrsize-sliceoffset;
1209+
}
1210+
1211+
result= (varattrib*)palloc(length+VARHDRSZ);
1212+
VARATT_SIZEP(result)=length+VARHDRSZ;
1213+
1214+
if (VARATT_IS_COMPRESSED(attr))
1215+
VARATT_SIZEP(result) |=VARATT_FLAG_COMPRESSED;
1216+
1217+
if (length==0)return (result);/* Can save a lot of work at this point! */
1218+
1219+
startchunk=sliceoffset /TOAST_MAX_CHUNK_SIZE;
1220+
endchunk= (sliceoffset+length-1) /TOAST_MAX_CHUNK_SIZE;
1221+
numchunks= (endchunk-startchunk )+1;
1222+
1223+
startoffset=sliceoffset %TOAST_MAX_CHUNK_SIZE;
1224+
endoffset= (sliceoffset+length-1) %TOAST_MAX_CHUNK_SIZE;
1225+
1226+
/*
1227+
* Open the toast relation and it's index
1228+
*/
1229+
toastrel=heap_open(attr->va_content.va_external.va_toastrelid,
1230+
AccessShareLock);
1231+
toasttupDesc=toastrel->rd_att;
1232+
toastidx=index_open(toastrel->rd_rel->reltoastidxid);
1233+
1234+
/*
1235+
* Setup a scan key to fetch from the index. This is either two keys
1236+
* or three depending on the number of chunks.
1237+
*/
1238+
ScanKeyEntryInitialize(&toastkey[0],
1239+
(bits16)0,
1240+
(AttrNumber)1,
1241+
(RegProcedure)F_OIDEQ,
1242+
ObjectIdGetDatum(attr->va_content.va_external.va_valueid));
1243+
/*
1244+
* Now dependent on number of chunks:
1245+
*/
1246+
1247+
if (numchunks==1)
1248+
{
1249+
ScanKeyEntryInitialize(&toastkey[1],
1250+
(bits16)0,
1251+
(AttrNumber)2,
1252+
(RegProcedure)F_INT4EQ,
1253+
Int32GetDatum(startchunk));
1254+
nscankeys=2;
1255+
}
1256+
else
1257+
{
1258+
ScanKeyEntryInitialize(&toastkey[1],
1259+
(bits16)0,
1260+
(AttrNumber)2,
1261+
(RegProcedure)F_INT4GE,
1262+
Int32GetDatum(startchunk));
1263+
ScanKeyEntryInitialize(&toastkey[2],
1264+
(bits16)0,
1265+
(AttrNumber)2,
1266+
(RegProcedure)F_INT4LE,
1267+
Int32GetDatum(endchunk));
1268+
nscankeys=3;
1269+
}
1270+
1271+
/*
1272+
* Read the chunks by index
1273+
*
1274+
* The index is on (valueid, chunkidx) so they will come in order
1275+
*/
1276+
nextidx=startchunk;
1277+
toastscan=index_beginscan(toastidx, false,nscankeys,&toastkey[0]);
1278+
while ((indexRes=index_getnext(toastscan,ForwardScanDirection))!=NULL)
1279+
{
1280+
toasttup.t_self=indexRes->heap_iptr;
1281+
heap_fetch(toastrel,SnapshotToast,&toasttup,&buffer,toastscan);
1282+
pfree(indexRes);
1283+
1284+
if (toasttup.t_data==NULL)
1285+
continue;
1286+
ttup=&toasttup;
1287+
1288+
/*
1289+
* Have a chunk, extract the sequence number and the data
1290+
*/
1291+
residx=DatumGetInt32(heap_getattr(ttup,2,toasttupDesc,&isnull));
1292+
Assert(!isnull);
1293+
chunk=DatumGetPointer(heap_getattr(ttup,3,toasttupDesc,&isnull));
1294+
Assert(!isnull);
1295+
chunksize=VARATT_SIZE(chunk)-VARHDRSZ;
1296+
1297+
/*
1298+
* Some checks on the data we've found
1299+
*/
1300+
if ((residx!=nextidx)|| (residx>endchunk)|| (residx<startchunk))
1301+
elog(ERROR,"unexpected chunk number %d (expected %d) for toast value %u",
1302+
residx,nextidx,
1303+
attr->va_content.va_external.va_valueid);
1304+
if (residx<totalchunks-1)
1305+
{
1306+
if (chunksize!=TOAST_MAX_CHUNK_SIZE)
1307+
elog(ERROR,"unexpected chunk size %d in chunk %d for toast value %u",
1308+
chunksize,residx,
1309+
attr->va_content.va_external.va_valueid);
1310+
}
1311+
else
1312+
{
1313+
if ((residx*TOAST_MAX_CHUNK_SIZE+chunksize)!=attrsize)
1314+
elog(ERROR,"unexpected chunk size %d in chunk %d for toast value %u",
1315+
chunksize,residx,
1316+
attr->va_content.va_external.va_valueid);
1317+
}
1318+
1319+
/*
1320+
* Copy the data into proper place in our result
1321+
*/
1322+
chcpystrt=0;
1323+
chcpyend=chunksize-1;
1324+
if (residx==startchunk)chcpystrt=startoffset;
1325+
if (residx==endchunk)chcpyend=endoffset;
1326+
1327+
memcpy(((char*)VARATT_DATA(result))+
1328+
(residx*TOAST_MAX_CHUNK_SIZE-sliceoffset)+chcpystrt,
1329+
VARATT_DATA(chunk)+chcpystrt,
1330+
(chcpyend-chcpystrt)+1);
1331+
1332+
ReleaseBuffer(buffer);
1333+
nextidx++;
1334+
}
1335+
1336+
/*
1337+
* Final checks that we successfully fetched the datum
1338+
*/
1339+
if (nextidx!= (endchunk+1))
1340+
elog(ERROR,"missing chunk number %d for toast value %u",
1341+
nextidx,
1342+
attr->va_content.va_external.va_valueid);
1343+
1344+
/*
1345+
* End scan and close relations
1346+
*/
1347+
index_endscan(toastscan);
1348+
index_close(toastidx);
1349+
heap_close(toastrel,AccessShareLock);
1350+
1351+
returnresult;
1352+
}
1353+
10841354
#endif/* TUPLE_TOASTER_ACTIVE */

0 commit comments

Comments
 (0)
[8]ページ先頭
©2009-2025 Movatter.jp