NotificationsYou must be signed in to change notification settings
Fork4.9k
Star17.7k

Commitaaf2661

committed

Fix backslash-escaping multibyte chars in COPY FROM.

If a multi-byte character is escaped with a backslash in TEXT mode input,and the encoding is one of the client-only encodings where the bytes afterthe first one can have an ASCII byte "embedded" in the char, we didn'tskip the character correctly. After a backslash, we only skipped the firstbyte of the next character, so if it was a multi-byte character, we wouldtry to process its second byte as if it was a separate character. If itwas one of the characters with special meaning, like '\n', '\r', oranother '\\', that would cause trouble.One such exmple is the byte sequence '\x5ca45c2e666f6f' in Big5 encoding.That's supposed to be [backslash][two-byte character][.][f][o][o], butbecause the second byte of the two-byte character is 0x5c, we incorrectlytreat it as another backslash. And because the next character is a dot, weparse it as end-of-copy marker, and throw an "end-of-copy marker corrupt"error.Backpatch to all supported versions.Reviewed-by: John Naylor, Kyotaro HoriguchiDiscussion:https://www.postgresql.org/message-id/a897f84f-8dca-8798-3139-07da5bb38728%40iki.fi

1 parentf0f9ed6 commitaaf2661Copy full SHA for aaf2661

File tree

1 file changed

-1

lines changed

src/backend/commands
- copy.c

1 file changed

-1

lines changed

`‎src/backend/commands/copy.c`

Lines changed: 9 additions & 1 deletion

Original file line number	Diff line number	Diff line change
`@@ -3615,7 +3615,7 @@ CopyReadLineText(CopyState cstate)`
`3615`	`3615`	`break;`
`3616`	`3616`	`}`
`3617`	`3617`	`elseif (!cstate->csv_mode)`
`3618`		`-`
	`3618`	`+{`
`3619`	`3619`	`/*`
`3620`	`3620`	`* If we are here, it means we found a backslash followed by`
`3621`	`3621`	`* something other than a period. In non-CSV mode, anything`
`@@ -3626,8 +3626,16 @@ CopyReadLineText(CopyState cstate)`
`3626`	`3626`	`* backslashes are not special, so we want to process the`
`3627`	`3627`	`* character after the backslash just like a normal character,`
`3628`	`3628`	`* so we don't increment in those cases.`
	`3629`	`+ *`
	`3630`	`+ * Set 'c' to skip whole character correctly in multi-byte`
	`3631`	`+ * encodings. If we don't have the whole character in the`
	`3632`	`+ * buffer yet, we might loop back to process it, after all,`
	`3633`	`+ * but that's OK because multi-byte characters cannot have any`
	`3634`	`+ * special meaning.`
`3629`	`3635`	`*/`
`3630`	`3636`	`raw_buf_ptr++;`
	`3637`	`+c=c2;`
	`3638`	`+}`
`3631`	`3639`	`}`
`3632`	`3640`
`3633`	`3641`	`/*`

0 commit comments

Comments

(0)

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Commitaaf2661

File tree

1 file changed

1 file changed

`‎src/backend/commands/copy.c`

0 commit comments