Movatterモバイル変換


[0]ホーム

URL:


Skip to content

Navigation Menu

Sign in
Appearance settings

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

Sign up
Appearance settings

[HttpFoundation] fix PDO session handler under high concurrency#10652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Merged
fabpot merged 3 commits intosymfony:2.3fromTobion:fix-pdo-session-concurrency
Apr 11, 2014

Conversation

@Tobion
Copy link
Contributor

QA
Bug fix?yes
New feature?no
BC breaks?no
Deprecations?no
Tests pass?yes
Fixed tickets#8448 andhttp://trac.symfony-project.org/ticket/4777 (which was never really fixed as you can see here)
LicenseMIT

What I find rather bad with the class design is that it depends on the table definition, but it's not part of the class. Also it doesn't make use of open() and close() which could be used to make the database connection lazy instead of having is open all the time when not needed. Doctrine also only lazy connects, but we use PDO directly here.
Furthermore, the session handlers should not throw exceptions, from what I read, but return false when an error occurs. This is not followed in this class. Maybe@Drak knows how php session management behaves when the session handlers return false?

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

This was one main problem: It creates duplicate keys when session created meanwhile (between select and insert). There is no need to create an entry in read() at all.

Copy link
ContributorAuthor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others.Learn more.

For oraclesysdate was used which is of type DATE. But we expect an INTEGER column, otherwisegc() would not work. So this didn't make sense.

@fabpot
Copy link
Member

@Tobion Thank you so much for working on this issue, this is much appreciated. Is it ready to be merged? Or do you also want to implement lazy connections as well in this PR?

@ghost
Copy link

@Tobion The PR looks good.

If I remember correctly you should not throw exceptions insidewrite() orclose() because they will never be caught. In general you should return bool where required as it tells the PHP engine (which calls these methods internally) if things worked correctly - no return is also OK since it evaluates to false anyhow. If the methods return false they do not trigger errors.read() evaluates false to an empty string. If a read() fails, it's taken to be a record not found.

Is that helpful?

@Tobion
Copy link
ContributorAuthor

@fabpot yes it's ready. Lazy connections would only be for master anyway.

So if we want to make it respect the SessionHandlerInterface we should not throw exceptions but return false. But then I wonder how to best warn developers if something went wrong. For example when a developer forgots to create the table and we just return false everywhere, the dev would have hard times debugging the problem. One solution could be to use a logger in the class and log errors instead of throwing exceptions, or we can usetrigger_error() which then depends how the error handler is configured.@Drak what do you think?

@ghost
Copy link

Oh gosh sorry I meant to say that in my post but forgot. I was going to suggest adding a logger or raise a PHP userspace error. Logger would be better IMO.

@Tobion
Copy link
ContributorAuthor

Warning As of PHP 5.0.5 the write and close handlers are called after object destruction and therefore cannot use objects or throw exceptions.
www.php.net/manual/en/function.session-set-save-handler.php

What does it mean they cannot use objects? PDO and the possible logger are objects as well.

@Tobion
Copy link
ContributorAuthor

Ok how I interpret it, is that objects are only destructed before write/close when you didn't call session_write_close() or session_register_shutdown() explicitly but relied on PHP to do it automatically at termination of the script (legacy style). So we should be fine.

fabpot added a commit that referenced this pull requestApr 11, 2014
…rency (Tobion)This PR was merged into the 2.3 branch.Discussion----------[HttpFoundation] fix PDO session handler under high concurrency| Q             | A| ------------- | ---| Bug fix?      | yes| New feature?  | no| BC breaks?    | no| Deprecations? | no| Tests pass?   | yes| Fixed tickets |#8448 andhttp://trac.symfony-project.org/ticket/4777 (which was never really fixed as you can see here)| License       | MIT- The first commit fixes PDO session handler under high concurrency.- The second commit uses MERGE SQL for MS SQL Server. Tested withhttp://sqlfiddle.com/#!6/66b6d/14- The third commit uses INSERT OR REPLACE for sqlite session handlerhttp://sqlfiddle.com/#!7/e6707/3What I find rather bad with the class design is that it depends on the table definition, but it's not part of the class. Also it doesn't make use of open() and close() which could be used to make the database connection lazy instead of having is open all the time when not needed. Doctrine also only lazy connects, but we use PDO directly here.Furthermore, the session handlers should not throw exceptions, from what I read, but return false when an error occurs. This is not followed in this class. Maybe@Drak knows how php session management behaves when the session handlers return false?Commits-------5c08e29 [HttpFoundation] use insert or replace for sqlite session handler05ea19a [HttpFoundation] use MERGE SQL for MS SQL Server session storagee58d7cf [HttpFoundation] fix PDO session handler under high concurrency
@fabpotfabpot merged commit5c08e29 intosymfony:2.3Apr 11, 2014
@TobionTobion deleted the fix-pdo-session-concurrency branchApril 11, 2014 18:33
@ghost
Copy link

@Tobion Yes, this is already handled by the symfony session code, which registers the shutdown handler assession_write_close() automatically

@ruudk
Copy link
Contributor

Just deployed this code to my server and seeing a lot of errors like this:

PHP Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction'

Am I the only one?

@Tobion
Copy link
ContributorAuthor

Can you give us more information please. Like which database are you using?

@Tobion
Copy link
ContributorAuthor

Also since this class only throws RuntimeExceptions, the PDOException you got probably comes from somewhere else. A problem could be that you use the same connection which now interferes. Maybe you leave a transaction open without commiting or rolling back.

@ruudk
Copy link
Contributor

@Tobion Sorry, I am using MySQL and I have a PDO instance only for sessions and use Doctrine2 for other queries (that has a separate connection).

@Tobion
Copy link
ContributorAuthor

The PdoSessionHandler throws\RuntimeException(sprintf('PDOException was thrown when trying to write the session data: %s', $e->getMessage()), 0, $e); which is not what you reported. So are you sure the source the the handler?

@ruudk
Copy link
Contributor

Hmm, could be coming from somewhere else then. I've switched to Redis Sessions. Thanks for helping.

fabpot added a commit that referenced this pull requestApr 18, 2014
This PR was merged into the 2.3 branch.Discussion----------[HttpFoundation] Fix DbalSessionHandler| Q             | A| ------------- | ---| Bug fix?      | yes| New feature?  | no| BC breaks?    | no| Deprecations? | no| Tests pass?   | yes| Fixed tickets | -| License       | MITThis is basically the same as#10652 for the DbalSessionHandler.- First commit fixes fix DbalSessionHandler for high concurrency, interface compliance, compatibility with all drivers (oci8, mysqli, pdo with mysql, sqlsrv, sqlite).- Second commit updates phpdoc of SessionHandlerInterface and unifies parameters of all handlers according to interface (so inheritdoc actually makes sense).Commits-------524bf84 [HttpFoundation] update phpdoc of SessionHandlerInterface and unify parameters of all handlers according to interfaceccdfbe6 [Doctrine Bridge] fix DbalSessionHandler for high concurrency, interface compliance, compatibility with all drivers (oci8, mysqli, pdo with mysql, sqlsrv, sqlite)
@onewheelskyward
Copy link

We just ran into this problem with postgres on 2.3.13, it looks like that database might need some attention. Has anyone else encountered this before I start down the path?

@Tobion
Copy link
ContributorAuthor

@onewheelskyward what problem did you encounter?

@onewheelskyward
Copy link

@Tobion As soon as we updated to 2.3.13, we got the duplicate session id errors. A revert to 2.3.12 fixed it for the time being.

Uncaught exception 'PDOException' with message 'SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint "sessions_pkey"DETAIL: Key (session_id)=(keychangedtoprotecttheinnocent) already exists.' in /path/to/app/code/vendor/symfony/symfony/src/Symfony/Component/HttpFoundation/Session/Storage/Handler/PdoSessionHandler.php:201

@Tobion
Copy link
ContributorAuthor

Hm I don't see how this is possible. Postgres uses a DELETE followed by INSERT inside a transaction. Seehttps://github.com/symfony/symfony/blob/2.3/src/Symfony/Component/HttpFoundation/Session/Storage/Handler/PdoSessionHandler.php#L189 So the INSERT normally cannot create duplicate key.
Are you sure you are using the above class? Also the exeption that is thrown should bePDOException was thrown when trying to write the session data: ...
Does the problem occur only sometimes or for every session?

@onewheelskyward
Copy link

I double checked the config, we are using that class. It seemed to be intermittent but we reverted as soon as we saw the errors. I'll try to recreate on our dev environment with a load test to be sure. Our PDO calls are set as follows, could that be why we got the statement exception instead of the in-function exception handling?

        - [setAttribute, [3, 2]] # \PDO::ATTR_ERRMODE, \PDO::ERRMODE_EXCEPTION

@tcql
Copy link

Just to follow up, I also am seeing the Unique constraint violation with Postgres 9.3.4 with Symfony 2.4+. I'm also pretty puzzled because a single transaction with a delete and then an update should precludeany possibility of this happening.

@onewheelskyward
Copy link

Now that's interesting. And I agree with you on the implementation of the
code. Must be something else happening. I haven't had time to go back and
investigate yet, please let me know if you find anything. We're using
postgres 9.3.4 as well.

On Tue, May 13, 2014 at 11:22 AM, tchannelnotifications@github.com wrote:

Just to follow up, I also am seeing the Unique constraint violation with
Postgres 9.3.4 with Symfony 2.4+. I'm also pretty puzzled because a single
transaction with a delete and then an update should preclude _any_possibility of this happening.


Reply to this email directly or view it on GitHubhttps://github.com//pull/10652#issuecomment-42992727
.

@tcql
Copy link

I'm poking around, looks like itcould have something to do with postgres' default transaction / locking setup, but I'm not positive. Theoretically, if two transactions try doing the same action at the same time, given Postgres' default READ COMMITTED transaction mode

T1T2
LOCK row for DELETE
DELETEsee row is LOCKed, waiting...
INSERTwaiting...
COMMITwaiting...
release LOCK
LOCK the row
DELETE
INSERT
COMMIT

But it seems more like the deletions and insertions of multiple transactions are overlapping somehow

EDIT: yep, related to locking / transaction isolation. running "LOCK table $this->table IN EXCLUSIVE MODE" at the beginning of the transaction fixes things. But a full table lock is very undesirable for concurrency / performance.

@tcql
Copy link

Okay, apologies for the annoying multiple posts. Here's what I came up with that fixes it for me:

https://gist.github.com/tchannel/391759d0551639ae0a31

@Tobion
Copy link
ContributorAuthor

Yes that's how it should be. Maybe your database is configured with a different transaction isolation level, i.e. Read uncommitted

@tcql
Copy link

@Tobion in postgres, READ COMMITTED and READ UNCOMMITTED are the same.In Theory READ COMMITTED should work appropriately, but it doesn't.

@Tobion
Copy link
ContributorAuthor

@tchannel can you then open a ticket on postgress or find out why it does not block. Have you tried to execute it manually through postgres command line using two connections? One should block then. Maybe it's a PDO problem.
Using Serializable is not a real solution. We need to use explicit locks anyway (which serializable does not do) to fix#4976
WOuld be good if you find the best way to block a row (existent or non-existent yet) in read() with postgres. For the other DBs I already have a solution in mind.

@tcql
Copy link

@Tobion oh very weird. Doing it manually with twopsql instances is VERY interesting. If I open both transactions, issue a delete in one, then issue a delete in the second, the second waits for the lock to release (as it should), so I can't enter anything else.

Then, I run the insert in the first transaction, and commit.

Now the second transaction releases because the lock is free. But... the delete count is ZERO. It's like committing the first transaction is somehow doing things as individual commands still... The delete lock is released before the insert is fully processed, and the second transaction runs it's delete before the first transaction's insert goes through.

So definitely not a PDO problem, it's an issue with postgres

@Tobion
Copy link
ContributorAuthor

Actually that behavior could be the one described for read commited:http://www.postgresql.org/docs/9.1/static/transaction-iso.html

The delete of the second transaction is just skipped because it didn't match before blocking and the delete is not re-evaluated for the whole table again.

This works where delete is executed after concurred insert, but your example above will indeed create duplicate key.

T1T2
LOCK row for DELETE
DELETE
INSERTsee row is LOCKed, waiting...
COMMITwaiting...
release LOCK
LOCK the row
DELETE
INSERT
COMMIT

@tcql
Copy link

eesh. That's... unexpected? I made a bug report / question on the postgres-bugs mailing list. I'll follow up on here when/if I hear anything

@Tobion
Copy link
ContributorAuthor

I think it's expected but just highly incomprehensible on first sight.
We need to find a fix, but more in regards to#4976 which comprises this one.

@Tobion
Copy link
ContributorAuthor

@onewheelskyward and @tchannel please have a look at#10908. Would be nice if you can test it.

@gheydon
Copy link

I have had a problem upgrade past 2.3.13. I get the following excetion thrown.

Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[07002]: [Microsoft][SQL Server Native Client 10.0]COUNT field incorrect or syntax error' in E:\Inetpub\test\api\vendor\symfony\symfony\src\Symfony\Component\HttpFoundation\Session\Storage\Handler\PdoSessionHandler.php:181Stack trace:#0 E:\Inetpub\test\api\vendor\symfony\symfony\src\Symfony\Component\HttpFoundation\Session\Storage\Handler\PdoSessionHandler.php(181): PDOStatement->execute()#1 E:\Inetpub\test\api\vendor\symfony\symfony\src\Symfony\Component\HttpFoundation\Session\Storage\Proxy\SessionHandlerProxy.php(77): Symfony\Component\HttpFoundation\Session\Storage\Handler\PdoSessionHandler->write('ifamr16dbjnalto...', '_sf2_attributes...')#2 [internal function]: Symfony\Component\HttpFoundation\Session\Storage\Proxy\SessionHandlerProxy->write('ifamr16dbjnalto...', '_sf2_attributes...')#3 [internal function]: session_write_close()#4 {main}

The server in question is running sqlsrv 2005.

@Tobion
Copy link
ContributorAuthor

@gheydon I think sqlsrv 2005 does not support the MERGE SQL statement used inhttps://github.com/symfony/symfony/blob/2.3/src/Symfony/Component/HttpFoundation/Session/Storage/Handler/PdoSessionHandler.php#L236 Seems to be available since 2008.
Can you please give me the output of$this->pdo->getAttribute(\PDO::ATTR_SERVER_VERSION) so I use the fallback for it?

@Tobion
Copy link
ContributorAuthor

@tchannel and@gheydon see#11009 for solutions to your problems.

fabpot added a commit that referenced this pull requestJun 4, 2014
This PR was merged into the 2.3 branch.Discussion----------[HttpFoundation] smaller fixes for PdoSessionHandler| Q             | A| ------------- | ---| Bug fix?      | yes| New feature?  | no| BC breaks?    | no| Deprecations? | no| Tests pass?   | yes| Fixed tickets |#10652| License       | MITFor both the PdoSessionHandler and DbalSessionHandler-#10652 (comment): Transactional DELETE + INSERT does not work as expected-#10652 (comment): sqlsrv 2005 does not support the MERGE SQL, and if used it requires an HOLDLOCK- missing time update for sqlsrv and oracleCommits-------a0e1d4d [Doctrine Bridge] fix DBAL session handler according to PdoSessionHandler00d707f [HttpFoundation] use different approach for duplicate keys in postgres, fix merge for sqlsrv and oracle
fabpot added a commit to symfony/http-foundation that referenced this pull requestJun 4, 2014
This PR was merged into the 2.3 branch.Discussion----------[HttpFoundation] smaller fixes for PdoSessionHandler| Q             | A| ------------- | ---| Bug fix?      | yes| New feature?  | no| BC breaks?    | no| Deprecations? | no| Tests pass?   | yes| Fixed tickets | #10652| License       | MITFor both the PdoSessionHandler and DbalSessionHandler-symfony/symfony#10652 (comment): Transactional DELETE + INSERT does not work as expected-symfony/symfony#10652 (comment): sqlsrv 2005 does not support the MERGE SQL, and if used it requires an HOLDLOCK- missing time update for sqlsrv and oracleCommits-------a0e1d4d [Doctrine Bridge] fix DBAL session handler according to PdoSessionHandler00d707f [HttpFoundation] use different approach for duplicate keys in postgres, fix merge for sqlsrv and oracle
@RuslanZavacky
Copy link
Contributor

Is this going to be merged to 2.4 & 2.5?

@Tobion
Copy link
ContributorAuthor

It is already.

@1emming1emming mentioned this pull requestAug 8, 2014
Sign up for freeto join this conversation on GitHub. Already have an account?Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@Tobion@fabpot@ruudk@onewheelskyward@tcql@gheydon@RuslanZavacky@srinikumar11@stof

[8]ページ先頭

©2009-2025 Movatter.jp