Uh oh!
There was an error while loading.Please reload this page.
- Notifications
You must be signed in to change notification settings - Fork9.7k
[HttpFoundation] fix PDO session handler under high concurrency#10652
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to ourterms of service andprivacy statement. We’ll occasionally send you account related emails.
Already on GitHub?Sign in to your account
Uh oh!
There was an error while loading.Please reload this page.
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
This was one main problem: It creates duplicate keys when session created meanwhile (between select and insert). There is no need to create an entry in read() at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others.Learn more.
For oraclesysdate was used which is of type DATE. But we expect an INTEGER column, otherwisegc() would not work. So this didn't make sense.
fabpot commentedApr 11, 2014
@Tobion Thank you so much for working on this issue, this is much appreciated. Is it ready to be merged? Or do you also want to implement lazy connections as well in this PR? |
ghost commentedApr 11, 2014
@Tobion The PR looks good. If I remember correctly you should not throw exceptions inside Is that helpful? |
Tobion commentedApr 11, 2014
@fabpot yes it's ready. Lazy connections would only be for master anyway. So if we want to make it respect the SessionHandlerInterface we should not throw exceptions but return false. But then I wonder how to best warn developers if something went wrong. For example when a developer forgots to create the table and we just return false everywhere, the dev would have hard times debugging the problem. One solution could be to use a logger in the class and log errors instead of throwing exceptions, or we can use |
ghost commentedApr 11, 2014
Oh gosh sorry I meant to say that in my post but forgot. I was going to suggest adding a logger or raise a PHP userspace error. Logger would be better IMO. |
Tobion commentedApr 11, 2014
What does it mean they cannot use objects? PDO and the possible logger are objects as well. |
Tobion commentedApr 11, 2014
Ok how I interpret it, is that objects are only destructed before write/close when you didn't call session_write_close() or session_register_shutdown() explicitly but relied on PHP to do it automatically at termination of the script (legacy style). So we should be fine. |
…rency (Tobion)This PR was merged into the 2.3 branch.Discussion----------[HttpFoundation] fix PDO session handler under high concurrency| Q | A| ------------- | ---| Bug fix? | yes| New feature? | no| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets |#8448 andhttp://trac.symfony-project.org/ticket/4777 (which was never really fixed as you can see here)| License | MIT- The first commit fixes PDO session handler under high concurrency.- The second commit uses MERGE SQL for MS SQL Server. Tested withhttp://sqlfiddle.com/#!6/66b6d/14- The third commit uses INSERT OR REPLACE for sqlite session handlerhttp://sqlfiddle.com/#!7/e6707/3What I find rather bad with the class design is that it depends on the table definition, but it's not part of the class. Also it doesn't make use of open() and close() which could be used to make the database connection lazy instead of having is open all the time when not needed. Doctrine also only lazy connects, but we use PDO directly here.Furthermore, the session handlers should not throw exceptions, from what I read, but return false when an error occurs. This is not followed in this class. Maybe@Drak knows how php session management behaves when the session handlers return false?Commits-------5c08e29 [HttpFoundation] use insert or replace for sqlite session handler05ea19a [HttpFoundation] use MERGE SQL for MS SQL Server session storagee58d7cf [HttpFoundation] fix PDO session handler under high concurrency
ghost commentedApr 11, 2014
@Tobion Yes, this is already handled by the symfony session code, which registers the shutdown handler as |
ruudk commentedApr 14, 2014
Just deployed this code to my server and seeing a lot of errors like this: Am I the only one? |
Tobion commentedApr 14, 2014
Can you give us more information please. Like which database are you using? |
Tobion commentedApr 14, 2014
Also since this class only throws RuntimeExceptions, the PDOException you got probably comes from somewhere else. A problem could be that you use the same connection which now interferes. Maybe you leave a transaction open without commiting or rolling back. |
ruudk commentedApr 14, 2014
@Tobion Sorry, I am using MySQL and I have a PDO instance only for sessions and use Doctrine2 for other queries (that has a separate connection). |
Tobion commentedApr 14, 2014
The PdoSessionHandler throws |
ruudk commentedApr 16, 2014
Hmm, could be coming from somewhere else then. I've switched to Redis Sessions. Thanks for helping. |
This PR was merged into the 2.3 branch.Discussion----------[HttpFoundation] Fix DbalSessionHandler| Q | A| ------------- | ---| Bug fix? | yes| New feature? | no| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets | -| License | MITThis is basically the same as#10652 for the DbalSessionHandler.- First commit fixes fix DbalSessionHandler for high concurrency, interface compliance, compatibility with all drivers (oci8, mysqli, pdo with mysql, sqlsrv, sqlite).- Second commit updates phpdoc of SessionHandlerInterface and unifies parameters of all handlers according to interface (so inheritdoc actually makes sense).Commits-------524bf84 [HttpFoundation] update phpdoc of SessionHandlerInterface and unify parameters of all handlers according to interfaceccdfbe6 [Doctrine Bridge] fix DbalSessionHandler for high concurrency, interface compliance, compatibility with all drivers (oci8, mysqli, pdo with mysql, sqlsrv, sqlite)
onewheelskyward commentedMay 6, 2014
We just ran into this problem with postgres on 2.3.13, it looks like that database might need some attention. Has anyone else encountered this before I start down the path? |
Tobion commentedMay 6, 2014
@onewheelskyward what problem did you encounter? |
onewheelskyward commentedMay 6, 2014
@Tobion As soon as we updated to 2.3.13, we got the duplicate session id errors. A revert to 2.3.12 fixed it for the time being. Uncaught exception 'PDOException' with message 'SQLSTATE[23505]: Unique violation: 7 ERROR: duplicate key value violates unique constraint "sessions_pkey"DETAIL: Key (session_id)=(keychangedtoprotecttheinnocent) already exists.' in /path/to/app/code/vendor/symfony/symfony/src/Symfony/Component/HttpFoundation/Session/Storage/Handler/PdoSessionHandler.php:201 |
Tobion commentedMay 6, 2014
Hm I don't see how this is possible. Postgres uses a DELETE followed by INSERT inside a transaction. Seehttps://github.com/symfony/symfony/blob/2.3/src/Symfony/Component/HttpFoundation/Session/Storage/Handler/PdoSessionHandler.php#L189 So the INSERT normally cannot create duplicate key. |
onewheelskyward commentedMay 6, 2014
I double checked the config, we are using that class. It seemed to be intermittent but we reverted as soon as we saw the errors. I'll try to recreate on our dev environment with a load test to be sure. Our PDO calls are set as follows, could that be why we got the statement exception instead of the in-function exception handling? |
tcql commentedMay 13, 2014
Just to follow up, I also am seeing the Unique constraint violation with Postgres 9.3.4 with Symfony 2.4+. I'm also pretty puzzled because a single transaction with a delete and then an update should precludeany possibility of this happening. |
onewheelskyward commentedMay 13, 2014
Now that's interesting. And I agree with you on the implementation of the On Tue, May 13, 2014 at 11:22 AM, tchannelnotifications@github.com wrote:
|
tcql commentedMay 13, 2014
I'm poking around, looks like itcould have something to do with postgres' default transaction / locking setup, but I'm not positive. Theoretically, if two transactions try doing the same action at the same time, given Postgres' default READ COMMITTED transaction mode
But it seems more like the deletions and insertions of multiple transactions are overlapping somehow EDIT: yep, related to locking / transaction isolation. running "LOCK table $this->table IN EXCLUSIVE MODE" at the beginning of the transaction fixes things. But a full table lock is very undesirable for concurrency / performance. |
tcql commentedMay 13, 2014
Okay, apologies for the annoying multiple posts. Here's what I came up with that fixes it for me: |
Tobion commentedMay 13, 2014
Yes that's how it should be. Maybe your database is configured with a different transaction isolation level, i.e. Read uncommitted |
tcql commentedMay 13, 2014
@Tobion in postgres, READ COMMITTED and READ UNCOMMITTED are the same.In Theory READ COMMITTED should work appropriately, but it doesn't. |
Tobion commentedMay 13, 2014
@tchannel can you then open a ticket on postgress or find out why it does not block. Have you tried to execute it manually through postgres command line using two connections? One should block then. Maybe it's a PDO problem. |
tcql commentedMay 13, 2014
@Tobion oh very weird. Doing it manually with two Then, I run the insert in the first transaction, and commit. Now the second transaction releases because the lock is free. But... the delete count is ZERO. It's like committing the first transaction is somehow doing things as individual commands still... The delete lock is released before the insert is fully processed, and the second transaction runs it's delete before the first transaction's insert goes through. So definitely not a PDO problem, it's an issue with postgres |
Tobion commentedMay 13, 2014
Actually that behavior could be the one described for read commited:http://www.postgresql.org/docs/9.1/static/transaction-iso.html The delete of the second transaction is just skipped because it didn't match before blocking and the delete is not re-evaluated for the whole table again. This works where delete is executed after concurred insert, but your example above will indeed create duplicate key.
|
tcql commentedMay 13, 2014
eesh. That's... unexpected? I made a bug report / question on the postgres-bugs mailing list. I'll follow up on here when/if I hear anything |
Tobion commentedMay 13, 2014
I think it's expected but just highly incomprehensible on first sight. |
Tobion commentedMay 15, 2014
@onewheelskyward and @tchannel please have a look at#10908. Would be nice if you can test it. |
gheydon commentedMay 28, 2014
I have had a problem upgrade past 2.3.13. I get the following excetion thrown. Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[07002]: [Microsoft][SQL Server Native Client 10.0]COUNT field incorrect or syntax error' in E:\Inetpub\test\api\vendor\symfony\symfony\src\Symfony\Component\HttpFoundation\Session\Storage\Handler\PdoSessionHandler.php:181Stack trace:#0 E:\Inetpub\test\api\vendor\symfony\symfony\src\Symfony\Component\HttpFoundation\Session\Storage\Handler\PdoSessionHandler.php(181): PDOStatement->execute()#1 E:\Inetpub\test\api\vendor\symfony\symfony\src\Symfony\Component\HttpFoundation\Session\Storage\Proxy\SessionHandlerProxy.php(77): Symfony\Component\HttpFoundation\Session\Storage\Handler\PdoSessionHandler->write('ifamr16dbjnalto...', '_sf2_attributes...')#2 [internal function]: Symfony\Component\HttpFoundation\Session\Storage\Proxy\SessionHandlerProxy->write('ifamr16dbjnalto...', '_sf2_attributes...')#3 [internal function]: session_write_close()#4 {main} |
Tobion commentedMay 28, 2014
@gheydon I think sqlsrv 2005 does not support the MERGE SQL statement used inhttps://github.com/symfony/symfony/blob/2.3/src/Symfony/Component/HttpFoundation/Session/Storage/Handler/PdoSessionHandler.php#L236 Seems to be available since 2008. |
Tobion commentedMay 28, 2014
This PR was merged into the 2.3 branch.Discussion----------[HttpFoundation] smaller fixes for PdoSessionHandler| Q | A| ------------- | ---| Bug fix? | yes| New feature? | no| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets |#10652| License | MITFor both the PdoSessionHandler and DbalSessionHandler-#10652 (comment): Transactional DELETE + INSERT does not work as expected-#10652 (comment): sqlsrv 2005 does not support the MERGE SQL, and if used it requires an HOLDLOCK- missing time update for sqlsrv and oracleCommits-------a0e1d4d [Doctrine Bridge] fix DBAL session handler according to PdoSessionHandler00d707f [HttpFoundation] use different approach for duplicate keys in postgres, fix merge for sqlsrv and oracle
This PR was merged into the 2.3 branch.Discussion----------[HttpFoundation] smaller fixes for PdoSessionHandler| Q | A| ------------- | ---| Bug fix? | yes| New feature? | no| BC breaks? | no| Deprecations? | no| Tests pass? | yes| Fixed tickets | #10652| License | MITFor both the PdoSessionHandler and DbalSessionHandler-symfony/symfony#10652 (comment): Transactional DELETE + INSERT does not work as expected-symfony/symfony#10652 (comment): sqlsrv 2005 does not support the MERGE SQL, and if used it requires an HOLDLOCK- missing time update for sqlsrv and oracleCommits-------a0e1d4d [Doctrine Bridge] fix DBAL session handler according to PdoSessionHandler00d707f [HttpFoundation] use different approach for duplicate keys in postgres, fix merge for sqlsrv and oracle
RuslanZavacky commentedJul 15, 2014
Is this going to be merged to 2.4 & 2.5? |
Tobion commentedJul 16, 2014
It is already. |
What I find rather bad with the class design is that it depends on the table definition, but it's not part of the class. Also it doesn't make use of open() and close() which could be used to make the database connection lazy instead of having is open all the time when not needed. Doctrine also only lazy connects, but we use PDO directly here.
Furthermore, the session handlers should not throw exceptions, from what I read, but return false when an error occurs. This is not followed in this class. Maybe@Drak knows how php session management behaves when the session handlers return false?