Sitelet https://web.archive.org/web/20201121082147/https://github.com/sqlalchemy/sqlalchemy/issues/5408
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gevent with SQL Alchemy and Pyodbc #5408

Closed
jlongo-daitan opened this issue Jun 19, 2020 · 6 comments
Closed

Gevent with SQL Alchemy and Pyodbc #5408

jlongo-daitan opened this issue Jun 19, 2020 · 6 comments

Comments

@jlongo-daitan
Copy link

@jlongo-daitan jlongo-daitan commented Jun 19, 2020

I've tried to search in Google, but I haven't had good results.

Is it possible to use gevent with SQL Alchemy and pyodbc?

I could make it work with my current Flask Restplus application, but when SQL Alchemy comes into play, the async behavior stop working.

@zzzeek
Copy link
Member

@zzzeek zzzeek commented Jun 19, 2020

pyodbc doesn't have any async capbiltities. the only database among those supported that itself has async capabilitiy as all is postgresql. there's some discussion about postgresql with pyodbc and async at mkleehammer/pyodbc#348 and essentially there's no real rationale for it to support this.

there's no actual reason to use gevent with relational database access however and it will only make your code less reliable and more difficult to debug. I speak from considerable experience in this area.

@jlongo-daitan
Copy link
Author

@jlongo-daitan jlongo-daitan commented Jun 19, 2020

@zzzeek I agree the code can become less reliable and difficult to debug. However, I believe there will be performance improvements for the application in general due to the async approach if the application is I/O bound.

For example, here http://blog.devsda.fr/python-gevent.html they reported significant performance improvements when using gevent with pymssql.

@zzzeek
Copy link
Member

@zzzeek zzzeek commented Jun 20, 2020

So I can't get you gevent support here, and also my studied opinion is that it's very hard to make Python IO bound with a high performance database on a local network, as it is not Go, Erlang, Node, or C, it's interpreted Python and has very high CPU overhead relative to compiled languages , see https://techspot.zzzeek.org/2015/02/15/asynchronous-python-and-databases/ , http://calpaterson.com/async-python-is-not-faster.html etc. I have yet to see a real benchmark showing async Python to be faster.

Also with the monkeypatching approach you put yourself in great danger of CPU bound processes hanging up your database, I've dealt with an actual outage of that nature, and additionally I've dealt with several across multiple applications which were directly caused by the usage of eventlet or gevent which would then pile up requests that were blocked due to mis-configuration of the number of greenlets relative to what the database was tailored for.

@jlongo-daitan
Copy link
Author

@jlongo-daitan jlongo-daitan commented Jun 20, 2020

@zzzeek thx for the explanation

@zzzeek
Copy link
Member

@zzzeek zzzeek commented Aug 7, 2020

in case it helps SQLAlchemy 1.4 will have native asyncio support. no path for pyodbc that I'm familiar with though :(. closing this for now.

@zzzeek zzzeek closed this Aug 7, 2020
@jlongo-daitan
Copy link
Author

@jlongo-daitan jlongo-daitan commented Aug 7, 2020

ok thx

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.