Question or problem about Python programming:
I have a SQLAlchemy query object and want to get the text of the compiled SQL statement, with all its parameters bound (e.g. no %s or other variables waiting to be bound by the statement compiler or MySQLdb dialect engine, etc).
Calling str() on the query reveals something like this:
SELECT id WHERE date_added <= %s AND date_added >= %s ORDER BY count DESC
I’ve tried looking in query._params but it’s an empty dict. I wrote my own compiler using this example of the sqlalchemy.ext.compiler.compiles decorator but even the statement there still has %s where I want data.
I can’t quite figure out when my parameters get mixed in to create the query; when examining the query object they’re always an empty dictionary (though the query executes fine and the engine prints it out when you turn echo logging on).
I’m starting to get the message that SQLAlchemy doesn’t want me to know the underlying query, as it breaks the general nature of the expression API’s interface all the different DB-APIs. I don’t mind if the query gets executed before I found out what it was; I just want to know!
How to solve the problem:
Solution 1:
This blog provides an updated answer.
Quoting from the blog post, this is suggested and worked for me.
>>> from sqlalchemy.dialects import postgresql >>> print str(q.statement.compile(dialect=postgresql.dialect()))
Where q is defined as:
>>> q = DBSession.query(model.Name).distinct(model.Name.value) \
.order_by(model.Name.value)
Or just any kind of session.query().
Thanks to Nicolas Cadou for the answer! I hope it helps others who come searching here.
Solution 2:
The documentation uses literal_binds to print a query q including parameters:
print(q.statement.compile(compile_kwargs={"literal_binds": True}))
the above approach has the caveats that it is only supported for basic types, such as ints and strings, and furthermore if a bindparam() without a pre-set value is used directly, it won’t be able to stringify that either.
The documentation also issues this warning:
Never use this technique with string content received from untrusted
input, such as from web forms or other user-input applications.
SQLAlchemy’s facilities to coerce Python values into direct SQL string
values are not secure against untrusted input and do not validate the
type of data being passed. Always use bound parameters when
programmatically invoking non-DDL SQL statements against a relational
database.
Solution 3:
This should work with Sqlalchemy >= 0.6
from sqlalchemy.sql import compiler from psycopg2.extensions import adapt as sqlescape # or use the appropiate escape function from your db driver def compile_query(query): dialect = query.session.bind.dialect statement = query.statement comp = compiler.SQLCompiler(dialect, statement) comp.compile() enc = dialect.encoding params = {} for k,v in comp.params.iteritems(): if isinstance(v, unicode): v = v.encode(enc) params[k] = sqlescape(v) return (comp.string.encode(enc) % params).decode(enc)
Solution 4:
For the MySQLdb backend I modified albertov’s awesome answer (thanks so much!) a bit. I’m sure they could be merged to check if comp.positional was True but that’s slightly beyond the scope of this question.
def compile_query(query): from sqlalchemy.sql import compiler from MySQLdb.converters import conversions, escape dialect = query.session.bind.dialect statement = query.statement comp = compiler.SQLCompiler(dialect, statement) comp.compile() enc = dialect.encoding params = [] for k in comp.positiontup: v = comp.params[k] if isinstance(v, unicode): v = v.encode(enc) params.append( escape(v, conversions) ) return (comp.string.encode(enc) % tuple(params)).decode(enc)
Solution 5:
Thing is, sqlalchemy never mixes the data with your query. The query and the data are passed separately to your underlying database driver – the interpolation of data happens in your database.
Sqlalchemy passes the query as you’ve seen in str(myquery) to the database, and the values will go in a separate tuple.
You could use some approach where you interpolate the data with the query yourself (as albertov suggested below), but that’s not the same thing that sqlalchemy is executing.
