# SQLAlchemy 0.9 Documentation

Search:
Release: 0.9.3 | Release Date: February 19, 2014

# 0.9 Changelog

## 0.9.3

Released: February 19, 2014

### orm

- [orm] [feature] Added new `MapperEvents.before_configured()` event which allows an event at the start of `configure_mappers()`, as well as `__declare_first__()` hook within declarative to complement `__declare_last__()`.

- [orm] [bug] Fixed bug where `Query.get()` would fail to consistently raise the `InvalidRequestError` that invokes when called on a query with existing criterion, when the given identity is already present in the identity map.

  This change is also backported to: 0.8.5

  References: #2951

- [orm] [bug] [sqlite] Fixed bug in SQLite “join rewriting” where usage of an exists() construct would fail to be rewritten properly, such as when the exists is mapped to a column_property in an intricate nested-join scenario. Also fixed a somewhat related issue where join rewriting would fail on the columns clause of the SELECT statement if the targets were aliased tables, as opposed to individual aliased columns.

  References: #2967

- [orm] [bug] Fixed an 0.9 regression where ORM instance or mapper events applied to a base class such as a declarative base with the propagate=True flag would fail to apply to existing mapped classes which also used inheritance due to an assertion. Addtionally, repaired an attribute error which could occur during removal of such an event, depending on how it was first assigned.

  References: #2949

- [orm] [bug] Improved the initialization logic of composite attributes such that calling `MyClass.attribute` will not require that the configure mappers step has occurred, e.g. it will just work without throwing any error.

  References: #2935

- [orm] [bug] More issues with [ticket:2932] first resolved in 0.9.2 where using a column key of the form `<tablename>_<columnname>` matching that of an aliased column in the text would still not match at the ORM level, which is ultimately due to a core column-matching issue. Additional rules have been added so that the column `_label` is taken into account when working with a `TextAsFrom` construct or with literal columns.

  References: #2932

### orm declarative

- [bug] [orm] [declarative] Fixed bug where `AbstractConcreteBase` would fail to be fully usable within declarative relationship configuration, as its string classname would not be available in the registry of classnames at mapper configuration time. The class now explicitly adds itself to the class regsitry, and additionally both `AbstractConcreteBase` as well as `ConcreteBase` set themselves up before mappers are configured within the `configure_mappers()` setup, using the new `MapperEvents.before_configured()` event.

  References: #2950

### engine

- [engine] [bug] [pool] Fixed a critical regression caused by #2880 where the newly concurrent ability to return connections from the pool means that the “first_connect” event is now no longer synchronized either, thus leading to dialect mis-configurations under even minimal concurrency situations.

  This change is also backported to: 0.8.5

  References: #2964, #2880

### sql

- [sql] [bug] Fixed bug where calling `Insert.values()` with an empty list or tuple would raise an IndexError. It now produces an empty insert construct as would be the case with an empty dictionary.

  This change is also backported to: 0.8.5

  References: #2944

- [sql] [bug] Fixed bug where `in_()` would go into an endless loop if erroneously passed a column expression whose comparator included the `__getitem__()` method, such as a column that uses the `postgresql.ARRAY` type.

  This change is also backported to: 0.8.5

  References: #2957

- [sql] [bug] Fixed regression in new “naming convention” feature where conventions would fail if the referred table in a foreign key contained a schema name. Pull request courtesy Thomas Farvour.

  References: pull request github:67

- [sql] [bug] Fixed bug where so-called “literal render” of `bindparam()` constructs would fail if the bind were constructed with a callable, rather than a direct value. This prevented ORM expressions from being rendered with the “literal_binds” compiler flag.

### postgresql

- [postgresql] [feature] Added the `TypeEngine.python_type` convenience accessor onto the `postgresql.ARRAY` type. Pull request courtesy Alexey Terentev.

  References: pull request github:64

- [postgresql] [bug] Added an additional message to psycopg2 disconnect detection, “could not send data to server”, which complements the existing “could not receive data from server” and has been observed by users.

  This change is also backported to: 0.8.5

  References: #2936

- [postgresql] [bug]

  > Support has been improved for Postgresql reflection behavior on very old (pre 8.1) versions of Postgresql, and potentially other PG engines such as Redshift (assuming Redshift reports the version as < 8.1). The query for “indexes” as well as “primary keys” relies upon inspecting a so-called “int2vector” datatype, which refuses to coerce to an array prior to 8.1 causing failures regarding the “ANY()” operator used in the query. Extensive googling has located the very hacky, but recommended-by-PG-core-developer query to use when PG version < 8.1 is in use, so index and primary key constraint reflection now work on these versions.

  This change is also backported to: 0.8.5

- [postgresql] [bug] Revised this very old issue where the Postgresql “get primary key” reflection query were updated to take into account primary key constraints that were renamed; the newer query fails on very old versions of Postgresql such as version 7, so the old query is restored in those cases when server_version_info < (8, 0) is detected.

  This change is also backported to: 0.8.5

  References: #2291

- [postgresql] [bug] Added server version detection to the newly added dialect startup query for “show standard_conforming_strings”; as this variable was added as of PG 8.2, we skip the query for PG versions who report a version string earlier than that.

  References: #2946

### mysql

- [mysql] [feature] Added new MySQL-specific `mysql.DATETIME` which includes fractional seconds support; also added fractional seconds support to `mysql.TIMESTAMP`. DBAPI support is limited, though fractional seconds are known to be supported by MySQL Connector/Python. Patch courtesy Geert JM Vanderkelen.

  This change is also backported to: 0.8.5

  References: #2941

- [mysql] [bug] Added support for the `PARTITION BY` and `PARTITIONS` MySQL table keywords, specified as `mysql_partition_by='value'` and `mysql_partitions='value'` to `Table`. Pull request courtesy Marcus McCurdy.

  This change is also backported to: 0.8.5

  References: #2966, pull request bitbucket:12

- [mysql] [bug] Fixed bug which prevented MySQLdb-based dialects (e.g. pymysql) from working in Py3K, where a check for “connection charset” would fail due to Py3K’s more strict value comparison rules. The call in question wasn’t taking the database version into account in any case as the server version was still None at that point, so the method overall has been simplified to rely upon connection.character_set_name().

  This change is also backported to: 0.8.5

  References: #2933

- [mysql] [bug] [cymysql] Fixed bug in cymysql dialect where a version string such as `'33a-MariaDB'` would fail to parse properly. Pull request courtesy Matt Schmidt.

  References: #2934, pull request github:69

### sqlite

- [sqlite] [bug] The SQLite dialect will now skip unsupported arguments when reflecting types; such as if it encounters a string like `INTEGER(5)`, the `INTEGER` type will be instantiated without the “5” being included, based on detecting a `TypeError` on the first attempt.

- [sqlite] [bug] Support has been added to SQLite type reflection to fully support the “type affinity” contract specified at http://www.sqlite.org/datatype3.html. In this scheme, keywords like `INT`, `CHAR`, `BLOB` or `REAL` located in the type name generically associate the type with one of five affinities. Pull request courtesy Erich Blume.

  See also

  Type Reflection

  References: pull request github:65

### misc

- [feature] [examples] Added optional “changed” column to the versioned rows example, as well as support for when the versioned `Table` has an explicit `schema` argument. Pull request courtesy jplaverdure.

  References: pull request github:41

- [bug] [ext] Fixed bug where the `AutomapBase` class of the new automap extension would fail if classes were pre-arranged in single or potentially joined inheritance patterns. The repaired joined inheritance issue could also potentially apply when using `DeferredReflection` as well.

## 0.9.2

Released: February 2, 2014

### orm

- [orm] [feature] Added a new parameter `Operators.op.is_comparison`. This flag allows a custom op from `Operators.op()` to be considered as a “comparison” operator, thus usable for custom `relationship.primaryjoin` conditions.

  See also

  Using custom operators in join conditions

- [orm] [feature] Support is improved for supplying a `join()` construct as the target of `relationship.secondary` for the purposes of creating very complex `relationship()` join conditions. The change includes adjustments to query joining, joined eager loading to not render a SELECT subquery, changes to lazy loading such that the “secondary” target is properly included in the SELECT, and changes to declarative to better support specification of a join() object with classes as targets.

  The new use case is somewhat experimental, but a new documentation section has been added.

  See also

  Composite “Secondary” Joins

- [orm] [bug] Fixed error message when an iterator object is passed to `class_mapper()` or similar, where the error would fail to render on string formatting. Pullreq courtesy Kyle Stark.

  This change is also backported to: 0.8.5

  References: pull request github:58

- [orm] [bug] Fixed bug in new `TextAsFrom` construct where `Column`- oriented row lookups were not matching up to the ad-hoc `ColumnClause` objects that `TextAsFrom` generates, thereby making it not usable as a target in `Query.from_statement()`. Also fixed `Query.from_statement()` mechanics to not mistake a `TextAsFrom` for a `Select` construct. This bug is also an 0.9 regression as the `Text.columns()` method is called to accommodate the `text.typemap` argument.

  References: #2932

- [orm] [bug] Added a new directive used within the scope of an attribute “set” operation to disable autoflush, in the case that the attribute needs to lazy-load the “old” value, as in when replacing one-to-one values or some kinds of many-to-one. A flush at this point otherwise occurs at the point that the attribute is None and can cause NULL violations.

  References: #2921

- [orm] [bug] Fixed an 0.9 regression where the automatic aliasing applied by `Query` and in other situations where selects or joins were aliased (such as joined table inheritance) could fail if a user-defined `Column` subclass were used in the expression. In this case, the subclass would fail to propagate ORM-specific “annotations” along needed by the adaptation. The “expression annotations” system has been corrected to account for this case.

  References: #2918

- [orm] [bug] Fixed a bug involving the new flattened JOIN structures which are used with `joinedload()` (thereby causing a regression in joined eager loading) as well as `aliased()` in conjunction with the `flat=True` flag and joined-table inheritance; basically multiple joins across a “parent JOIN sub” entity using different paths to get to a target class wouldn’t form the correct ON conditions. An adjustment / simplification made in the mechanics of figuring out the “left side” of the join in the case of an aliased, joined-inh class repairs the issue.

  References: #2908

### engine

- [engine] [feature] [pool] Added a new pool event `PoolEvents.invalidate()`. Called when a DBAPI connection is to be marked as “invaldated” and discarded from the pool.

### sql

- [sql] [feature] Added `MetaData.reflect.**dialect_kwargs` to support dialect-level reflection options for all `Table` objects reflected.

- [sql] [feature] Added a new feature which allows automated naming conventions to be applied to `Constraint` and `Index` objects. Based on a recipe in the wiki, the new feature uses schema-events to set up names as various schema objects are associated with each other. The events then expose a configuration system through a new argument `MetaData.naming_convention`. This system allows production of both simple and custom naming schemes for constraints and indexes on a per-`MetaData` basis.

  See also

  Configuring Constraint Naming Conventions

  References: #2923

- [sql] [feature] Options can now be specified on a `PrimaryKeyConstraint` object independently of the specification of columns in the table with the `primary_key=True` flag; use a `PrimaryKeyConstraint` object with no columns in it to achieve this result.

  Previously, an explicit `PrimaryKeyConstraint` would have the effect of those columns marked as `primary_key=True` being ignored; since this is no longer the case, the `PrimaryKeyConstraint` will now assert that either one style or the other is used to specify the columns, or if both are present, that the column lists match exactly. If an inconsistent set of columns in the `PrimaryKeyConstraint` and within the `Table` marked as `primary_key=True` are present, a warning is emitted, and the list of columns is taken only from the `PrimaryKeyConstraint` alone as was the case in previous releases.

  See also

  `PrimaryKeyConstraint`

  References: #2910

- [sql] [feature] The system by which schema constructs and certain SQL constructs accept dialect-specific keyword arguments has been enhanced. This system includes commonly the `Table` and `Index` constructs, which accept a wide variety of dialect-specific arguments such as `mysql_engine` and `postgresql_where`, as well as the constructs `PrimaryKeyConstraint`, `UniqueConstraint`, `Update`, `Insert` and `Delete`, and also newly added kwarg capability to `ForeignKeyConstraint` and `ForeignKey`. The change is that participating dialects can now specify acceptable argument lists for these constructs, allowing an argument error to be raised if an invalid keyword is specified for a particular dialect. If the dialect portion of the keyword is unrecognized, a warning is emitted only; while the system will actually make use of setuptools entrypoints in order to locate non-local dialects, the use case where certain dialect-specific arguments are used in an environment where that third-party dialect is uninstalled remains supported. Dialects also have to explicitly opt-in to this system, so that external dialects which aren’t making use of this system will remain unaffected.

  References: #2866

- [sql] [bug] The behavior of `Table.tometadata()` has been adjusted such that the schema target of a `ForeignKey` will not be changed unless that schema matches that of the parent table. That is, if a table “schema_a.user” has a foreign key to “schema_b.order.id”, the “schema_b” target will be maintained whether or not the “schema” argument is passed to `Table.tometadata()`. However if a table “schema_a.user” refers to “schema_a.order.id”, the presence of “schema_a” will be updated on both the parent and referred tables. This is a behavioral change hence isn’t likely to be backported to 0.8; it is assumed that the previous behavior is pretty buggy however and that it’s unlikely anyone was relying upon it.

  Additionally, a new parameter has been added `Table.tometadata.referred_schema_fn`. This refers to a callable function which will be used to determine the new referred schema for any `ForeignKeyConstraint` encountered in the tometadata operation. This callable can be used to revert to the previous behavior or to customize how referred schemas are treated on a per-constraint basis.

  References: #2913

- [sql] [bug] Fixed bug whereby binary type would fail in some cases if used with a “test” dialect, such as a DefaultDialect or other dialect with no DBAPI.

- [sql] [bug] [py3k] Fixed bug where “literal binds” wouldn’t work with a bound parameter that’s a binary type. A similar, but different, issue is fixed in 0.8.

- [sql] [bug] Fixed regression whereby the “annotation” system used by the ORM was leaking into the names used by standard functions in `sqlalchemy.sql.functions`, such as `func.coalesce()` and `func.max()`. Using these functions in ORM attributes and thus producing annotated versions of them could corrupt the actual function name rendered in the SQL.

  References: #2927

- [sql] [bug] Fixed 0.9 regression where the new sortable support for `RowProxy` would lead to `TypeError` when compared to non-tuple types as it attempted to apply tuple() to the “other” object unconditionally. The full range of Python comparison operators have now been implemented on `RowProxy`, using an approach that guarantees a comparison system that is equivalent to that of a tuple, and the “other” object is only coerced if it’s an instance of RowProxy.

  References: #2924, #2848

- [sql] [bug] A `UniqueConstraint` created inline with a `Table` that has no columns within it will be skipped. Pullreq courtesy Derek Harland.

  References: pull request bitbucket:11

- [sql] [bug] [orm] Fixed the multiple-table “UPDATE..FROM” construct, only usable on MySQL, to correctly render the SET clause among multiple columns with the same name across tables. This also changes the name used for the bound parameter in the SET clause to “<tablename>_<colname>” for the non-primary table only; as this parameter is typically specified using the `Column` object directly this should not have an impact on applications. The fix takes effect for both `Table.update()` as well as `Query.update()` in the ORM.

  References: #2912

### schema

- [schema] [bug] Restored `sqlalchemy.schema.SchemaVisitor` to the `.schema` module. Pullreq courtesy Sean Dague.

  References: pull request github:57

### postgresql

- [postgresql] [feature] Added a new dialect-level argument `postgresql_ignore_search_path`; this argument is accepted by both the `Table` constructor as well as by the `MetaData.reflect()` method. When in use against Postgresql, a foreign-key referenced table which specifies a remote schema name will retain that schema name even if the name is present in the `search_path`; the default behavior since 0.7.3 has been that schemas present in `search_path` would not be copied to reflected `ForeignKey` objects. The documentation has been updated to describe in detail the behavior of the `pg_get_constraintdef()` function and how the `postgresql_ignore_search_path` feature essentially determines if we will honor the schema qualification reported by this function or not.

  See also

  Remote-Schema Table Introspection and Postgresql search_path

  References: #2922

### mysql

- [mysql] [bug] Some missing methods added to the cymysql dialect, including _get_server_version_info() and _detect_charset(). Pullreq courtesy Hajime Nakagami.

  This change is also backported to: 0.8.5

  References: pull request github:61

- [mysql] [bug] [sql] Added new test coverage for so-called “down adaptions” of SQL types, where a more specific type is adapted to a more generic one - this use case is needed by some third party tools such as `sqlacodegen`. The specific cases that needed repair within this test suite were that of `mysql.ENUM` being downcast into a `types.Enum`, and that of SQLite date types being cast into generic date types. The `adapt()` method needed to become more specific here to counteract the removal of a “catch all” `**kwargs` collection on the base `TypeEngine` class that was removed in 0.9.

  References: #2917

- [mysql] [bug] The MySQL CAST compilation now takes into account aspects of a string type such as “charset” and “collation”. While MySQL wants all character- based CAST calls to use the CHAR type, we now create a real CHAR object at CAST time and copy over all the parameters it has, so that an expression like `cast(x, mysql.TEXT(charset='utf8'))` will render `CAST(t.col AS CHAR CHARACTER SET utf8)`.

- [mysql] [bug] Added new “unicode returns” detection to the MySQL dialect and to the default dialect system overall, such that any dialect can add extra “tests” to the on-first-connect “does this DBAPI return unicode directly?” detection. In this case, we are adding a check specifically against the “utf8” encoding with an explicit “utf8_bin” collation type (after checking that this collation is available) to test for some buggy unicode behavior observed with MySQLdb version 1.2.3. While MySQLdb has resolved this issue as of 1.2.4, the check here should guard against regressions. The change also allows the “unicode” checks to log in the engine logs, which was not previously the case.

  References: #2906

- [mysql] [bug] [engine] [pool] `Connection` now associates a new `RootTransaction` or `TwoPhaseTransaction` with its immediate `_ConnectionFairy` as a “reset handler” for the span of that transaction, which takes over the task of calling commit() or rollback() for the “reset on return” behavior of `Pool` if the transaction was not otherwise completed. This resolves the issue that a picky transaction like that of MySQL two-phase will be properly closed out when the connection is closed without an explicit rollback or commit (e.g. no longer raises “XAER_RMFAIL” in this case - note this only shows up in logging as the exception is not propagated within pool reset). This issue would arise e.g. when using an orm `Session` with `twophase` set, and then `Session.close()` is called without an explicit rollback or commit. The change also has the effect that you will now see an explicit “ROLLBACK” in the logs when using a `Session` object in non-autocommit mode regardless of how that session was discarded. Thanks to Jeff Dairiki and Laurence Rowe for isolating the issue here.

  References: #2907

### sqlite

- [sqlite] [bug] Fixed bug whereby SQLite compiler failed to propagate compiler arguments such as “literal binds” into a CAST expression.

### mssql

- [mssql] [feature] Added an option `mssql_clustered` to the `UniqueConstraint` and `PrimaryKeyConstraint` constructs; on SQL Server, this adds the `CLUSTERED` keyword to the constraint construct within DDL. Pullreq courtesy Derek Harland.

  References: pull request bitbucket:11

### oracle

- [oracle] [bug] It’s been observed that the usage of a cx_Oracle “outputtypehandler” in Python 2.xx in order to coerce string values to Unicode is inordinately expensive; even though cx_Oracle is written in C, when you pass the Python `unicode` primitive to cursor.var() and associate with an output handler, the library counts every conversion as a Python function call with all the requisite overhead being recorded; this despite the fact when running in Python 3, all strings are also unconditionally coerced to unicode but it does not incur this overhead, meaning that cx_Oracle is failing to use performant techniques in Py2K. As SQLAlchemy cannot easily select for this style of type handler on a per-column basis, the handler was assembled unconditionally thereby adding the overhead to all string access.

  So this logic has been replaced with SQLAlchemy’s own unicode conversion system, which now only takes effect in Py2K for columns that are requested as unicode. When C extensions are used, SQLAlchemy’s system appears to be 2-3x faster than cx_Oracle’s. Additionally, SQLAlchemy’s unicode conversion has been enhanced such that when the “conditional” converter is required (now needed for the Oracle backend), the check for “already unicode” is now performed in C and no longer introduces significant overhead.

  This change has two impacts on the cx_Oracle backend. One is that string values in Py2K which aren’t specifically requested with the Unicode type or convert_unicode=True will now come back as `str`, not `unicode` - this behavior is similar to a backend such as MySQL. Additionally, when unicode values are requested with the cx_Oracle backend, if the C extensions are not used, there is now an additional overhead of an isinstance() check per column. This tradeoff has been made as it can be worked around and no longer places a performance burden on the likely majority of Oracle result columns that are non-unicode strings.

  References: #2911

### misc

- [bug] [examples] Added a tweak to the “history_meta” example where the check for “history” on a relationship-bound attribute will now no longer emit any SQL if the relationship is unloaded.

- [bug] [pool] The argument names for the `PoolEvents.reset()` event have been renamed to `dbapi_connection` and `connection_record` in order to maintain consistency with all the other pool events. It is expected that any existing listeners for this relatively new and seldom-used event are using positional style to receive arguments in any case.

- [bug] [cextensions] [py3k] Fixed an issue where the C extensions in Py3K are using the wrong API to specify the top-level module function, which breaks in Python 3.4b2. Py3.4b2 changes PyMODINIT_FUNC to return “void” instead of “PyObject *”, so we now make sure to use “PyMODINIT_FUNC” instead of “PyObject *” directly. Pull request courtesy cgohlke.

  References: pull request github:55

## 0.9.1

Released: January 5, 2014

### orm

- [orm] [feature] [extensions] A new, experimental extension `sqlalchemy.ext.automap` is added. This extension expands upon the functionality of Declarative as well as the `DeferredReflection` class to produce a base class which automatically generates mapped classes and relationships based on table metadata.

  See also

  Automap Extension

  Automap

- [orm] [bug] [events] Fixed regression where using a `functools.partial()` with the event system would cause a recursion overflow due to usage of inspect.getargspec() on it in order to detect a legacy calling signature for certain events, and apparently there’s no way to do this with a partial object. Instead we skip the legacy check and assume the modern style; the check itself now only occurs for the SessionEvents.after_bulk_update and SessionEvents.after_bulk_delete events. Those two events will require the new signature style if assigned to a “partial” event listener.

  References: #2905

- [orm] [bug] Fixed bug where using new `Session.info` attribute would fail if the `.info` argument were only passed to the `sessionmaker` creation call but not to the object itself. Courtesy Robin Schoonover.

  References: pull request bitbucket:9

- [orm] [bug] Fixed regression where we don’t check the given name against the correct string class when setting up a backref based on a name, therefore causing the error “too many values to unpack”. This was related to the Py3k conversion.

  References: #2901

- [orm] [bug] Fixed regression where we apparently still create an implicit alias when saying query(B).join(B.cs), where “C” is a joined inh class; however, this implicit alias was created only considering the immediate left side, and not a longer chain of joins along different joined-inh subclasses of the same base. As long as we’re still implicitly aliasing in this case, the behavior is dialed back a bit so that it will alias the right side in a wider variety of cases.

  References: #2903

### orm declarative

- [bug] [orm] [declarative] Fixed an extremely unlikely memory issue where when using `DeferredReflection` to define classes pending for reflection, if some subset of those classes were discarded before the `DeferredReflection.prepare()` method were called to reflect and map the class, a strong reference to the class would remain held within the declarative internals. This internal collection of “classes to map” now uses weak references against the classes themselves.

- [bug] [orm] [declarative] A quasi-regression where apparently in 0.8 you can set a class-level attribute on declarative to simply refer directly to an `InstrumentedAttribute` on a superclass or on the class itself, and it acts more or less like a synonym; in 0.9, this fails to set up enough bookkeeping to keep up with the more liberalized backref logic from #2789. Even though this use case was never directly considered, it is now detected by declarative at the “setattr()” level as well as when setting up a subclass, and the mirrored/renamed attribute is now set up as a `synonym()` instead.

  References: #2900

### sql

- [sql] [feature] Conjunctions like `and_()` and `or_()` can now accept Python generators as a single argument, e.g.:

      and_(x == y for x, y in tuples)

  The logic here looks for a single argument `*args` where the first element is an instance of `types.GeneratorType`.

### schema

- [schema] [feature] The `Table.extend_existing` and `Table.autoload_replace` parameters are now available on the `MetaData.reflect()` method.

## 0.9.0

Released: December 30, 2013

### orm

- [orm] [feature] The `exc.StatementError` or DBAPI-related subclass now can accomodate additional information about the “reason” for the exception; the `Session` now adds some detail to it when the exception occurs within an autoflush. This approach is taken as opposed to combining `FlushError` with a Python 3 style “chained exception” approach so as to maintain compatibility both with Py2K code as well as code that already catches `IntegrityError` or similar.

- [orm] [feature] [backrefs] Added new argument `include_backrefs=True` to the `validates()` function; when set to False, a validation event will not be triggered if the event was initated as a backref to an attribute operation from the other side.

  See also

  include_backrefs=False option for @validates

  References: #1535

- [orm] [feature] A new API for specifying the `FOR UPDATE` clause of a `SELECT` is added with the new `Query.with_for_update()` method, to complement the new `GenerativeSelect.with_for_update()` method. Pull request courtesy Mario Lassnig.

  See also

  New FOR UPDATE support on select(), Query()

  References: pull request github:42

- [orm] [bug] An adjustment to the `subqueryload()` strategy which ensures that the query runs after the loading process has begun; this is so that the subqueryload takes precedence over other loaders that may be hitting the same attribute due to other eager/noload situations at the wrong time.

  This change is also backported to: 0.8.5

  References: #2887

- [orm] [bug] Fixed bug when using joined table inheritance from a table to a select/alias on the base, where the PK columns were also not same named; the persistence system would fail to copy primary key values from the base table to the inherited table upon INSERT.

  This change is also backported to: 0.8.5

  References: #2885

- [orm] [bug] `composite()` will raise an informative error message when the columns/attribute (names) passed don’t resolve to a Column or mapped attribute (such as an erroneous tuple); previously raised an unbound local.

  This change is also backported to: 0.8.5

  References: #2889

- [orm] [bug] Fixed a regression introduced by #2818 where the EXISTS query being generated would produce a “columns being replaced” warning for a statement with two same-named columns, as the internal SELECT wouldn’t have use_labels set.

  This change is also backported to: 0.8.4

  References: #2818

- [orm] [bug] [collections] [py3k] Added support for the Python 3 method `list.clear()` within the ORM collection instrumentation system; pull request courtesy Eduardo Schettino.

  References: pull request github:40

- [orm] [bug] Some refinements to the `AliasedClass` construct with regards to descriptors, like hybrids, synonyms, composites, user-defined descriptors, etc. The attribute adaptation which goes on has been made more robust, such that if a descriptor returns another instrumented attribute, rather than a compound SQL expression element, the operation will still proceed. Addtionally, the “adapted” operator will retain its class; previously, a change in class from `InstrumentedAttribute` to `QueryableAttribute` (a superclass) would interact with Python’s operator system such that an expression like `aliased(MyClass.x) > MyClass.x` would reverse itself to read `myclass.x < myclass_1.x`. The adapted attribute will also refer to the new `AliasedClass` as its parent which was not always the case before.

  References: #2872

- [orm] [bug] The `viewonly` flag on `relationship()` will now prevent attribute history from being written on behalf of the target attribute. This has the effect of the object not being written to the Session.dirty list if it is mutated. Previously, the object would be present in Session.dirty, but no change would take place on behalf of the modified attribute during flush. The attribute still emits events such as backref events and user-defined events and will still receive mutations from backrefs.

  See also

  viewonly=True on relationship() prevents history from taking effect

  References: #2833

- [orm] [bug] Added support for new `Session.info` attribute to `scoped_session`.

- [orm] [bug] Fixed bug where usage of new `Bundle` object would cause the `Query.column_descriptions` attribute to fail.

- [orm] [bug] [sqlite] [sql] Fixed a regression introduced by the join rewriting feature of #2369 and #2587 where a nested join with one side already an aliased select would fail to translate the ON clause on the outside correctly; in the ORM this could be seen when using a SELECT statement as a “secondary” table.

  References: #2858

### orm declarative

- [bug] [orm] [declarative] Declarative does an extra check to detect if the same `Column` is mapped multiple times under different properties (which typically should be a `synonym()` instead) or if two or more `Column` objects are given the same name, raising a warning if this condition is detected.

  References: #2828

- [bug] [orm] [declarative] The `DeferredReflection` class has been enhanced to provide automatic reflection support for the “secondary” table referred to by a `relationship()`. “secondary”, when specified either as a string table name, or as a `Table` object with only a name and `MetaData` object will also be included in the reflection process when `DeferredReflection.prepare()` is called.

  References: #2865

- [bug] [orm] [declarative] Fixed bug where in Py2K a unicode literal would not be accepted as the string name of a class or other argument within declarative using `relationship()`.

### engine

- [engine] [feature] The `engine_from_config()` function has been improved so that we will be able to parse dialect-specific arguments from string configuration dictionaries. Dialect classes can now provide their own list of parameter types and string-conversion routines. The feature is not yet used by the built-in dialects, however.

  References: #2875

- [engine] [bug] A DBAPI that raises an error on `connect()` which is not a subclass of dbapi.Error (such as `TypeError`, `NotImplementedError`, etc.) will propagate the exception unchanged. Previously, the error handling specific to the `connect()` routine would both inappropriately run the exception through the dialect’s `Dialect.is_disconnect()` routine as well as wrap it in a `sqlalchemy.exc.DBAPIError`. It is now propagated unchanged in the same way as occurs within the execute process.

  This change is also backported to: 0.8.4

  References: #2881

- [engine] [bug] [pool] The `QueuePool` has been enhanced to not block new connection attempts when an existing connection attempt is blocking. Previously, the production of new connections was serialized within the block that monitored overflow; the overflow counter is now altered within it’s own critical section outside of the connection process itself.

  This change is also backported to: 0.8.4

  References: #2880

- [engine] [bug] [pool] Made a slight adjustment to the logic which waits for a pooled connection to be available, such that for a connection pool with no timeout specified, it will every half a second break out of the wait to check for the so-called “abort” flag, which allows the waiter to break out in case the whole connection pool was dumped; normally the waiter should break out due to a notify_all() but it’s possible this notify_all() is missed in very slim cases. This is an extension of logic first introduced in 0.8.0, and the issue has only been observed occasionally in stress tests.

  This change is also backported to: 0.8.4

  References: #2522

- [engine] [bug] Fixed bug where SQL statement would be improperly ASCII-encoded when a pre-DBAPI `StatementError` were raised within `Connection.execute()`, causing encoding errors for non-ASCII statements. The stringification now remains within Python unicode thus avoiding encoding errors.

  This change is also backported to: 0.8.4

  References: #2871

- [engine] [bug] The `create_engine()` routine and the related `make_url()` function no longer considers the `+` sign to be a space within the password field. The parsing has been adjuted to match RFC 1738 exactly, in that both `username` and `password` expect only `:`, `@`, and `/` to be encoded.

  See also

  The “password” portion of a create_engine() no longer considers the + sign as an encoded space

  References: #2873

- [engine] [bug] The `RowProxy` object is now sortable in Python as a regular tuple is; this is accomplished via ensuring tuple() conversion on both sides within the `__eq__()` method as well as the addition of a `__lt__()` method.

  See also

  RowProxy now has tuple-sorting behavior

  References: #2848

### sql

- [sql] [feature] New improvements to the `text()` construct, including more flexible ways to set up bound parameters and return types; in particular, a `text()` can now be turned into a full FROM-object, embeddable in other statements as an alias or CTE using the new method `TextClause.columns()`. The `text()` construct can also render “inline” bound parameters when the construct is compiled in a “literal bound” context.

  See also

  New text() Capabilities

  References: #2882, #2877

- [sql] [feature] A new API for specifying the `FOR UPDATE` clause of a `SELECT` is added with the new `GenerativeSelect.with_for_update()` method. This method supports a more straightforward system of setting dialect-specific options compared to the `for_update` keyword argument of `select()`, and also includes support for the SQL standard `FOR UPDATE OF` clause. The ORM also includes a new corresponding method `Query.with_for_update()`. Pull request courtesy Mario Lassnig.

  See also

  New FOR UPDATE support on select(), Query()

  References: pull request github:42

- [sql] [feature] The precision used when coercing a returned floating point value to Python `Decimal` via string is now configurable. The flag `decimal_return_scale` is now supported by all `Numeric` and `Float` types, which will ensure this many digits are taken from the native floating point value when it is converted to string. If not present, the type will make use of the value of `.scale`, if the type supports this setting and it is non-None. Otherwise the original default length of 10 is used.

  See also

  Floating Point String-Conversion Precision Configurable for Native Floating Point Types

  References: #2867

- [sql] [bug] Fixed issue where a primary key column that has a Sequence on it, yet the column is not the “auto increment” column, either because it has a foreign key constraint or `autoincrement=False` set, would attempt to fire the Sequence on INSERT for backends that don’t support sequences, when presented with an INSERT missing the primary key value. This would take place on non-sequence backends like SQLite, MySQL.

  This change is also backported to: 0.8.5

  References: #2896

- [sql] [bug] Fixed bug with `Insert.from_select()` method where the order of the given names would not be taken into account when generating the INSERT statement, thus producing a mismatch versus the column names in the given SELECT statement. Also noted that `Insert.from_select()` implies that Python-side insert defaults cannot be used, since the statement has no VALUES clause.

  This change is also backported to: 0.8.5

  References: #2895

- [sql] [bug] The `cast()` function, when given a plain literal value, will now apply the given type to the given literal value on the bind parameter side according to the type given to the cast, in the same manner as that of the `type_coerce()` function. However unlike `type_coerce()`, this only takes effect if a non-clauseelement value is passed to `cast()`; an existing typed construct will retain its type.

- [sql] [bug] The `ForeignKey` class more aggressively checks the given column argument. If not a string, it checks that the object is at least a `ColumnClause`, or an object that resolves to one, and that the `.table` attribute, if present, refers to a `TableClause` or subclass, and not something like an `Alias`. Otherwise, a `ArgumentError` is raised.

  References: #2883

- [sql] [bug] The precedence rules for the `ColumnOperators.collate()` operator have been modified, such that the COLLATE operator is now of lower precedence than the comparison operators. This has the effect that a COLLATE applied to a comparison will not render parenthesis around the comparison, which is not parsed by backends such as MSSQL. The change is backwards incompatible for those setups that were working around the issue by applying `Operators.collate()` to an individual element of the comparison expression, rather than the comparison expression as a whole.

  See also

  The precedence rules for COLLATE have been changed

  References: #2879

- [sql] [enhancement] The exception raised when a `BindParameter` is present in a compiled statement without a value now includes the key name of the bound parameter in the error message.

  This change is also backported to: 0.8.5

### schema

- [schema] [bug] Fixed a regression caused by #2812 where the repr() for table and column names would fail if the name contained non-ascii characters.

  References: #2868

### postgresql

- [postgresql] [feature] Support for Postgresql JSON has been added, using the new `JSON` type. Huge thanks to Nathan Rice for implementing and testing this.

  References: #2581, pull request github:50

- [postgresql] [feature] Added support for Postgresql TSVECTOR via the `postgresql.TSVECTOR` type. Pull request courtesy Noufal Ibrahim.

  References: pull request bitbucket:8

- [postgresql] [bug] Fixed bug where index reflection would mis-interpret indkey values when using the pypostgresql adapter, which returns these values as lists vs. psycopg2’s return type of string.

  This change is also backported to: 0.8.4

  References: #2855

- [postgresql] [bug] Now using psycopg2 UNICODEARRAY extension for handling unicode arrays with psycopg2 + normal “native unicode” mode, in the same way the UNICODE extension is used.

- [postgresql] [bug] Fixed bug where values within an ENUM weren’t escaped for single quote signs. Note that this is backwards-incompatible for existing workarounds that manually escape the single quotes.

  See also

  Postgresql CREATE TYPE <x> AS ENUM now applies quoting to values

  References: #2878

### mysql

- [mysql] [bug] Improvements to the system by which SQL types generate within `__repr__()`, particularly with regards to the MySQL integer/numeric/ character types which feature a wide variety of keyword arguments. The `__repr__()` is important for use with Alembic autogenerate for when Python code is rendered in a migration script.

  References: #2893

### mssql

- [mssql] [bug] [firebird] The “asdecimal” flag used with the `Float` type will now work with Firebird as well as the mssql+pyodbc dialects; previously the decimal conversion was not occurring.

  This change is also backported to: 0.8.5

- [mssql] [bug] [pymssql] Added “Net-Lib error during Connection reset by peer” message to the list of messages checked for “disconnect” within the pymssql dialect. Courtesy John Anderson.

  This change is also backported to: 0.8.5

  References: pull request github:51

- [mssql] [bug] Fixed bug introduced in 0.8.0 where the `DROP INDEX` statement for an index in MSSQL would render incorrectly if the index were in an alternate schema; the schemaname/tablename would be reversed. The format has been also been revised to match current MSSQL documentation. Courtesy Derek Harland.

  This change is also backported to: 0.8.4

  References: pull request bitbucket:7

### oracle

- [oracle] [bug] Added ORA-02396 “maximum idle time” error code to list of “is disconnect” codes with cx_oracle.

  This change is also backported to: 0.8.4

  References: #2864

- [oracle] [bug] Fixed bug where Oracle `VARCHAR` types given with no length (e.g. for a `CAST` or similar) would incorrectly render `None CHAR` or similar.

  This change is also backported to: 0.8.4

  References: #2870

### firebird

- [firebird] [bug] The firebird dialect will quote identifiers which begin with an underscore. Courtesy Treeve Jelbert.

  This change is also backported to: 0.8.5

  References: #2897

- [firebird] [bug] Fixed bug in Firebird index reflection where the columns within the index were not sorted correctly; they are now sorted in order of RDB$FIELD_POSITION.

  This change is also backported to: 0.8.5

- [firebird] [bug] Changed the queries used by Firebird to list table and view names to query from the `rdb$relations` view instead of the `rdb$relation_fields` and `rdb$view_relations` views. Variants of both the old and new queries are mentioned on many FAQ and blogs, however the new queries are taken straight from the “Firebird FAQ” which appears to be the most official source of info.

  References: #2898

### misc

- [bug] [declarative] Error message when a string arg sent to `relationship()` which doesn’t resolve to a class or mapper has been corrected to work the same way as when a non-string arg is received, which indicates the name of the relationship which had the configurational error.

  This change is also backported to: 0.8.5

  References: #2888

- [bug] [ext] Fixed bug which prevented the `serializer` extension from working correctly with table or column names that contain non-ASCII characters.

  This change is also backported to: 0.8.4

  References: #2869

- [bug] [examples] Fixed bug which prevented history_meta recipe from working with joined inheritance schemes more than one level deep.

- [removed] The “informix” and “informixdb” dialects have been removed; the code is now available as a separate repository on Bitbucket. The IBM-DB project has provided production-level Informix support since the informixdb dialect was first added.

## 0.9.0b1

Released: October 26, 2013

### general

- [general] [feature] [py3k] The C extensions are ported to Python 3 and will build under any supported CPython 2 or 3 environment.

  References: #2161

- [general] [feature] [py3k] The codebase is now “in-place” for Python 2 and 3, the need to run 2to3 has been removed. Compatibility is now against Python 2.6 on forward.

  References: #2671

- [general] A large refactoring of packages has reorganized the import structure of many Core modules as well as some aspects of the ORM modules. In particular `sqlalchemy.sql` has been broken out into several more modules than before so that the very large size of `sqlalchemy.sql.expression` is now pared down. The effort has focused on a large reduction in import cycles. Additionally, the system of API functions in `sqlalchemy.sql.expression` and `sqlalchemy.orm` has been reorganized to eliminate redundancy in documentation between the functions vs. the objects they produce.

### orm

- [orm] [feature] Added new option to `relationship()` `distinct_target_key`. This enables the subquery eager loader strategy to apply a DISTINCT to the innermost SELECT subquery, to assist in the case where duplicate rows are generated by the innermost query which corresponds to this relationship (there’s not yet a general solution to the issue of dupe rows within subquery eager loading, however, when joins outside of the innermost subquery produce dupes). When the flag is set to `True`, the DISTINCT is rendered unconditionally, and when it is set to `None`, DISTINCT is rendered if the innermost relationship targets columns that do not comprise a full primary key. The option defaults to False in 0.8 (e.g. off by default in all cases), None in 0.9 (e.g. automatic by default). Thanks to Alexander Koval for help with this.

  See also

  Subquery Eager Loading will apply DISTINCT to the innermost SELECT for some queries

  This change is also backported to: 0.8.3

  References: #2836

- [orm] [feature] The association proxy now returns `None` when fetching a scalar attribute off of a scalar relationship, where the scalar relationship itself points to `None`, instead of raising an `AttributeError`.

  See also

  Association Proxy Missing Scalar returns None

  References: #2810

- [orm] [feature] Added new method `AttributeState.load_history()`, works like `AttributeState.history` but also fires loader callables.

  See also

  attributes.get_history() will query from the DB by default if value not present

  References: #2787

- [orm] [feature] Added a new load option `orm.load_only()`. This allows a series of column names to be specified as loading “only” those attributes, deferring the rest.

  References: #1418

- [orm] [feature] The system of loader options has been entirely rearchitected to build upon a much more comprehensive base, the `Load` object. This base allows any common loader option like `joinedload()`, `defer()`, etc. to be used in a “chained” style for the purpose of specifying options down a path, such as `joinedload("foo").subqueryload("bar")`. The new system supersedes the usage of dot-separated path names, multiple attributes within options, and the usage of `_all()` options.

  See also

  New Query Options API; load_only() option

  References: #1418

- [orm] [feature] The `composite()` construct now maintains the return object when used in a column-oriented `Query`, rather than expanding out into individual columns. This makes use of the new `Bundle` feature internally. This behavior is backwards incompatible; to select from a composite column which will expand out, use `MyClass.some_composite.clauses`.

  See also

  Composite attributes are now returned as their object form when queried on a per-attribute basis

  References: #2824

- [orm] [feature] A new construct `Bundle` is added, which allows for specification of groups of column expressions to a `Query` construct. The group of columns are returned as a single tuple by default. The behavior of `Bundle` can be overridden however to provide any sort of result processing to the returned row. The behavior of `Bundle` is also embedded into composite attributes now when they are used in a column-oriented `Query`.

  See also

  Column Bundles for ORM queries

  Composite attributes are now returned as their object form when queried on a per-attribute basis

  References: #2824

- [orm] [feature] The `version_id_generator` parameter of `Mapper` can now be specified to rely upon server generated version identifiers, using triggers or other database-provided versioning features, or via an optional programmatic value, by setting `version_id_generator=False`. When using a server-generated version identfier, the ORM will use RETURNING when available to immediately load the new version value, else it will emit a second SELECT.

  References: #2793

- [orm] [feature] The `eager_defaults` flag of `Mapper` will now allow the newly generated default values to be fetched using an inline RETURNING clause, rather than a second SELECT statement, for backends that support RETURNING.

  References: #2793

- [orm] [feature] Added a new attribute `Session.info` to `Session`; this is a dictionary where applications can store arbitrary data local to a `Session`. The contents of `Session.info` can be also be initialized using the `info` argument of `Session` or `sessionmaker`.

- [orm] [feature] Removal of event listeners is now implemented. The feature is provided via the `event.remove()` function.

  See also

  Event Removal API

  References: #2268

- [orm] [feature] The mechanism by which attribute events pass along an `AttributeImpl` as an “initiator” token has been changed; the object is now an event-specific object called `attributes.Event`. Additionally, the attribute system no longer halts events based on a matching “initiator” token; this logic has been moved to be specific to ORM backref event handlers, which are the typical source of the re-propagation of an attribute event onto subsequent append/set/remove operations. End user code which emulates the behavior of backrefs must now ensure that recursive event propagation schemes are halted, if the scheme does not use the backref handlers. Using this new system, backref handlers can now peform a “two-hop” operation when an object is appended to a collection, associated with a new many-to-one, de-associated with the previous many-to-one, and then removed from a previous collection. Before this change, the last step of removal from the previous collection would not occur.

  See also

  Backref handlers can now propagate more than one level deep

  References: #2789

- [orm] [feature] A major change regarding how the ORM constructs joins where the right side is itself a join or left outer join. The ORM is now configured to allow simple nesting of joins of the form `a JOIN (b JOIN c ON b.id=c.id) ON a.id=b.id`, rather than forcing the right side into a `SELECT` subquery. This should allow significant performance improvements on most backends, most particularly MySQL. The one database backend that has for many years held back this change, SQLite, is now addressed by moving the production of the `SELECT` subquery from the ORM to the SQL compiler; so that a right-nested join on SQLite will still ultimately render with a `SELECT`, while all other backends are no longer impacted by this workaround.

  As part of this change, a new argument `flat=True` has been added to the `orm.aliased()`, `Join.alias()`, and `orm.with_polymorphic()` functions, which allows an “alias” of a JOIN to be produced which applies an anonymous alias to each component table within the join, rather than producing a subquery.

  See also

  Many JOIN and LEFT OUTER JOIN expressions will no longer be wrapped in (SELECT * FROM ..) AS ANON_1

  References: #2587

- [orm] [bug] Fixed bug where using an annotation such as `remote()` or `foreign()` on a `Column` before association with a parent `Table` could produce issues related to the parent table not rendering within joins, due to the inherent copy operation performed by an annotation.

  This change is also backported to: 0.8.3

  References: #2813

- [orm] [bug] Fixed bug where `Query.exists()` failed to work correctly without any WHERE criterion. Courtesy Vladimir Magamedov.

  This change is also backported to: 0.8.3

  References: #2818

- [orm] [bug] Fixed a potential issue in an ordered sequence implementation used by the ORM to iterate mapper hierarchies; under the Jython interpreter this implementation wasn’t ordered, even though cPython and Pypy maintained ordering.

  This change is also backported to: 0.8.3

  References: #2794

- [orm] [bug] Fixed bug in ORM-level event registration where the “raw” or “propagate” flags could potentially be mis-configured in some “unmapped base class” configurations.

  This change is also backported to: 0.8.3

  References: #2786

- [orm] [bug] A performance fix related to the usage of the `defer()` option when loading mapped entities. The function overhead of applying a per-object deferred callable to an instance at load time was significantly higher than that of just loading the data from the row (note that `defer()` is meant to reduce DB/network overhead, not necessarily function call count); the function call overhead is now less than that of loading data from the column in all cases. There is also a reduction in the number of “lazy callable” objects created per load from N (total deferred values in the result) to 1 (total number of deferred cols).

  This change is also backported to: 0.8.3

  References: #2778

- [orm] [bug] Fixed bug whereby attribute history functions would fail when an object we moved from “persistent” to “pending” using the `make_transient()` function, for operations involving collection-based backrefs.

  This change is also backported to: 0.8.3

  References: #2773

- [orm] [bug] A warning is emitted when trying to flush an object of an inherited class where the polymorphic discriminator has been assigned to a value that is invalid for the class.

  This change is also backported to: 0.8.2

  References: #2750

- [orm] [bug] Fixed bug in polymorphic SQL generation where multiple joined-inheritance entities against the same base class joined to each other as well would not track columns on the base table independently of each other if the string of joins were more than two entities long.

  This change is also backported to: 0.8.2

  References: #2759

- [orm] [bug] Fixed bug where sending a composite attribute into `Query.order_by()` would produce a parenthesized expression not accepted by some databases.

  This change is also backported to: 0.8.2

  References: #2754

- [orm] [bug] Fixed the interaction between composite attributes and the `aliased()` function. Previously, composite attributes wouldn’t work correctly in comparison operations when aliasing was applied.

  This change is also backported to: 0.8.2

  References: #2755

- [orm] [bug] [ext] Fixed bug where `MutableDict` didn’t report a change event when `clear()` was called.

  This change is also backported to: 0.8.2

  References: #2730

- [orm] [bug] Fixed bug where list instrumentation would fail to represent a setslice of `[0:0]` correctly, which in particular could occur when using `insert(0, item)` with the association proxy. Due to some quirk in Python collections, the issue was much more likely with Python 3 rather than 2.

  This change is also backported to: 0.8.3, 0.7.11

  References: #2807

- [orm] [bug] `attributes.get_history()` when used with a scalar column-mapped attribute will now honor the “passive” flag passed to it; as this defaults to `PASSIVE_OFF`, the function will by default query the database if the value is not present. This is a behavioral change vs. 0.8.

  See also

  attributes.get_history() will query from the DB by default if value not present

  References: #2787

- [orm] [bug] [associationproxy] Added additional criterion to the ==, != comparators, used with scalar values, for comparisons to None to also take into account the association record itself being non-present, in addition to the existing test for the scalar endpoint on the association record being NULL. Previously, comparing `Cls.scalar == None` would return records for which `Cls.associated` were present and `Cls.associated.scalar` is None, but not rows for which `Cls.associated` is non-present. More significantly, the inverse operation `Cls.scalar != None` would return `Cls` rows for which `Cls.associated` was non-present.

  The case for `Cls.scalar != 'somevalue'` is also modified to act more like a direct SQL comparison; only rows for which `Cls.associated` is present and `Associated.scalar` is non-NULL and not equal to `'somevalue'` are returned. Previously, this would be a simple `NOT EXISTS`.

  Also added a special use case where you can call `Cls.scalar.has()` with no arguments, when `Cls.scalar` is a column-based value - this returns whether or not `Cls.associated` has any rows present, regardless of whether or not `Cls.associated.scalar` is NULL or not.

  See also

  Association Proxy SQL Expression Improvements and Fixes

  References: #2751

- [orm] [bug] Fixed an obscure bug where the wrong results would be fetched when joining/joinedloading across a many-to-many relationship to a single-table-inheriting subclass with a specific discriminator value, due to “secondary” rows that would come back. The “secondary” and right-side tables are now inner joined inside of parenthesis for all ORM joins on many-to-many relationships so that the left->right join can accurately filtered. This change was made possible by finally addressing the issue with right-nested joins outlined in #2587.

  See also

  Many JOIN and LEFT OUTER JOIN expressions will no longer be wrapped in (SELECT * FROM ..) AS ANON_1

  References: #2369

- [orm] [bug] The “auto-aliasing” behavior of the `Query.select_from()` method has been turned off. The specific behavior is now availble via a new method `Query.select_entity_from()`. The auto-aliasing behavior here was never well documented and is generally not what’s desired, as `Query.select_from()` has become more oriented towards controlling how a JOIN is rendered. `Query.select_entity_from()` will also be made available in 0.8 so that applications which rely on the auto-aliasing can shift their applications to use this method.

  See also

  Query.select_from() no longer applies the clause to corresponding entities

  References: #2736

### orm declarative

- [feature] [orm] [declarative] Added a convenience class decorator `as_declarative()`, is a wrapper for `declarative_base()` which allows an existing base class to be applied using a nifty class-decorated approach.

  This change is also backported to: 0.8.3

- [feature] [orm] [declarative] ORM descriptors such as hybrid properties can now be referenced by name in a string argument used with `order_by`, `primaryjoin`, or similar in `relationship()`, in addition to column-bound attributes.

  This change is also backported to: 0.8.2

  References: #2761

### engine

- [engine] [feature] `repr()` for the `URL` of an `Engine` will now conceal the password using asterisks. Courtesy Gunnlaugur Þór Briem.

  This change is also backported to: 0.8.3

  References: #2821

- [engine] [feature] New events added to `ConnectionEvents`:

  - `ConnectionEvents.engine_connect()`
  - `ConnectionEvents.set_connection_execution_options()`
  - `ConnectionEvents.set_engine_execution_options()`

  References: #2770

- [engine] [bug] [oracle] Dialect.initialize() is not called a second time if an `Engine` is recreated, due to a disconnect error. This fixes a particular issue in the Oracle 8 dialect, but in general the dialect.initialize() phase should only be once per dialect.

  This change is also backported to: 0.8.3

  References: #2776

- [engine] [bug] [pool] Fixed bug where `QueuePool` would lose the correct checked out count if an existing pooled connection failed to reconnect after an invalidate or recycle event.

  This change is also backported to: 0.8.3

  References: #2772

- [engine] [bug] Fixed bug where the `reset_on_return` argument to various `Pool` implementations would not be propagated when the pool was regenerated. Courtesy Eevee.

  This change is also backported to: 0.8.2

  References: pull request github:6

- [engine] [bug] The regexp used by the `make_url()` function now parses ipv6 addresses, e.g. surrounded by brackets.

  This change is also backported to: 0.8.3, 0.7.11

  References: #2851

- [engine] [bug] The method signature of `Dialect.reflecttable()`, which in all known cases is provided by `DefaultDialect`, has been tightened to expect `include_columns` and `exclude_columns` arguments without any kw option, reducing ambiguity - previously `exclude_columns` was missing.

  References: #2748

### sql

- [sql] [feature] Added support for “unique constraint” reflection, via the `Inspector.get_unique_constraints()` method. Thanks for Roman Podolyaka for the patch.

  This change is also backported to: 0.8.4

  References: #1443

- [sql] [feature] The `update()`, `insert()`, and `delete()` constructs will now interpret ORM entities as target tables to be operated upon, e.g.:

      from sqlalchemy import insert, update, delete

      ins = insert(SomeMappedClass).values(x=5)

      del_ = delete(SomeMappedClass).where(SomeMappedClass.id == 5)

      upd = update(SomeMappedClass).where(SomeMappedClass.id == 5).values(name='ed')

  This change is also backported to: 0.8.3

- [sql] [feature] [postgresql] [mysql] The Postgresql and MySQL dialects now support reflection/inspection of foreign key options, including ON UPDATE, ON DELETE. Postgresql also reflects MATCH, DEFERRABLE, and INITIALLY. Coutesy ijl.

  References: #2183

- [sql] [feature] A `bindparam()` construct with a “null” type (e.g. no type specified) is now copied when used in a typed expression, and the new copy is assigned the actual type of the compared column. Previously, this logic would occur on the given `bindparam()` in place. Additionally, a similar process now occurs for `bindparam()` constructs passed to `ValuesBase.values()` for an `Insert` or `Update` construct, within the compilation phase of the construct.

  These are both subtle behavioral changes which may impact some usages.

  See also

  A bindparam() construct with no type gets upgraded via copy when a type is available

  References: #2850

- [sql] [feature] An overhaul of expression handling for special symbols particularly with conjunctions, e.g. `None` `expression.null()` `expression.true()` `expression.false()`, including consistency in rendering NULL in conjunctions, “short-circuiting” of `and_()` and `or_()` expressions which contain boolean constants, and rendering of boolean constants and expressions as compared to “1” or “0” for backends that don’t feature `true`/`false` constants.

  See also

  Improved rendering of Boolean constants, NULL constants, conjunctions

  References: #2734, #2804, #2823

- [sql] [feature] The typing system now handles the task of rendering “literal bind” values, e.g. values that are normally bound parameters but due to context must be rendered as strings, typically within DDL constructs such as CHECK constraints and indexes (note that “literal bind” values become used by DDL as of #2742). A new method `TypeEngine.literal_processor()` serves as the base, and `TypeDecorator.process_literal_param()` is added to allow wrapping of a native literal rendering method.

  See also

  The typing system now handles the task of rendering “literal bind” values

  References: #2838

- [sql] [feature] The `Table.tometadata()` method now produces copies of all `SchemaItem.info` dictionaries from all `SchemaItem` objects within the structure including columns, constraints, foreign keys, etc. As these dictionaries are copies, they are independent of the original dictionary. Previously, only the `.info` dictionary of `Column` was transferred within this operation, and it was only linked in place, not copied.

  References: #2716

- [sql] [feature] The `default` argument of `Column` now accepts a class or object method as an argument, in addition to a standalone function; will properly detect if the “context” argument is accepted or not.

- [sql] [feature] Added new method to the `insert()` construct `Insert.from_select()`. Given a list of columns and a selectable, renders `INSERT INTO (table) (columns) SELECT ..`. While this feature is highlighted as part of 0.9 it is also backported to 0.8.3.

  See also

  INSERT from SELECT

  References: #722

- [sql] [feature] Provided a new attribute for `TypeDecorator` called `TypeDecorator.coerce_to_is_types`, to make it easier to control how comparisons using `==` or `!=` to `None` and boolean types goes about producing an `IS` expression, or a plain equality expression with a bound parameter.

  References: #2734, #2744

- [sql] [feature] A `label()` construct will now render as its name alone in an `ORDER BY` clause, if that label is also referred to in the columns clause of the select, instead of rewriting the full expression. This gives the database a better chance to optimize the evaulation of the same expression in two different contexts.

  See also

  Label constructs can now render as their name alone in an ORDER BY

  References: #1068

- [sql] [bug] Fixed bug where `type_coerce()` would not interpret ORM elements with a `__clause_element__()` method properly.

  This change is also backported to: 0.8.3

  References: #2849

- [sql] [bug] The `Enum` and `Boolean` types now bypass any custom (e.g. TypeDecorator) type in use when producing the CHECK constraint for the “non native” type. This so that the custom type isn’t involved in the expression within the CHECK, since this expression is against the “impl” value and not the “decorated” value.

  This change is also backported to: 0.8.3

  References: #2842

- [sql] [bug] The `.unique` flag on `Index` could be produced as `None` if it was generated from a `Column` that didn’t specify `unique` (where it defaults to `None`). The flag will now always be `True` or `False`.

  This change is also backported to: 0.8.3

  References: #2825

- [sql] [bug] Fixed bug in default compiler plus those of postgresql, mysql, and mssql to ensure that any literal SQL expression values are rendered directly as literals, instead of as bound parameters, within a CREATE INDEX statement. This also changes the rendering scheme for other DDL such as constraints.

  This change is also backported to: 0.8.3

  References: #2742

- [sql] [bug] A `select()` that is made to refer to itself in its FROM clause, typically via in-place mutation, will raise an informative error message rather than causing a recursion overflow.

  This change is also backported to: 0.8.3

  References: #2815

- [sql] [bug] Fixed bug where using the `column_reflect` event to change the `.key` of the incoming `Column` would prevent primary key constraints, indexes, and foreign key constraints from being correctly reflected.

  This change is also backported to: 0.8.3

  References: #2811

- [sql] [bug] The `ColumnOperators.notin_()` operator added in 0.8 now properly produces the negation of the expression “IN” returns when used against an empty collection.

  This change is also backported to: 0.8.3

- [sql] [bug] [postgresql] Fixed bug where the expression system relied upon the `str()` form of a some expressions when referring to the `.c` collection on a `select()` construct, but the `str()` form isn’t available since the element relies on dialect-specific compilation constructs, notably the `__getitem__()` operator as used with a Postgresql `ARRAY` element. The fix also adds a new exception class `UnsupportedCompilationError` which is raised in those cases where a compiler is asked to compile something it doesn’t know how to.

  This change is also backported to: 0.8.3

  References: #2780

- [sql] [bug] Multiple fixes to the correlation behavior of `Select` constructs, first introduced in 0.8.0:

  - To satisfy the use case where FROM entries should be correlated outwards to a SELECT that encloses another, which then encloses this one, correlation now works across multiple levels when explicit correlation is established via `Select.correlate()`, provided that the target select is somewhere along the chain contained by a WHERE/ORDER BY/columns clause, not just nested FROM clauses. This makes `Select.correlate()` act more compatibly to that of 0.7 again while still maintaining the new “smart” correlation.
  - When explicit correlation is not used, the usual “implicit” correlation limits its behavior to just the immediate enclosing SELECT, to maximize compatibility with 0.7 applications, and also prevents correlation across nested FROMs in this case, maintaining compatibility with 0.8.0/0.8.1.
  - The `Select.correlate_except()` method was not preventing the given FROM clauses from correlation in all cases, and also would cause FROM clauses to be incorrectly omitted entirely (more like what 0.7 would do), this has been fixed.
  - Calling select.correlate_except(None) will enter all FROM clauses into correlation as would be expected.

  This change is also backported to: 0.8.2

  References: #2668, #2746

- [sql] [bug] Fixed bug whereby joining a select() of a table “A” with multiple foreign key paths to a table “B”, to that table “B”, would fail to produce the “ambiguous join condition” error that would be reported if you join table “A” directly to “B”; it would instead produce a join condition with multiple criteria.

  This change is also backported to: 0.8.2

  References: #2738

- [sql] [bug] [reflection] Fixed bug whereby using `MetaData.reflect()` across a remote schema as well as a local schema could produce wrong results in the case where both schemas had a table of the same name.

  This change is also backported to: 0.8.2

  References: #2728

- [sql] [bug] Removed the “not implemented” `__iter__()` call from the base `ColumnOperators` class, while this was introduced in 0.8.0 to prevent an endless, memory-growing loop when one also implements a `__getitem__()` method on a custom operator and then calls erroneously `list()` on that object, it had the effect of causing column elements to report that they were in fact iterable types which then throw an error when you try to iterate. There’s no real way to have both sides here so we stick with Python best practices. Careful with implementing `__getitem__()` on your custom operators!

  This change is also backported to: 0.8.2

  References: #2726

- [sql] [bug] Fixed regression dating back to 0.7.9 whereby the name of a CTE might not be properly quoted if it was referred to in multiple FROM clauses.

  This change is also backported to: 0.8.3, 0.7.11

  References: #2801

- [sql] [bug] [cte] Fixed bug in common table expression system where if the CTE were used only as an `alias()` construct, it would not render using the WITH keyword.

  This change is also backported to: 0.8.3, 0.7.11

  References: #2783

- [sql] [bug] Fixed bug in `CheckConstraint` DDL where the “quote” flag from a `Column` object would not be propagated.

  This change is also backported to: 0.8.3, 0.7.11

  References: #2784

- [sql] [bug] The “name” attribute is set on `Index` before the “attach” events are called, so that attachment events can be used to dynamically generate a name for the index based on the parent table and/or columns.

  References: #2835

- [sql] [bug] The erroneous kw arg “schema” has been removed from the `ForeignKey` object. this was an accidental commit that did nothing; a warning is raised in 0.8.3 when this kw arg is used.

  References: #2831

- [sql] [bug] A rework to the way that “quoted” identifiers are handled, in that instead of relying upon various `quote=True` flags being passed around, these flags are converted into rich string objects with quoting information included at the point at which they are passed to common schema constructs like `Table`, `Column`, etc. This solves the issue of various methods that don’t correctly honor the “quote” flag such as `Engine.has_table()` and related methods. The `quoted_name` object is a string subclass that can also be used explicitly if needed; the object will hold onto the quoting preferences passed and will also bypass the “name normalization” performed by dialects that standardize on uppercase symbols, such as Oracle, Firebird and DB2. The upshot is that the “uppercase” backends can now work with force-quoted names, such as lowercase-quoted names and new reserved words.

  See also

  Schema identifiers now carry along their own quoting information

  References: #2812

- [sql] [bug] The resolution of `ForeignKey` objects to their target `Column` has been reworked to be as immediate as possible, based on the moment that the target `Column` is associated with the same `MetaData` as this `ForeignKey`, rather than waiting for the first time a join is constructed, or similar. This along with other improvements allows earlier detection of some foreign key configuration issues. Also included here is a rework of the type-propagation system, so that it should be reliable now to set the type as `None` on any `Column` that refers to another via `ForeignKey` - the type will be copied from the target column as soon as that other column is associated, and now works for composite foreign keys as well.

  See also

  Columns can reliably get their type from a column referred to via ForeignKey

  References: #1765

### postgresql

- [postgresql] [feature] Support for Postgresql 9.2 range types has been added. Currently, no type translation is provided, so works directly with strings or psycopg2 2.5 range extension types at the moment. Patch courtesy Chris Withers.

  This change is also backported to: 0.8.2

- [postgresql] [feature] Added support for “AUTOCOMMIT” isolation when using the psycopg2 DBAPI. The keyword is available via the `isolation_level` execution option. Patch courtesy Roman Podolyaka.

  This change is also backported to: 0.8.2

  References: #2072

- [postgresql] [feature] Added support for rendering `SMALLSERIAL` when a `SmallInteger` type is used on a primary key autoincrement column, based on server version detection of Postgresql version 9.2 or greater.

  References: #2840

- [postgresql] [bug] Removed a 128-character truncation from the reflection of the server default for a column; this code was original from PG system views which truncated the string for readability.

  This change is also backported to: 0.8.3

  References: #2844

- [postgresql] [bug] Parenthesis will be applied to a compound SQL expression as rendered in the column list of a CREATE INDEX statement.

  This change is also backported to: 0.8.3

  References: #2742

- [postgresql] [bug] Fixed bug where Postgresql version strings that had a prefix preceding the words “Postgresql” or “EnterpriseDB” would not parse. Courtesy Scott Schaefer.

  This change is also backported to: 0.8.3

  References: #2819

- [postgresql] [bug] The behavior of `extract()` has been simplified on the Postgresql dialect to no longer inject a hardcoded `::timestamp` or similar cast into the given expression, as this interfered with types such as timezone-aware datetimes, but also does not appear to be at all necessary with modern versions of psycopg2.

  This change is also backported to: 0.8.2

  References: #2740

- [postgresql] [bug] Fixed bug in HSTORE type where keys/values that contained backslashed quotes would not be escaped correctly when using the “non native” (i.e. non-psycopg2) means of translating HSTORE data. Patch courtesy Ryan Kelly.

  This change is also backported to: 0.8.2

  References: #2766

- [postgresql] [bug] Fixed bug where the order of columns in a multi-column Postgresql index would be reflected in the wrong order. Courtesy Roman Podolyaka.

  This change is also backported to: 0.8.2

  References: #2767

### mysql

- [mysql] [feature] The `mysql_length` parameter used with `Index` can now be passed as a dictionary of column names/lengths, for use with composite indexes. Big thanks to Roman Podolyaka for the patch.

  This change is also backported to: 0.8.2

  References: #2704

- [mysql] [feature] The MySQL `mysql.SET` type now features the same auto-quoting behavior as that of `mysql.ENUM`. Quotes are not required when setting up the value, but quotes that are present will be auto-detected along with a warning. This also helps with Alembic where the SET type doesn’t render with quotes.

  References: #2817

- [mysql] [bug] The change in #2721, which is that the `deferrable` keyword of `ForeignKeyConstraint` is silently ignored on the MySQL backend, will be reverted as of 0.9; this keyword will now render again, raising errors on MySQL as it is not understood - the same behavior will also apply to the `initially` keyword. In 0.8, the keywords will remain ignored but a warning is emitted. Additionally, the `match` keyword now raises a `CompileError` on 0.9 and emits a warning on 0.8; this keyword is not only silently ignored by MySQL but also breaks the ON UPDATE/ON DELETE options.

  To use a `ForeignKeyConstraint` that does not render or renders differently on MySQL, use a custom compilation option. An example of this usage has been added to the documentation, see MySQL Foreign Keys.

  This change is also backported to: 0.8.3

  References: #2721, #2839

- [mysql] [bug] MySQL-connector dialect now allows options in the create_engine query string to override those defaults set up in the connect, including “buffered” and “raise_on_warnings”.

  This change is also backported to: 0.8.3

  References: #2515

- [mysql] [bug] Fixed bug when using multi-table UPDATE where a supplemental table is a SELECT with its own bound parameters, where the positioning of the bound parameters would be reversed versus the statement itself when using MySQL’s special syntax.

  This change is also backported to: 0.8.2

  References: #2768

- [mysql] [bug] Added another conditional to the `mysql+gaerdbms` dialect to detect so-called “development” mode, where we should use the `rdbms_mysqldb` DBAPI. Patch courtesy Brett Slatkin.

  This change is also backported to: 0.8.2

  References: #2715

- [mysql] [bug] The `deferrable` keyword argument on `ForeignKey` and `ForeignKeyConstraint` will not render the `DEFERRABLE` keyword on the MySQL dialect. For a long time we left this in place because a non-deferrable foreign key would act very differently than a deferrable one, but some environments just disable FKs on MySQL, so we’ll be less opinionated here.

  This change is also backported to: 0.8.2

  References: #2721

- [mysql] [bug] Updates to MySQL reserved words for versions 5.5, 5.6, courtesy Hanno Schlichting.

  This change is also backported to: 0.8.3, 0.7.11

  References: #2791

- [mysql] [bug] Fix and test parsing of MySQL foreign key options within reflection; this complements the work in #2183 where we begin to support reflection of foreign key options such as ON UPDATE/ON DELETE cascade.

  References: #2839

- [mysql] [bug] Improved support for the cymysql driver, supporting version 0.6.5, courtesy Hajime Nakagami.

### sqlite

- [sqlite] [bug] The newly added SQLite DATETIME arguments storage_format and regexp apparently were not fully implemented correctly; while the arguments were accepted, in practice they would have no effect; this has been fixed.

  This change is also backported to: 0.8.3

  References: #2781

- [sqlite] [bug] Added `sqlalchemy.types.BIGINT` to the list of type names that can be reflected by the SQLite dialect; courtesy Russell Stuart.

  This change is also backported to: 0.8.2

  References: #2764

### mssql

- [mssql] [bug] When querying the information schema on SQL Server 2000, removed a CAST call that was added in 0.8.1 to help with driver issues, which apparently is not compatible on 2000. The CAST remains in place for SQL Server 2005 and greater.

  This change is also backported to: 0.8.2

  References: #2747

- [mssql] [bug] [pyodbc] Fixes to MSSQL with Python 3 + pyodbc, including that statements are passed correctly.

  References: #2355

### oracle

- [oracle] [feature] [py3k] The Oracle unit tests with cx_oracle now pass fully under Python 3.

- [oracle] [bug] Fixed bug where Oracle table reflection using synonyms would fail if the synonym and the table were in different remote schemas. Patch to fix courtesy Kyle Derr.

  This change is also backported to: 0.8.3

  References: #2853

### firebird

- [firebird] [feature] Added new flag `retaining=True` to the kinterbasdb and fdb dialects. This controls the value of the `retaining` flag sent to the `commit()` and `rollback()` methods of the DBAPI connection. Due to historical concerns, this flag defaults to `True` in 0.8.2, however in 0.9.0b1 this flag defaults to `False`.

  This change is also backported to: 0.8.2

  References: #2763

- [firebird] [feature] The `fdb` dialect is now the default dialect when specified without a dialect qualifier, i.e. `firebird://`, per the Firebird project publishing `fdb` as their official Python driver.

  References: #2504

- [firebird] [bug] Type lookup when reflecting the Firebird types LONG and INT64 has been fixed so that LONG is treated as INTEGER, INT64 treated as BIGINT, unless the type has a “precision” in which case it’s treated as NUMERIC. Patch courtesy Russell Stuart.

  This change is also backported to: 0.8.2

  References: #2757

### misc

- [feature] Added a new flag `system=True` to `Column`, which marks the column as a “system” column which is automatically made present by the database (such as Postgresql `oid` or `xmin`). The column will be omitted from the `CREATE TABLE` statement but will otherwise be available for querying. In addition, the `CreateColumn` construct can be appled to a custom compilation rule which allows skipping of columns, by producing a rule that returns `None`.

  This change is also backported to: 0.8.3

- [feature] [examples] Improved the examples in `examples/generic_associations`, including that `discriminator_on_association.py` makes use of single table inheritance do the work with the “discriminator”. Also added a true “generic foreign key” example, which works similarly to other popular frameworks in that it uses an open-ended integer to point to any other table, foregoing traditional referential integrity. While we don’t recommend this pattern, information wants to be free.

  This change is also backported to: 0.8.3

- [feature] [core] Added a new variant to `UpdateBase.returning()` called `ValuesBase.return_defaults()`; this allows arbitrary columns to be added to the RETURNING clause of the statement without interfering with the compilers usual “implicit returning” feature, which is used to efficiently fetch newly generated primary key values. For supporting backends, a dictionary of all fetched values is present at `ResultProxy.returned_defaults`.

  References: #2793

- [feature] [pool] Added pool logging for “rollback-on-return” and the less used “commit-on-return”. This is enabled with the rest of pool “debug” logging.

  References: #2752

- [bug] [examples] Added “autoincrement=False” to the history table created in the versioning example, as this table shouldn’t have autoinc on it in any case, courtesy Patrick Schmid.

  This change is also backported to: 0.8.3

- [bug] [ext] Fixed bug whereby if a composite type were set up with a function instead of a class, the mutable extension would trip up when it tried to check that column for being a `MutableComposite` (which it isn’t). Courtesy asldevi.

  This change is also backported to: 0.8.2

- [bug] [examples] Fixed an issue with the “versioning” recipe whereby a many-to-one reference could produce a meaningless version for the target, even though it was not changed, when backrefs were present. Patch courtesy Matt Chisholm.

  This change is also backported to: 0.8.2

- [requirements] The Python mock library is now required in order to run the unit test suite. While part of the standard library as of Python 3.3, previous Python installations will need to install this in order to run unit tests or to use the `sqlalchemy.testing` package for external dialects.

  This change is also backported to: 0.8.2