dabo Commit
Revision 7278
Date: 2012-10-13 11:39:16 -0700 (Sat, 13 Oct 2012)
Author: Nate
Trac: http://trac.dabodev.com/changeset/7278
Changed:
U trunk/dabo/biz/dBizobj.py
U trunk/dabo/db/dBackend.py
U trunk/dabo/db/dConnectInfo.py
U trunk/dabo/db/dConnection.py
U trunk/dabo/db/dbMySQL.py
Log:
MySQL will auto-close connections after a certain period. If you try to execute a statement, it will through a 2006 operational error. You can call the connections ping method to reestablish the connection and try to execute again. I wanted to overwrite the standard dabo cursor (dCursorMixin) with a MySQL specific cursor. However this wasn't possible without a framework change. We already had a facility in place to grab the specific backend dbapi cursor class from the dBackend object. I extended that model so that instead of hardcoding the main dCursorMixin class in the object we can grab it from the dBackend object. It will auto-default to dCursorMixin if we don't override it in the subclass.
Tested with MySQL and the SQLite adapter and it works. We should really consider moving the CreateCursor and _getCursorClass methods on dBizobj down into the db layer?\226?\128?\166
Diff:
Modified: trunk/dabo/biz/dBizobj.py
===================================================================
--- trunk/dabo/biz/dBizobj.py 2012-10-11 19:13:53 UTC (rev 7277)
+++ trunk/dabo/biz/dBizobj.py 2012-10-13 18:39:16 UTC (rev 7278)
@@ -21,8 +21,6 @@
class dBizobj(dObject):
""" The middle tier, where the business logic resides."""
- # Class to instantiate for the cursor object
- dCursorMixinClass = dCursorMixin
# Tell dObject that we'll call before and afterInit manually:
_call_beforeInit, _call_afterInit, _call_initProperties = False, False, False
@@ -131,6 +129,7 @@
if conn:
# Base cursor class : the cursor class from the db api
self.dbapiCursorClass = self._cursorFactory.getDictCursorClass()
+ self.dCursorMixinClass = self._cursorFactory.getMainCursorClass()
# If there are any problems in the createCursor process, an
# exception will be raised in that method.
self.createCursor()
@@ -459,7 +458,7 @@
# in some out-of-context child cursor, but that should be rare. In the
# common case, all the changes would have already been made in the above
# block, and isAnyChanged() will return False very quickly in that case.
- if self.isAnyChanged():
+ if self.isAnyChanged():
try:
self.scan(self.save, startTransaction=False,
saveTheChildren=saveTheChildren, scanRequeryChildren=False)
@@ -561,7 +560,7 @@
# in some out-of-context child cursor, but that should be rare. In the
# common case, all the cancellations would have already happened in the
# above block, and isAnyChanged() will return False very quickly.
- if self.isAnyChanged():
+ if self.isAnyChanged():
self.scanChangedRows(self.cancel, allCursors=False,
includeNewUnchanged=True, cancelTheChildren=cancelTheChildren,
ignoreNoRecords=ignoreNoRecords, reverse=True)
@@ -1313,7 +1312,7 @@
If the operator is specified, it will be used literally in the evaluation instead of the
equals sign, unless it is one of the following strings, which will be interpreted
as indicated:
-
+
| eq, equals: =
| ne, nequals: !=
| gt: >
@@ -1355,9 +1354,9 @@
"""Allows you to filter by any valid Python expression.
Use the field alias names, for example::
-
+
biz.filterByExpression('cust_name[0].lower() = 'a')
-
+
where cust_name is a field alias name in this record.
"""
self._CurrentCursor.filterByExpression(expr)
@@ -1444,7 +1443,7 @@
an error message that describes the reason why the data is not
valid; this message will be propagated back up to the UI where it can
be displayed to the user so that they can correct the problem.
-
+
Example::
if not myNonEmptyField:
@@ -1515,7 +1514,7 @@
For internal use only! Should never be called from a developer's code.
Its purpose is to keep child cursor in sync with parent cursor.
The updateChildren parameter meaning:
-
+
| None - the fastest one, doesn't update parent nor requery child cursor
| False - update child cursor with current parent
| True - do both, update child cursor's parent and requery child cursor.
@@ -1598,7 +1597,7 @@
If incremental is True (default is False), then we only compare the first
characters up until the length of val.
-
+
Returns the RowNumber of the found record, or -1 if no match found.
"""
ret = self._CurrentCursor.seek(val, fld, caseSensitive, near,
@@ -2117,7 +2116,7 @@
"""
Gets the structure of the DataSource table. Returns a list
of 3-tuples, where the 3-tuple's elements are:
-
+
| 0: the field name (string)
| 1: the field type ('I', 'N', 'C', 'M', 'B', 'D', 'T')
| 2: boolean specifying whether this is a pk field.
@@ -2129,7 +2128,7 @@
"""
Gets the structure of the DataSource table. Returns a list
of 3-tuples, where the 3-tuple's elements are:
-
+
| 0: the field name (string)
| 1: the field type ('I', 'N', 'C', 'M', 'B', 'D', 'T')
| 2: boolean specifying whether this is a pk field.
@@ -2576,7 +2575,7 @@
Hook method called after a field's value has been set.
Your hook method needs to accept two arguments:
-
+
| -> fld : The name of the changed field.
| -> row : The RowNumber of the changed field.
@@ -3105,7 +3104,7 @@
_("""If set, treated as cursor real table name where DataSource
is an alias for it. This allows coexistence of many business objects
with same data source on single form. (str)
-
+
Example:
class StockBase(dBizobj):
def initProperties(self):
Modified: trunk/dabo/db/dBackend.py
===================================================================
--- trunk/dabo/db/dBackend.py 2012-10-11 19:13:53 UTC (rev 7277)
+++ trunk/dabo/db/dBackend.py 2012-10-13 18:39:16 UTC (rev 7278)
@@ -12,9 +12,9 @@
from dabo.db import dTable
from dNoEscQuoteStr import dNoEscQuoteStr
from dabo.lib.utils import ustr
+from dCursorMixin import dCursorMixin
-
class dBackend(dObject):
"""Abstract class inherited by the specific Dabo database connectors."""
# Pattern for determining if a function is present in a string
@@ -53,7 +53,11 @@
"""override in subclasses"""
return None
+ def getMainCursorClass(self):
+ """override in subclasses if they need something other than dCursorMixin"""
+ return dCursorMixin
+
def getCursor(self, cursorClass):
"""override in subclasses if necessary"""
return cursorClass(self._connection)
Modified: trunk/dabo/db/dConnectInfo.py
===================================================================
--- trunk/dabo/db/dConnectInfo.py 2012-10-11 19:13:53 UTC (rev 7277)
+++ trunk/dabo/db/dConnectInfo.py 2012-10-13 18:39:16 UTC (rev 7278)
@@ -86,6 +86,13 @@
return None
+ def getMainCursorClass(self):
+ try:
+ return self._backendObject.getMainCursorClass()
+ except TypeError:
+ return None
+
+
def encrypt(self, val):
if self.Application:
return self.Application.encrypt(val)
Modified: trunk/dabo/db/dConnection.py
===================================================================
--- trunk/dabo/db/dConnection.py 2012-10-11 19:13:53 UTC (rev 7277)
+++ trunk/dabo/db/dConnection.py 2012-10-13 18:39:16 UTC (rev 7278)
@@ -44,6 +44,10 @@
return self._connectInfo.getDictCursorClass()
+ def getMainCursorClass(self):
+ return self._connectInfo.getMainCursorClass()
+
+
def getCursor(self, cursorClass):
return self.getBackendObject().getCursor(cursorClass)
Modified: trunk/dabo/db/dbMySQL.py
===================================================================
--- trunk/dabo/db/dbMySQL.py 2012-10-11 19:13:53 UTC (rev 7277)
+++ trunk/dabo/db/dbMySQL.py 2012-10-13 18:39:16 UTC (rev 7278)
@@ -7,9 +7,19 @@
import dabo.dException as dException
from dNoEscQuoteStr import dNoEscQuoteStr as dNoEQ
from dabo.lib.utils import ustr
+from dCursorMixin import dCursorMixin
+class MySQLAutoReconnectCursor(dCursorMixin):
+ def execute(self, sql, params=None, errorClass=None, convertQMarks=False):
+ from MySQLdb import OperationalError
+ try:
+ super(MySQLAutoReconnectCursor, self).execute(sql, params=params, errorClass=OperationalError, convertQMarks=convertQMarks)
+ except OperationalError:
+ self.connection.ping(True)
+ super(MySQLAutoReconnectCursor, self).execute(sql, params=params, errorClass=None, convertQMarks=convertQMarks)
+
class MySQL(dBackend):
"""Class providing MySQL connectivity. Uses MySQLdb."""
@@ -79,10 +89,19 @@
def getDictCursorClass(self):
- import MySQLdb.cursors as cursors
- return cursors.DictCursor
+ return MySQLdb.cursors.DictCursor
+ def getMainCursorClass(self):
+ return MySQLAutoReconnectCursor
+ #def getCursor(self, CursorClass):
+ # raise
+ # print "CursorClass is: %s" % CursorClass
+ # if CursorClass == dCrusorMixin:
+ # return MySQLAutoReconnectCursor(self._connection)
+ # else:
+ # return CursorClass(self._connection)
+
def beginTransaction(self, cursor):
""" Begin a SQL transaction."""
cursor.execute("START TRANSACTION")
_______________________________________________
Post Messages to: Dabo-dev@mail.leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/20121013183917.73867318B0B@mail.paulmcnett.com
Użytkownik Nate Lowrie napisał:
> ===================================================================
> --- trunk/dabo/biz/dBizobj.py 2012-10-11 19:13:53 UTC (rev 7277)
> +++ trunk/dabo/biz/dBizobj.py 2012-10-13 18:39:16 UTC (rev 7278)
> @@ -21,8 +21,6 @@
>
> class dBizobj(dObject):
> """ The middle tier, where the business logic resides."""
> - # Class to instantiate for the cursor object
> - dCursorMixinClass = dCursorMixin
> # Tell dObject that we'll call before and afterInit manually:
> _call_beforeInit, _call_afterInit, _call_initProperties = False, False, False
Hi.
This change breaks all my applications, since they don't get customized cursor class anymore,
they get default one from the backend instead.
Why not simply reference dBizobj.dCursorMixinClass class attribute with your customized
cursor if you need it?
You should create dBizobj.getCursorMixinClass() method at last to allow override default cursor
creation chain behaviour.
--
Regards
Jacek Kałucki
_______________________________________________
Post Messages to: Dabo-dev@mail.leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/507A9627.30301@rz.onet.pl
It shouldn't break your apps. In dBackend, I have it defaulted to use dCursorMixin if the specific backend doesn't override it, which is the original behavior. Are you saying it doesn't do that?
Regards,
Nate
On Oct 14, 2012, at 6:38 AM, Jacek Kałucki <laborm@rz.onet.pl> wrote:
> Użytkownik Nate Lowrie napisał:
>> ===================================================================
>> --- trunk/dabo/biz/dBizobj.py 2012-10-11 19:13:53 UTC (rev 7277)
>> +++ trunk/dabo/biz/dBizobj.py 2012-10-13 18:39:16 UTC (rev 7278)
>> @@ -21,8 +21,6 @@
>>
>> class dBizobj(dObject):
>> """ The middle tier, where the business logic resides."""
>> - # Class to instantiate for the cursor object
>> - dCursorMixinClass = dCursorMixin
>> # Tell dObject that we'll call before and afterInit manually:
>> _call_beforeInit, _call_afterInit, _call_initProperties = False, False, False
>
> Hi.
>
> This change breaks all my applications, since they don't get customized cursor class anymore,
> they get default one from the backend instead.
> Why not simply reference dBizobj.dCursorMixinClass class attribute with your customized
> cursor if you need it?
> You should create dBizobj.getCursorMixinClass() method at last to allow override default cursor
> creation chain behaviour.
> --
> Regards
> Jacek Kałucki
>
[excessive quoting removed by server]
_______________________________________________
Post Messages to: Dabo-dev@mail.leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/F456EF96-5987-4656-A491-27824A36EAF7@gmail.com
Just committed a fix to a small typo. Try it now.
Regards,
Nate
On Oct 14, 2012, at 6:38 AM, Jacek Kałucki <laborm@rz.onet.pl> wrote:
> Użytkownik Nate Lowrie napisał:
>> ===================================================================
>> --- trunk/dabo/biz/dBizobj.py 2012-10-11 19:13:53 UTC (rev 7277)
>> +++ trunk/dabo/biz/dBizobj.py 2012-10-13 18:39:16 UTC (rev 7278)
>> @@ -21,8 +21,6 @@
>>
>> class dBizobj(dObject):
>> """ The middle tier, where the business logic resides."""
>> - # Class to instantiate for the cursor object
>> - dCursorMixinClass = dCursorMixin
>> # Tell dObject that we'll call before and afterInit manually:
>> _call_beforeInit, _call_afterInit, _call_initProperties = False, False, False
>
> Hi.
>
> This change breaks all my applications, since they don't get customized cursor class anymore,
> they get default one from the backend instead.
> Why not simply reference dBizobj.dCursorMixinClass class attribute with your customized
> cursor if you need it?
> You should create dBizobj.getCursorMixinClass() method at last to allow override default cursor
> creation chain behaviour.
> --
> Regards
> Jacek Kałucki
>
[excessive quoting removed by server]
_______________________________________________
Post Messages to: Dabo-dev@mail.leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/EEF0FAFB-91C8-41D2-9288-C8FB496C8C9C@gmail.com
On 10/14/12 2:07 PM, Nathan Lowrie wrote:
> Just committed a fix to a small typo. Try it now.
My MySQL version is broken at r7282 (works if I regress to r7277) with this traceback:
Traceback (most recent call last):
File "main_MU.py", line 2, in <module>
sbs_studio.main(mu_version=True)
File
"/pmcnett_home/home/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/sbs_studio.py",
line 144, in main
if app.SecurityManager and not app.SecurityManager.login():
File "/home/pmcnett/dabo-full/trunk/dabo/dSecurityManager.py", line 40, in login
if self.validateLogin(user, password):
File
"/pmcnett_home/home/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/App.py",
line 1751, in validateLogin
if biz.validateLogin(user, passwd):
File
"/pmcnett_home/home/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/biz/emps.py",
line 47, in validateLogin
self.save()
File "/home/pmcnett/dabo-full/trunk/dabo/biz/dBizobj.py", line 517, in save
cursor.save(includeNewUnchanged=True)
File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 1622, in save
saverow(row)
File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 1586, in saverow
self.__saverow(row)
File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 1745, in __saverow
self.BackendObject.noResultsOnSave()
File "/home/pmcnett/dabo-full/trunk/dabo/db/dBackend.py", line 117, in noResultsOnSave
raise dException.dException(_("No records updated"))
dabo.dException.dException: No records updated
Paul
_______________________________________________
Post Messages to: Dabo-dev@mail.leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/507C8EB4.3000806@ulmcnett.com
Hmm....
It looks like we are mapping correctly. I don't see anything regarding the new cursor class I put it. The only thing I can see that might cause it is moving the class declaration for the cursor mixin to the setConnection method. Add the following lines back in after line 23 on dBizobj.py and see if that solves your issue. If it does, we can commit that and still maintain the functionality.
# Class to instantiate for the cursor object
dCursorMixinClass = dCursorMixin
Regards,
Nathan Lowrie
lowrie.nate@gmail.com
On Oct 15, 2012, at 6:31 PM, Paul McNett <p@ulmcnett.com> wrote:
> On 10/14/12 2:07 PM, Nathan Lowrie wrote:
>> Just committed a fix to a small typo. Try it now.
>
>
> My MySQL version is broken at r7282 (works if I regress to r7277) with this traceback:
>
> Traceback (most recent call last):
> File "main_MU.py", line 2, in <module>
> sbs_studio.main(mu_version=True)
> File
> "/pmcnett_home/home/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/sbs_studio.py",
> line 144, in main
> if app.SecurityManager and not app.SecurityManager.login():
> File "/home/pmcnett/dabo-full/trunk/dabo/dSecurityManager.py", line 40, in login
> if self.validateLogin(user, password):
> File
> "/pmcnett_home/home/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/App.py",
> line 1751, in validateLogin
> if biz.validateLogin(user, passwd):
> File
> "/pmcnett_home/home/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/biz/emps.py",
> line 47, in validateLogin
> self.save()
> File "/home/pmcnett/dabo-full/trunk/dabo/biz/dBizobj.py", line 517, in save
> cursor.save(includeNewUnchanged=True)
> File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 1622, in save
> saverow(row)
> File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 1586, in saverow
> self.__saverow(row)
> File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 1745, in __saverow
> self.BackendObject.noResultsOnSave()
> File "/home/pmcnett/dabo-full/trunk/dabo/db/dBackend.py", line 117, in noResultsOnSave
> raise dException.dException(_("No records updated"))
> dabo.dException.dException: No records updated
>
>
> Paul
>
>
[excessive quoting removed by server]
_______________________________________________
Post Messages to: Dabo-dev@mail.leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/FB25A39E-AD07-4C76-B379-E7F7F654E2B7@gmail.com
I was able to reproduce the error. This newest commit should fix things.
Regards,
Nate
On Oct 15, 2012, at 6:31 PM, Paul McNett <p@ulmcnett.com> wrote:
> On 10/14/12 2:07 PM, Nathan Lowrie wrote:
>> Just committed a fix to a small typo. Try it now.
>
>
> My MySQL version is broken at r7282 (works if I regress to r7277) with this traceback:
>
> Traceback (most recent call last):
> File "main_MU.py", line 2, in <module>
> sbs_studio.main(mu_version=True)
> File
> "/pmcnett_home/home/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/sbs_studio.py",
> line 144, in main
> if app.SecurityManager and not app.SecurityManager.login():
> File "/home/pmcnett/dabo-full/trunk/dabo/dSecurityManager.py", line 40, in login
> if self.validateLogin(user, password):
> File
> "/pmcnett_home/home/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/App.py",
> line 1751, in validateLogin
> if biz.validateLogin(user, passwd):
> File
> "/pmcnett_home/home/pmcnett/py/sbs/shutter_studio/trunk/clients/shutter_studio/biz/emps.py",
> line 47, in validateLogin
> self.save()
> File "/home/pmcnett/dabo-full/trunk/dabo/biz/dBizobj.py", line 517, in save
> cursor.save(includeNewUnchanged=True)
> File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 1622, in save
> saverow(row)
> File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 1586, in saverow
> self.__saverow(row)
> File "/home/pmcnett/dabo-full/trunk/dabo/db/dCursorMixin.py", line 1745, in __saverow
> self.BackendObject.noResultsOnSave()
> File "/home/pmcnett/dabo-full/trunk/dabo/db/dBackend.py", line 117, in noResultsOnSave
> raise dException.dException(_("No records updated"))
> dabo.dException.dException: No records updated
>
>
> Paul
>
>
[excessive quoting removed by server]
_______________________________________________
Post Messages to: Dabo-dev@mail.leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/1E8AABA5-014B-441F-802D-D2F64CE0B805@gmail.com
On 10/16/12 7:15 PM, Nathan Lowrie wrote:
> I was able to reproduce the error. This newest commit should fix things.
Thanks! I'll test it tomorrow.
Paul
_______________________________________________
Post Messages to: Dabo-dev@mail.leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/507E2640.4070601@ulmcnett.com
Użytkownik Nathan Lowrie napisał:
> I was able to reproduce the error. This newest commit should fix things.
OK. Thanks.
BTW, did you try to use KeepAliveInterval to resolve backend disconnection problem?
--
Regards
Jacek Kałucki
_______________________________________________
Post Messages to: Dabo-dev@mail.leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/507E6435.90602@rz.onet.pl
Jacek,
I had the keep alive interval set for every minute but the connection was still being closed after 5 minutes of inactivity. I am not sure why, though it might be because we aren't querying against any database objects... The keep alive interval also doesn't solve the issue of sleeping a laptop and coming back a few hours later.
Regards,
Nate
On Oct 17, 2012, at 3:54 AM, Jacek Kałucki <laborm@rz.onet.pl> wrote:
> Użytkownik Nathan Lowrie napisał:
>> I was able to reproduce the error. This newest commit should fix things.
>
> OK. Thanks.
> BTW, did you try to use KeepAliveInterval to resolve backend disconnection problem?
>
> --
> Regards
> Jacek Kałucki
>
[excessive quoting removed by server]
_______________________________________________
Post Messages to: Dabo-dev@mail.leafe.com
Subscription Maintenance: http://leafe.com/mailman/listinfo/dabo-dev
Searchable Archives: http://leafe.com/archives/search/dabo-dev
This message: http://leafe.com/archives/byMID/DDFDE454-69EB-4D4F-9C08-99EDE37C49B0@gmail.com