Hi Paul and Ed!

I'm finally back home after quite a long trip back from PyCon. During

the non-sleeping flight I finally had the time to look better into

Paul's code for generating the pdf. I concentrated on reportWriter.py,

and ignored reportDesigner.py for now.

I took the chance and did a complete refactoring of that, in order to

show you an example of what I was talking to you at PyCon, regarding

xml serialization.

Very little of the original code structure remains, but I don't want

this to look like throwing away Paul's effort. On the contrary, I used

all the reportlab specific code as is. I just tried to refactor away

everything that was not tightly related to it. Take this as a proof of

concept of my ideas on how things should be done, basing on my

experience. Whether to go this way or another is your decision, since

you guys are the two main developers.

I'll try to highlight the advantages of this design. First of all some

numbers: reportWriter.py is 1018 lines. After refacting a total of 755

lines remained, of which only 467 regard the report generation logic.

The other 288 have been refactored away inside a

dabo.lib.serialization package.

The code is laid out this way:

dabo.lib.reporting:

report.py Pages and bands

objects.py Drawable objects (Rect, String and Image)

run.py if __name__ == '__main__'

util.py A few lines of utility functions

dabo.lib.serialization:

__init__.py

serialization.py

attributes.py

children.py

xmlserializer.py

The dabo.lib.serialization package contains all the code involved into

dealing with xml. For now I only did deserialization from xml (i.e.

xml->python objects), the other way (objects->xml) is missing.

The modules serialization.py, attributes.py and children.py provide

the support for serialization, but the actual operation is done by

xmlserializer.py, therefore any other kind of serialization can be

supported, by simply having another such module like, say,

pickleserializer.py, or socketserializer.py.

Here's an example of how the deserialization works. Every tag in the

xml either represents an object, or represents an attribute of that

object. In the case of attributes, the tag name equals the attribute

name, and the tag's cdata contains the value.

If a tag represents an object, it can either be stored as an attribute

of another object, or it could be contained in a list, that in turn is

stored as an attribute of another object. Dictionaries can be

supported too, but I didn't need them yet.

Example:

from dabo.lib.serialization import *

class Foo(Serializable):

marginRight = LengthAttr(36)

size = PagesizesAttr('letter')

background = ArributeChild('report.Band')

objects = ObjectListChild(['objects.String', 'objects.Rect'])

fooInstance = deserialize(xmldata, Foo)

<foo>

<marginRight>"18 mm"</marginRight>

<size>"A4"</size>

<background>

[...]

</background>

<string>

[...]

</string>

<rect>

[...]

</rect>

</foo>

The function deserialize() takes the xml data and the class

representing the root tag, and returns an instance of that class.

The parsing mechanism goes like this:

- open <foo> tag: the tag name (foo) matches the name of the class

being provided to deserialize(), so instantiate it (and later return

it to the caller)

- open <marginRight> tag: since in Foo class there is a definition for

an attribute called marginRight, so this tag must represent it. Go

ahead, parse its cdata, validate it with the LengthAttr.validate()

method, get back an instance of that attribute and attach it to the

Foo class instance.

- open <background> tag: since in Foo class there is a definition for

a child object named background, this must be it. Recursively parse

the xml according to the rules defined in the report.Band class (not

shown in the example) and stick that class's instance to the Foo

object as an attribute.

- open <string> and <rect> tags: same recursive parsing as for

<background>, getting back instances of the classes object.String and

object.Rect. But this time store these objects inside a list, and

stick that list inside Foo's instance.

This mechanism is implemented by using metaclasses. This approach is

very much like how they do it in Struts (you haven't been at Struts

talk though, if I remember).

A serialize() function can easily be implemented that takes an

instance of such Serializable classes and spits out xml.

This way I was able to factor away all the code dealing with

dictionaries, because all the class instantiation is done for me by

the deserialize() call.

The remaining code, doing the actual interaction with reportlab, is

divided into two modules: report.py and objects.py. The former

contains the code for generating pages and drawing bands. I extended

the functionality to support detail bands spanning across multiple

pages. The latter module contains the drawable objects, that is Rect,

String and Image. For now I commented out Frameset for two reasons:

I'm lazy :), and I'm not sure it's a good idea to have paragraph

specification statically stored in the rfxml template. I rather think

that reportlab framesets should be used when some given text (i.e. out

of a database field) contains carriage returns, and is very long. In

that case a String drawable object cannot be used, and something else

(maybe called like "Paragraph"?) must be used instead, that will make

use of framesets... we can discuss this issue sometime.

Another big change that I've done regards the eval() calls. By simply

calling eval(exprCode), exprCode is evaluated in the local namespace,

therefore 'self' is accessible and potentially modifiable by the mean

user. Alright... statements cannot really be used because eval() takes

only expressions, but I think we should rather user exec, because one

may want to write an if/then/else statement in order to implement some

more interesting functionality. Therefore, at that time, being able to

access 'self' could be dangerous. And even if not used maliciously, it

exposes way too many underlying details. I thought that explicitly

passing a hand crafted environment could be more effective, and the

code also becomes self commenting. If one sees that the call to eval

is passing an environment like {'record':record, 'page':page} it

appears clear that the evaluated code is supposed to make use of these

objects, and nothing more.

Last thing: I changed a little bit the structure of the rfxml, by

moving the band tags inside the <page> tag. Really, I think that

<page> (corresponding to class Page) should probably be called with a

different name, maybe something like BandReportPageGenerator.

Alright, enough talking. I hope you guys have some time to look at my

code. I'm eager to hear your comments about it.

Cheers,

stefano

--- StripMime Report -- processed MIME parts ---

multipart/mixed

text/plain (text body -- kept)

application/x-zip-compressed

---

On Mar 29, 2005, at 10:26 AM, Stefano Masini wrote:

> Alright, enough talking. I hope you guys have some time to look at my

> code. I'm eager to hear your comments about it.

>

> Cheers,

> stefano

>

> --- StripMime Report -- processed MIME parts ---

> multipart/mixed

> text/plain (text body -- kept)

> application/x-zip-compressed

> ---

Guess we should have warned you - the list strips off all attachments,

and converts HTML email to plain text. You should email the code

off-list to Paul and me.

___/

/

__/

/

____/

Ed Leafe

http://leafe.com/

http://dabodev.com/

Stefano Masini wrote:

> I'm finally back home after quite a long trip back from PyCon. During

> the non-sleeping flight I finally had the time to look better into

> Paul's code for generating the pdf. I concentrated on reportWriter.py,

> and ignored reportDesigner.py for now.

>

> I took the chance and did a complete refactoring of that, in order to

> show you an example of what I was talking to you at PyCon, regarding xml

> serialization.

Hey Stephano... I just wanted you to know that I've seen your message

and downloaded your files, but haven't had a chance to look at them yet.

I'll be back home tomorrow for the first time since PyCon, and will try

to have a detailed look at your code over the weekend. I've only had a

few minutes here or there, but want to give your code the attention it

deserves before commenting on it.

Just so you don't think I'm ignoring you, as long as your code fits into

Dabo, solves all the needs, and I can understand it :), I'll most likely

commit your changes and go forward with them. But I have to look at them

first - bug me next week if I haven't responded yet.

Thanks! :)

--

pkm ~ http://paulmcnett.com

> Just so you don't think I'm ignoring you, as long as your code fits into

> Dabo, solves all the needs, and I can understand it :), I'll most likely

> commit your changes and go forward with them. But I have to look at them

> first - bug me next week if I haven't responded yet.

Oh yeah, definitely. When you have time. I'm very busy myself right

now. Luckily I had some time to work on it on my way back from pycon.

By the way, as I said, take this as a proof of concept of what we were

talking. More like a way to better explain my thoughts. If you find it

interesting, even better.

cheers,

stefano