[Twisted-web] Re: Setting up a project.
Govind Salinas
blix at sophiasuchtig.com
Wed Aug 6 17:40:49 EDT 2008
On Wed, Aug 6, 2008 at 11:19 AM, David Bolen <db3l.net at gmail.com> wrote:
> "Govind Salinas" <blix at sophiasuchtig.com> writes:
>
-- snip
>> Let me give you some information about the set up I have . Basically
>> I have a python program running somewhere. I will have a default set
>> of files that serve the content a particular way. The default set
>> will contain mako templates[1], style sheets and perhaps some images.
>> So I need a way to kick off the mako templates based on the URL. The
>> templates may include each other.
>>
>> Here is the layout
>>
>> content-dir/
>> __config__.py -- This is a file that currently just sets up the mapping
>> between a regex that selects a page and a path to
>> where that page's template is under the content-dir
>> main.html -- The html files here are really the mako templates
>> main.css
>> etc...
>>
>> My server class takes in a template dir, loads the config and starts a
>> server. When it gets a request it matches it against the regexes and
>> runs the template. There should be some initialization that sets up
>> objects that get passed to the template (in addition to values from
>> the regex parsing). This is all pretty brain-dead code.
>
> Are you saying "brain-dead" in a negative way here, or to imply that
> it's boilerplate or something you'd prefer the framework do for you?
> With twisted.web, you're pretty much in charge of this sort of stuff.
I was referring to the code I was writing. I guess I would consider it
boiler plate. I think some more complete examples in the howto
would have helped me here. I seemed to get the gist of what
resources were supposed to do, but wasn't sure how to put
it all together.
> I think you've pretty much hit most of the items you need to cover.
>
> For URL traversal, it's not clear from your description if you are
> using the regex mapping to also find the Resource object or just for
> template lookups by the Resource objects. Either approach is fine.
> Twisted.web itself will traverse a URL by following the chain of
> Resource objects and their children. This can be much more dynamic
> than a fixed URL->object mapping definition since the Resources can be
> created on the fly when needed. If your Resource tree is static, then
> a fixed mapping is fine too.
I had tried using Django for what I was doing but I did not sure if it
would end up as a nice fit. When I started using twisted, I borrowed
the regex->page mapping from them. This is how I implemented it:
def _parse_setup(self, path):
self._config = {}
execfile(path, {}, self._config)
if 'url_map' not in self._config:
return False
self._mappings = []
for exp, f in self._config['url_map']:
f = f.split('/')
f.insert(0, self._template_dir)
f = os.path.join(*f)
self._mappings.append((re.compile(exp), f))
return True
def render_GET(self, request):
path = request.path[1:]
uri = request.uri[1:]
for mapping, f in self._mappings:
match = mapping.match(uri)
if match:
vars = match.groupdict()
print 'serving:', f
lookup = TemplateLookup(directories=['/'])
return Template(filename=f, lookup=lookup).render(**vars)
path = os.path.join(self._template_dir, path)
if os.path.isfile(path):
f = None
try:
idx = path.rfind('.')
print 'serving static:', path, path[idx + 1:]
type = _mime_types.get(path[idx + 1:], 'text/plain')
request.setHeader('content-type', type)
print request.headers, type
f = file(path, 'r')
return f.read()
except:
print 'failed to serve:', path
raise
finally:
if f:
f.close()
else:
print '%s does not exist' % path
request.setResponseCode(404)
return self._page_not_found
I was not too clear on how the resources would be used, so I chose
this. And the files/paths are all static, even if the content is not.
One of the things I want to be able to do is allow the user to have
a choice between templates or be able to write their own. The url
mapping seemed like a good way of soft coding that so that a
different template could organize their code differently.
> Generally what I've done in the past is build up the fixed portion of
> my Resource tree during server initialization. My Resource classes
> have internal definitions for their template name (as a relative path
> when using directories of templates) and are just handed the template
> loader during construction, which they later use for rendering. The
> template loader is given the template root directory at
> initialization. The trade-off with your regex mapping is that you
> have to keep two fairly separate bits of code in sync, where I have
> the template information inside the object definition, but in my case
> it can also be more work to get a global view of template assignments.
>
> In terms of your question on static files (css/images/js), I just
> place those into their own sub-tree and then defined a static.File()
> object to serve them. This also has an advantage that if you place
> your server behind, say, Apache, in production use, you can define
> rules on the Apache side to let it directly serve the static files and
> never involve your code, for some improved efficiency.
That sounds like a good idea, I will do that.
> Another area that you may need to implement your own support would be
> for handling authorization (twisted.web provides support for managing
> sessions via cookies, but you have to decide when to render an
> authentication page and then store any needed authentication
> information in the session).
I have no use for that yet. However I will eventually need to do http
and/or ssh authentication (the ability to POST updates to the
repository is a planned feature). So this will come in handy.
>> I was hoping for some guidance on a better strategy for doing this.
>> Any help would be appreciated. Bonus points if it is something that
>> could easily be re-used on another webserver if someone wanted to host
>> this on an existing website.
>
> I think you need to help defined what parts of the above you perceive
> as needing to be "better". I've implemented several servers (many of
> which were embedded in a larger application) using the above approach
> and it seems just fine to me. As to re-usability, that's pretty much
> the same question for any code you write - e.g., if you're doing
> authentication support, making it reusable is pretty much the same as
> making any other piece of code reusable.
>
> If you're looking for a higher level framework approach (much more
> functionality "for free"), then twisted.web itself may not be the
> ideal solution for you. In the twisted world, nevow may provide a bit
> more of what you're looking for, and of course there are other high
> profile high-level web frameworks like django, turbogears, pylons, etc...
>
I took a quick look at nevow and I think I like the mako templates
better. It behaves more that a "python server pages" type of
template, which is conceptually similar to the little web programming
that I have done in the past.
--snip
> As a more concrete example, here are some snippets of one of my recent
> web server modules (in commercial use for the past year or so). Note
> this is a straight cut 'n paste, so there's various logic in addition
> to the pure web processing, and probably references to functions/objects
> that won't fully be explained.
>
-- snip
> Hope there's not too much "noise" in the code to prevent it from being
> helpful.
>
Actually that was perfect, I have a much better understanding of how things
are supposed to work now. I appreciate the help.
Thanks,
Govind.
More information about the Twisted-web
mailing list