[Twisted-Python] twisted cred: why does avatarId need to be a str?

[Glyph Lefkowitz]

On Sep 7, 2010, at 12:16 PM, Laurens Van Houtven wrote:

> Hi,
> 
> As some of you probably know I'm trying to beat OAuth2.0 into submission. I'm using twisted.cred to do it. The restriction that avatarIds need to be strs is somewhat getting in my way, and I'm not sure if the correct way to do it is:

Let me begin my answer with Epigram 34:

"The string is a stark data structure and everywhere it is passed there is much duplication of process. It is a perfect vehicle for hiding information."

If I recall correctly, the Divmod realm/checker integration required a particular format for avatar IDs; they had to be 'user at domain' so that the server could do virtual hosting.  In some cases (POP for example) that actually meant the user needed to type in user at domain as their username.  So, while strings may not be ideal, it's almost certainly possible for them to let you do what you want.  I wouldn't recommend that approach if you have a bunch of extra data to deal with... but you don't really give a clear picture of what extra data you do have to deal with :).

> 1) ignore the restriction (credentials types are so specific to OAuth that it's very unlikely you'll find anything already pluggable...)

but the avatarID isn't a credential.  It's a mapping between the checker (which is about authentication) and the realm (which is about application data).  There hopefully _should_ be such a thing as some "already pluggable" application data.

> 2) use the mind object (but then it becomes somewhat wonky what goes on the mind object and what should be the avatarId)

Use the mind object for what??? You're asking this question completely backwards.  Even after reading the whole email I'm still not quite sure what you're on about.  I'll do my best to answer a few specific quirks here, but please re-post starting with what you're actually trying to do, i.e., "I got up today, and I wanted to paint the shed."

If you're already most of the way into an implementation, feel free to present bits of the implementation itself, but you haven't really described enough of it here for me to understand what you're doing and why.

> I'll give you a practical example:
> An OAuth client makes a request for an access token. This basically means he trades some credentials in for an access token (I use the expression trade-in because most of the credentials can only be used once). It presents some credentials, which can be grouped into two parts:
> 
> 1) credentials that authenticate the client itself
> 2) credentials for the access grant (eg I've got this thing that says I should have access)

Only one of these things are really "credentials" from cred's perspective.

Cred's purpose is to tie together two things: a realm that implements some protocol/application logic, and a checker that implements some authentication logic.  _everything that happens before you start talking to the application_ is just authentication stuff, and is part of the checker.  Even if that stuff, itself, needs to be pluggable.

I don't know the specifics of how OAuth works, but I know vaguely what it does, and ultimately what it needs to do is give you access to an IResource that allows you to do ... stuff.  The stuff-doing part is the interesting part.  That is the part that the Realm gives you.  The Realm ought to be able to work with Basic authentication, and the objects returned from the realm should be able to work even with no authentication (if you wanted to just put your resource endpoint up for everyone to use, unauthenticated, which is granted not very sueful).

> Because (1) is pretty much always the same (and is common to other parts of txOAuth) and (2) depends on the specific kind of request, this results in two portals:

> 1) for authenticating the client, which returns an IClient

I have no idea what an "IClient" is.  It sounds like you're using cred internally so that you can make the actual OAuth-ing process plugabble, and that's great, as long as you separate it from the post-OAuth part of actually producing a resource.

> 2) for authenticating the credentials (which includes the authenticated client, since credentials are client-specific), which returns the token

That part sounds like what I would expect.

> Coming from Perspective Broker I think the Client would be an apt mind object for the second portal, since it represents the entity on the other side trying to do auth.

The Mind is the entity on the other side trying to get something done, not trying to auth.  The docs sorely need to be updated to reflect this, but the canonical example of the 'mind' is the IChatClient interface in twisted.words (used in a cred context by twisted.words.service).  Of course, the IChatClient provider may also be involved in authentication, for example receiving a NickServ challenge from the server, but by the time you start calling that object a 'mind', authentication is done.

> The appropriate credentials checker in the second portal checks (and invalidates, since most of the credentials are single use) the credentials. What avatarId is sent to the Realm? The Realm is also responsible for requesting the new token,

OK, and that's where things go wrong.

The realm is responsible for one thing only: returning an object which implements an interface that allows a user of the library to define some application logic.  It should not be participating in auth.  If the authentication layer needs to perform some additional logic on each request, it should wrap up the IResource (or whatever) provider returned by the Realm, so that the Realm can do the work.

(and furthermore, why is anybody responsible for "requesting the new token" in the first place?  why do any new tokens ever need to be requested on the service provider's side of things?)

> so it needs to know details about the credential that the credential checker just checked (and in doing so, invalidated). These are not available through the IClient mind object.
> 
> The alternative design is that all of the credentials checkers know how to request tokens, and instead give the tokenId as avatarId. This is still insufficient, because tokens can include extra information, such as scope, expiration time... That can again be solved by sending an opaque string value which contains all of the data in the token response, but in that case the Realm is completely useless and the checkers do all of the work. Additionally, this seems stupid because cred checkers are for checking credentials, not checking credentials + a bunch of other stuff.

This has historically been a weak point for cred.  While I'm trying to give you a better idea of how to use it "right", you should be keenly aware that it isn't perfect and doesn't necessarily handle every case, so there are some things that you might try to do which won't be well supported.  Since cred is so mysterious and subtle, sometimes people believe that if they could just grasp its essence all their problems will go away; but it only solves one problem, so if that problem isn't the one you're facing, you may still be out of luck :).

The idea is that the realm is supposed to be a backend that has its own storage and logic and so on, and the checker is supposed to have enough information to produce an avatar ID that points at something the realm knows about.  The fiction that cred promulgates is that checkers are somehow agnostic to realms, but can still perform this mapping reliably based on their credentials.

Of course, this can't be universally true: if the checker needs to map things into the realm, then it stands to reason it *does* need to know something about the realm.  This level of flexibility is needed very rarely though, so what ends up happening is that all the existing checkers just relay the 'username' field to be the avatar ID.  In the event that you do need more flexibility - let's say your accounts database uses a peculiar convention to encode the authentication type into the stored user ID so that the same user might get different behavior based on how they authenticate - you could easily write some custom checkers that wrap simple underlying checkers and sprinkle on a little special-case logic to mutate the avatarID appropriately before returning it.

I think you're actually a little lucky to be implementing OAuth rather than OpenID, because my understanding is that with OAuth the expectation is that, as a provider of a service (i.e. someone likely to be using the cred interfaces, not a client) you expect that users have already created an account, and so the realm already has some data.  "expiration time" can be handled by the checker, or at least by the code that's driving the portal, to set session expiration.  (I'm not really sure what "scope" is, perhaps you could elaborate on some of these additional fields.)

The place where cred really falls down is multi-field stuff.  For example, in OpenID, a new user might show up with some OpenID metadata, like their preferred nickname, email address, and location.  Cred provides no way to tell the realm "create an account like this" or "are there any other users with this email address", so if you need to do *that* kind of thing, you're going to need to work on expanding the API.

> Here's my theory:
> 13:43 <lvh> aa_: fwiw I *think* it's because cred's supposed to be pluggable
> 13:43 <lvh> aa_: so avatarId is supposed to be the equivalent of a username
> 13:44 <lvh> aa_: and it's str not unicode because there's a helluva lot more things
> that store bytes than unicode objects
> 
> So um, any thoughts from people that understand cred? Is it ever okay to send richer things than strings to a Realm?

It hadn't really occurred to me to abuse the 'mind' object to provide that sort of structured data, but being able to specify the mind as "something adaptable to (interface X which provides the info you need)" is a nice big loophole :).  And the realm can return whatever it wants; it doesn't need to be based on the avatarID.  If you wanted to cram some more structured fields in there to smuggle them to the realm, that's a better place to put them than the strictly-mandated it's-just-some-bytes avatarID.  (This would allow other 'regular' HTTP realms to work with your OAuth code, but provide extra OAuth stuff for realms that want it).  I'm still not convinced that your realm should actually know about these extra fields, but maybe you just haven't given the relevant example yet.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://twistedmatrix.com/pipermail/twisted-python/attachments/20100908/df134115/attachment.htm