[Twisted-Python] libamp - A C library implementation of AMP

[Eric P. Mangold]

Hello,

Peter le Bek and I have been working on a C library implementation
of AMP for about 4 months now, and I was hoping to get some feedback
and constructive criticism prior to a release. And I need help answering
one big question (below).

http://amp-protocol.net/Libamp

libamp is currently quite usable, for clients and servers, and is geared
towards being used with an async I/O library such as libevent, but it
could also be driving with synchronous I/O if there was some reason to.

And we have very good test coverage! The test suite runs valgrind-clean
and includes malloc()-failure tests, so pretty much all error-handling
cases are covered and won't leak memory.

http://teratorn.org/code/libamp/html_coverage/code/libamp/index.html

A number of basic AMP types are currently supported, but support for
more advanced types, like DateTime, Decimal, ListOf and AmpList is still
needed.

The best way to get a feel for the current API would be to read the
example programs, e.g.:

http://teratorn.org/code/darcsweb/darcsweb.cgi?r=libamp;a=headblob;f=/examples/asyncserver.c
http://teratorn.org/code/darcsweb/darcsweb.cgi?r=libamp;a=headblob;f=/examples/sumclient.c

and have a look at amp.h:

http://teratorn.org/code/darcsweb/darcsweb.cgi?r=libamp;a=headblob;f=/amp.h

The API centers around AMP_Proto objects, which you create for each AMP
peer you're talking to. An AMP_Proto includes a function pointer, which you
set with amp_set_write_handler(), that knows how to deliver (or queue)
bytes to the other side (be in stdout, a socket, etc).

Then you may use amp_call() to call remote commands. When new bytes come
in, e.g. via libevent, you drive the AMP_Proto with a call to
amp_consume_bytes() which parses the incoming data and performs
callbacks to 1) deliver the result of a previous amp_call() 2) invoke a
registered responder for a command.

amp_add_responder() adds a callback function for handling specific AMP
command requests.

The basic API for actually handling a command request, or for
constructing a response, is low-level, but usable - which leads up to
the big question I'm trying to answer:

Does AMP need a declarative syntax for defining AMP commands? Think e.g.
protobuf's .proto files.

With libamp you deal directly with AMP boxes - sets of key/value pairs,
and you need to manually verify that a command was called with all of
the keys that it expects, and that the values given coerce to the
type of data you expect. Viewing amp.h and the examples, you can see how
 the amp_put_*() and amp_get_*() family of functions is used to encode
 and decode C types to and from AMP_Box's.

This isn't terribly difficult and works well enough in the simple
example programs, but I feel there must be a better way, that alleviates
some of the manual error checking.

Right now I'm leaning toward coming up with some kind of .amp file
format, and an `ampc' tool to compile it to C/libamp code.

Support for .amp files could be added in Twisted, and you could then
share Command declarations between clients and servers written in
different languages.

What about the next step of declaring AMP "services" - a set of Commands
available via a known URL. Why do crummy web-services get to
have all the fun?

All thoughts and suggestions are welcome.

    -Eric


P.S. Anyone wanting to help with the C code is free to join #amp on
Freenode and lend a hand. It could use a once-over by folks familiar
with writing C libraries and best-practices thereof.