[Twisted-Python] twisted.vfs issues

[Jp Calderone]

On Wed, 28 Sep 2005 20:35:40 -0400, James Y Knight <foom at fuhm.net> wrote:
>On Sep 28, 2005, at 8:04 PM, Jp Calderone wrote:
>>  twisted.vfs should not import things from or depend upon  twisted.web2:
>>
>>    * web2 is unreleased
>>    * web2's APIs are unstable
>>    * vfs is more generally applicable than web2    * web2's stream 
>>abstraction is not generally agreed upon
>>
>>  If you like, we can talk more about how this interface should  work. 
>>However, my first inclination is to say that it should use  the existing 
>>producer/consumer APIs.  While these are not the best  APIs, they are used 
>>widely throughout Twisted, and therefore this  will give the greatest 
>>usability to the resulting VFS code.  While  there are adapters between 
>>these APIs and web2 streams, I still  recommend against web2 streams for 
>>the reasons mentioned above.
>
>Twisted.vfs should not depend upon a module in twisted.web2 when 
>twisted.vfs gets released. However, it is okay for it to depend upon  that 
>stream _code_ if it gets moved into twisted core before vfs is  released. 
>The idea all along has been to move t.w2.stream into  twisted core when it 
>is stable and useful. So I wouldn't worry about  tearing it out of t.vfs 
>quite yet.

That mildly addresses one of four points.  At the very least, the remaining three seem to remain valid.

>
>Now, my first inclination is that the current block API *is* the  right 
>primitive for a file.

It precludes writing large amounts of data to a file simply.

>
>Also, in particular, making it use the old producer abstraction as a 
>primitive is just asking for trouble. As the producer abstraction  lets the 
>producer send data asynchronously at any point, it becomes  almost 
>impossible to do a relatively simple operation like reading a  part of a 
>file. That is why, for web2, I had to drop it and make a  new API that has 
>the consumer request the data. I think the same  reasoning applies here.

The old API is not fantastic.  On the other hand, it's entirely servicable.  I don't understand why you think it is almost impossible to read part of a file using it.  In fact, I've done just this one several occasions.

>
>Again, I think that all requests for tearing various adapters and  other 
>bits out of twisted.vfs are currently completely premature. At  this point 
>in its development, it is critical that adapters for many  different systems 
>are created, to make sure that vfs has the  appropriate abstractions and 
>APIs to handle all use cases. And given  that vfs is itself heavily under 
>development, it makes no sense to  request that said adapters be adopted 
>upstream in each other project,  yet.

They can be removed from twisted.vfs without being removed from the Twisted repository.  Or they could be left in twisted.vfs but developed in a branch.  That is policy for major feature development, after all.

Jp