[Twisted-Python] pushing out same message on 100k TCPs

[Tobias Oberstein]

> > what is the most efficient/performant way of doing the following?
> >
> > I have a short message prepared .. say a string of 100 octets.
> > I want to push out that _same_ string on 100k connected TCPs (on a server).
> >
> > ==
> >
> > My thinking was: ideally, the 100 bytes would be transferred to
> > kernel/NIC space just _once_, and then the kernel is only told to
> > resend that buffer on the 100k connected TCPs.
> >
> > Does that make sense, is that even possible, with Twisted, or in general?
> 
> Not really, no.
> 
> The problem is that TCP requires the sender of data to buffer so that it can re-
> send. The only way to store one copy of the data whilst doing this would be to

Resends -- oh, yes, right. Forgot about that.

> store the socket buffer as a (fairly complex) linked list of reference-counted
> blocks, and use scatter-gather IO to the network card.

Doesn't a (modern) kernel do something like that for virtual memory pages ie.?

For if processes mmap files or load shared objects, the kernel must keep track
of pages which are needed by active processes.

So in theory, if my message is 1 VM page, kernel could pin that page (make it
non-swappable etc) until all TCP ACKs have arrived ..

> In short; this kind of thing seems easy and desirable but actually it's really hard

Probably quite hard ..

> and not useful.

When using VM pages (_if_ that would be possible) and thus no data duplication,
then why not useful?

However I now can see that it's likely outside of what can be done with a stock
kernel .. if it can be done at all anyway ..

Thanks!