Ticket #3014: ipv6.log

File ipv6.log, 16.3 KB (added by exarkun, 6 years ago)

the irc conversation

Line 
117:32 < exarkun> So, I think the thing that makes this difficult is just that Twisted is bad.
217:32 < exarkun> Er I mean listenTCP is a bogus API.
317:33 < exarkun> Hm
417:33 < exarkun> No, I really mean that listenTCP6 is a bogus API.
517:33 < exarkun> I didn't actually look at how that is interacting with name resolution.
617:33 < exarkun> I assume it is somehow though.
717:34 < therve> afaik, the problem is more connectTCP6
817:34 < exarkun> therve: It's the same, although it's more obvious with connectTCP6, yea
917:34 < exarkun> IPv4Address(...).connectStream(...) is a good API
1017:34 < exarkun> Because it lets you do IPv6Address(...).connectStream(...) too
1117:34 < exarkun> Polymorphism etc!
1217:34 < therve> yeah, but it's doesn't exist
1317:34 < exarkun> That's true.
1417:35 < exarkun> A slightly less good API is
1517:35 < therve> making that depend on a ticket opened 3 years ago is unfortunate
1617:35 < exarkun> connectTCP(IPv4Address(...), ...)
1717:35 < exarkun> since it lets you do connectTCP(IPv6Address(...), ...)
1817:35 < exarkun> (Or connectTCP(UNIXAddress(...), ...))
1917:35 < radix> or Hostname(...).toIPv6Address()
2017:35 < glyph> :-(
2117:36 < exarkun> radix: Really?
2217:36 < radix> exarkun: probably not. The weird thing about it is that it would return a Deferred.
2317:36 < radix> and you probably don't want a connect method that takes a deferred.
2417:36 < exarkun> Well!  That's interesting.
2517:36 < glyph> is an IPv6 socket a fundamentally different thing from an IPv4 socket?
2617:36 < exarkun> You could argue that connectTCP('hostname', ...) is insane.
2717:36 < radix> So it'd be Hostname(...).toIPv6Address().addCallback(connectTCP, ...)
2817:36 < exarkun> glyph: Those words are missing semantics, I think.
2917:37 < glyph> exarkun: I am missing the semantics in my brain why is why I didn't put them in the words.
3017:37 < exarkun> Or I could say "IPv6 socket is a nonsensical concept, don't say that ever again"
3117:37 < exarkun> The underlying BSD socket API makes it really clear what the high-level abstraction should be
3217:37 < glyph> what's the underlying API?
3317:38 < exarkun> socket(domain, type, protocol)
3417:38 < glyph> oh, that API.
3517:38 < radix> what's the clear thing? :)
3617:39 < exarkun> The clear thing is that there's a domain and a type (and we'll ignore protocol!  because who cares)
3717:39 < exarkun> So the high-level abstraction should have a domain and a type
3817:39 < radix> ok, I always forget which of those things you're supposed to ignore :)
3917:39 < exarkun> connectTCP makes it hard to support IPv6 because it smashes domain and type into one thing
4017:40 < exarkun> connectTCP in Twisted right now is really connectTCPIPv4, and if you want IPv6 you need connectTCPIPv6
4117:40 < exarkun> and that's ridiculous
4217:40 < exarkun> and you don't even have to look at ipv6 to see how it's ridiculous
4317:40 < exarkun> because connectUNIX is the same crummy thing
4417:40 < exarkun> So the high-level abstraction should let you talk about domains separately from types
4517:41 < exarkun> and it doesn't do that in a good way right now
4617:41 < radix> exarkun: maybe connectTCP is still a nice convenience API to have, especially if it will either use IPv6 or IPv4, automatically.
4717:41 < exarkun> sure, that's the connectTCP({IPv4,IPv6,UNIX}Address(...), ...) case
4817:41 < glyph> right, how do you *tell* if you want to use IPv6 or IPv4 though
4917:41 < exarkun> Yes, another good question!
5017:41 < radix> exarkun: no, I mean connectTCP(string, ...)
5117:42 < exarkun> radix: whatever, same difference, hang on a sec
5217:42 < dash> glyph: simple - you don't ever want ipv6
5317:42 < exarkun> if you have a "string", then you're thinking about the same thing glyph is thinking about
5417:42 < radix> I will hang on, but it doesn't sound like the same thing :)
5517:42  * dash runs away
5617:42 < exarkun> except again you don't have to bring ipv6 into it to get the problem
5717:43 < exarkun> if you pass "example.org" to connectTCP and there's two A records associated with example.org, which do you use?
5817:43 < radix> holy shit, is it raining?
5917:43 < exarkun> the way it works now, you make an arbitrary decision, which is a bad thing for the lowest-level api to do.
6017:43 < radix> worse! it's deathing
6117:43 < radix> exarkun: yeah, definitely.
6217:44 < exarkun> so is it any worse if you make the arbitrary decision between two A records, an A6 record, and three AAAA records?
6317:44 < radix> exarkun: I was kind of proposing connectTCP becoming a higher level API, while maintaining the same name and signature.
6417:44 < glyph> radix: it's *deathing*?
6517:44 < exarkun> radix: that might be a good idea.  in any case, disallowing a string to connectTCP now is probably not a good idea.
6617:44 < radix> glyph: sleet
6717:44 < radix> exarkun: yeah :)
6817:45 < radix> exarkun: I guess the thing I'm trying to figure out is whether we should also have it look up an A6 record for that string in the future
6917:45 < exarkun> Well
7017:45 < radix> (or AAAA or whatever TF we're supposed to use)
7117:45 < exarkun> Turns out
7217:45 < exarkun> This is pretty funny really
7317:45 < radix> heh
7417:45 < glyph> oh no
7517:45 < radix> are you going to paste that thng
7617:46 < exarkun> Turns out it has been looking up A6 and AAAA records for years
7717:46 < radix> blirk
7817:46 < exarkun> Good thing nobody uses IPv6 or that might've caused some problems.
7917:47 < radix> so basically Twisted is really slow :)
8017:47 < exarkun> Nah, it's actually no slower than if it only looked up A records!
8117:47 < exarkun> However, it is broken (since ANY queries aren't really very good)
8217:47 < radix> exarkun: did you learn some things about DNS
8317:47 < radix> exarkun: oh wait, the ANY thing??????
8417:47 < exarkun> I learned some things about DNS
8517:48 < radix> exarkun: SURELY we fixed that
8617:48 < radix> I discovered the ANY bug like ten *years* ago
8717:48 < exarkun> It may be that no one ever uses anything but the default resolver
8817:48 < exarkun> And that's what lets it actually work, I'm not sure
8917:48 < radix> ah, ok
9017:48 < radix> exarkun: so, yeah, maybe you don't remember
9117:48 < radix> exarkun: when I worked for April, I discovered that ANY is something you should never use
9217:48 < exarkun> Indeed.
9317:48 < exarkun> But apparently you didn't fix it in Twisted.
9417:48 < radix> because it often only returns cached values
9517:49 < radix> exarkun: crap. did I even file a bug? :(
9617:49 < radix> it's about getHostByName, right?
9717:49 < exarkun> It's about getHostByName, yes.
9817:49 < exarkun> I don't remember any ticket related to this.
9917:49 < radix> yeah. it does an ANY query. dangit.
10017:49 < radix> I'm sorry. :(
10117:49 < exarkun> We're wandering from the main point though, I think
10217:50 < exarkun> which is that there should be an api which parameterizes the domain separately from the type
10317:50 < exarkun> instead of encoding the type into the method name
10417:51 < PenguinOfDoom> IReactorInternetInternetInternetInternet
10517:52 < exarkun> therve: I don't have any idea if that answered any of the interesting questions.
10617:54 < therve> exarkun: I think for now it mainly raised other questions :/
10717:55 < PenguinOfDoom> therve: exarkun is a scientist, not a preacher. That's how all of his explanations are.
10817:55 < glyph> exarkun: Well, at some point, the correct answer here is "endpoints", right?
10917:55 < therve> glyph: in your dreams :)
11017:55 < therve> glyph: this is not only an API issue, unfortunately
11117:55 < glyph> therve: Twisted *IS* my dream ;-)
11217:55 < PenguinOfDoom> glyph: Twisted isn't pretend vaporware like endpoints, though.
11317:56 < PenguinOfDoom> well, mostly
11417:56 < glyph> OK, so, I think I'm getting lost on the details that are not API issues
11517:56 < glyph> (or bugs in listenTCP)
11617:57 < radix> hmm apparently it is called "*" in the RFC, not ANY
11717:57 < exarkun> indeed
11817:57 < exarkun> but who cares about the rfc, dig says it's "any"
11917:58 < radix> exarkun: well yes, I'm just trying to refer to something in an RFC in this ticket I'm filing
12017:58 < exarkun> radix: ah, excellent
12117:58 < radix> but it is really hard to find the stuff I need
12217:58 < exarkun> therve: Since I haven't really looked at the patch in depth, I can't think of any of the non-API issues
12317:59 < therve> the great thing is that I only spotted that because for some reason os X decides to return IPV6 records first
12417:59 < glyph> therve: sweeeeet
12517:59 < exarkun> radix: I can give you a link to some mailing list correspondence to link to from the ticket :)
12617:59 < glyph> therve: OS X is clearly the OS of the *future*
12717:59 < PenguinOfDoom> therve: That's because the RFC tells to return IPv6 records first.
12817:59 < glyph> because in the future the internet will burn down and nothing will work
12917:59 < glyph> o/`
13018:00 < exarkun> therve: Oh right, getaddrinfo?
13118:00 < therve> exarkun: yeah
13218:00 < exarkun> That part of the change seemed dubious to me.  But I'm not sure.
13318:00 < PenguinOfDoom> glyph: Whatever. In the future, I will be surfing web6 while you are wandering among the ruins of your old crappy internet
13418:00 < glyph> PenguinOfDoom: you aren't even surfing web 2 yet
13518:00 < radix> I thought I once read something by djb about */ANY, but I can't find it. The only thing I see that he mentions is about how MX records can be interleaved with other records, and that happens especially often in * queries.
13618:00 < glyph> no way you're going to get all the way to 6
13718:00 < PenguinOfDoom> glyph: Cisco ain't done making the surfboard yet
13818:00 < therve> exarkun: maybe
13918:01 < PenguinOfDoom> djb hates the hell out of ipv6
14018:01 < exarkun> therve: Isn't a simple solution to not use getaddrinfo for IPv4 connection attempts?
14118:01 < therve> radix: http://cr.yp.to/djbdns/killa6.html
14218:01 < therve> exarkun: well, how would you know?
14318:01 < radix> therve: this isn't about the ipv6 stuff, just about what qtype=* means
14418:02 < exarkun> therve: connectTCP('a.b.c.d', ...) is an ipv4 connection attempt; connectTCP('a::b', ...) is an ipv6 connection attempt; connectTCP('hostname', ...) is an ipv4 connection attempt
14518:02 < exarkun> therve: for example
14618:02 < therve> ah
14718:02 < therve> that's a solution, indeed
14818:02 < glyph> connectTCP(_6=True)
14918:02 < therve> glyph: dieinfirenow
15018:03 < exarkun> +1 :)
15118:03 < glyph> more seriously, presumably someone might want connectTCP('hostname', ...) to be IPv6.  How do they say that?
15218:03 < therve> using endpoints! ahah!
15318:03 < exarkun> Yes.
15418:03 < exarkun> Or more generally, "using the good API instead of the bad API"
15518:03 < glyph> I guess that's where I get a little lost.  What would the endpoint call, if not connectTCP6 or listenTCP6?
15618:03 < exarkun> maybe that's endpoints maybe it's something less drastically different
15718:04 < glyph> I think of endpoints as a higher level API around lower level stuff the reactor is doing.
15818:04 < glyph> not as a replacement for all of the existing reactor APIs
15918:04 < exarkun> yes.
16018:04 < exarkun> you can't do it if it's not implemented by the reactor
16118:04 < PenguinOfDoom> no :(
16218:04 < exarkun> but the way the reactor wants to work now is not awesome.
16318:04 < PenguinOfDoom> endpoints.connectTCP6('ip6porn.com', reactor=ip6reactor)
16418:05 < exarkun> I would suggest connectTCP(IPv6Address(...), ...) as a better low-level API than connectTCP6(...)
16518:05 < PenguinOfDoom> wait, that doesn't even begin to make sense :(
16618:05 < exarkun> PenguinOfDoom: Serious contributions only, please.
16718:05 < exarkun> PenguinOfDoom: We will have joke time afterwards.
16818:05 < PenguinOfDoom> :(
16918:05 < PenguinOfDoom> Okay, do I bring SCTP up now or during joke time?
17018:05 < exarkun> PenguinOfDoom: joke time
17118:05 < PenguinOfDoom> :(
17218:06 < exarkun> We have an example of a different type already - UDP.
17318:06 < dreid> PenguinOfDoom: that's not how endpoints are supposed to work.
17418:06 < kenaan> new names defect #3019 by radix: twisted.name's default getHostByName does not reliably fetch records because of its use of ANY (qtype=*)
17518:06 < glyph> radix: careful with that apostrophe, you might hurt somebody.
17618:07 < glyph> exarkun: I think I must have a broken mental model of how this is all supposed to work.  In my fantasy world, a user cares mostly about the hostname, with IPv6 or IPv4 being an irrelevant detail; connectTCP("hostname", ...) should therefore connect to either v6 or v4 hosts as appropriate and not tell anyone.
17718:07 < radix> my apostrophes are awesome and deadly.
17818:07 < glyph> exarkun: once you've done name resolution by yourself, connectTCP(IPv6Address) seems like a good API.
17918:07 < dreid> I agree.
18018:07 < exarkun> glyph: HostnameEndpoint("hostname").connectStream(...)
18118:07 < radix> hmm, ctrl-t doesn't work in firefox
18218:07 < PenguinOfDoom> glyph: In the real world, google.com gets an A6 record and half their users suddenly can't reach the site because they have a link-local ipv6 address, but no real ipv6 connectivity
18318:08 < glyph> exarkun: I'm pretty sure I'm confused though.  Perhaps that sort of magic should be reserved for something like strports?
18418:08 < radix> glyph: I changed it
18518:08 < radix> glyph: I hope you like what I changed it to
18618:08 < glyph> PenguinOfDoom: I was beginning to guess that might be the case, yes.
18718:08 < PenguinOfDoom> but wait, there's more. Vista *does* have ipv6 connectivity out of the box
18818:08 < PenguinOfDoom> so only people behind corporate firewalls will be screwed
18918:08 < glyph> PenguinOfDoom: ... but it's all routed through spyware.microsoft.com?
19018:09 < PenguinOfDoom> glyph: No, it uses the magical anycast tunnel, I think
19118:09 < PenguinOfDoom> not sure how that works with NAT
19218:09 < therve> oh yeah, XP ipv6 support seems to be totally broken
19318:09 < radix> PenguinOfDoom: whoah, are you serious?
19418:09 < exarkun> PenguinOfDoom: It's easy to recognize a link-local ipv6 address and decide to never try to make ipv6 connections.
19518:10 < exarkun> glyph: Did I answer your question or not?
19618:10 < PenguinOfDoom> exarkun: Maybe in your fantasy world of well-written C libraries.
19718:10 < glyph> exarkun: HostnameEndpoint(...)?
19818:10 < exarkun> glyph: Yes.
19918:10 < exarkun> PenguinOfDoom: My fantasy world has Python in it, not C.
20018:10 < PenguinOfDoom> radix: I tried it once, I think.
20118:10 < radix> PenguinOfDoom: did you connect to freenode? :)
20218:11 < glyph> exarkun: billions of Twisted programs everywhere are using connectTCP when that's what they mean, but I guess it's not really an important consideration given that we wouldn't be breaking them just by not changing its behavior
20318:11 < PenguinOfDoom> exarkun: Python can't even find out what local addresses the box has
20418:11 < PenguinOfDoom> radix: Now that gives me an idea.
20518:11 < exarkun> glyph: Not changing behavior is awesome.
20618:12 < PenguinOfDoom> ah crap, this is where amiaaaaornot.net would come in handy
20718:12 < glyph> exarkun: I would prefer to change behavior so that cool new functionality would magically start working with new versions of Twisted, but I can't really get worked up about this particular feature.  IPv6 just makes every particle of my body become sad.
20818:12 < PenguinOfDoom> I guess, since dreid is being an ass, I should just register the .org or the .com variant
20918:12 < dreid> PenguinOfDoom: I'm almost certain you already control the DNS.
21018:12 < exarkun> glyph: Just think about it this way
21118:13 < exarkun> glyph: There _is_ no cool new functionality.
21218:13 < PenguinOfDoom> oh whoops
21318:13 < PenguinOfDoom> dreid: Yes. It's hax time.
21418:13 < exarkun> glyph: No one cares about or benefits from IPv6.
21518:13 < glyph> exarkun: Exactly what I meant, yes. :)
21618:14 < exarkun> Freenode's IPv6 configuration is actually a good example of what the real world constrains you t o.
21718:14 < exarkun> There is no IPv6 address for chat.freenode.net.
21818:14 < exarkun> If you want ipv6, then you connect to ipv6.chat.freenode.net.
21918:14 < glyph> ooh, which ping thinks is an unknown host.
22018:15 < glyph> is there a ping6?
22118:15 < PenguinOfDoom> ping6, dude
22218:15 < glyph> oh god there is
22318:15 < glyph> fuck
22418:15 < PenguinOfDoom> hahahaha
22518:15 < glyph> that was a joke
22618:15 < exarkun> and traceroute6 and so on and so forth.
22718:15 < glyph> okay
22818:15 < PenguinOfDoom> mtr is automagic, though
22918:15 < PenguinOfDoom> so's telnet
23018:15 < glyph> well why didn't you guys say so before then
23118:15 < PenguinOfDoom> -4 or -6
23218:15 < glyph> the solution is obvious
23318:15 < glyph> we'll only have connectTCP
23418:15 < itamar> twisted6
23518:15 < itamar> or connectTCP6?
23618:15 < glyph> but we'll do sys.argv[0].endswith('6')
23718:15 < PenguinOfDoom> :/
23818:15 < exarkun> alright, we're in joke time now I guess
23918:16 < glyph> twistd6, trial6, etc
24018:16 < exarkun> so I'm gonna go get lunch