[Twisted-Python] Test coverage requirements

[ex vito]

On 2017-02-26, at 23:51, Jean-Paul Calderone <exarkun at twistedmatrix.com> wrote:

> I'm looking at some recent trunk commits (also, others) that seem to have non-trivial untested code at at ReviewProcess.  I can't tell if the codecov reports are wrong or if the development process documentation is wrong or if the commits just violate policy or (I guess) some mix of the three.
> 
> Can anyone shed any light on this?

Jean-Paul,

I'm the author of PR 652, one of the ones you pointed out.

Starting with https://codecov.io/gh/twisted/twisted/pull/652 and navigating to https://codecov.io/gh/twisted/twisted/pulls/652/src/src/twisted/internet/unix.py I'm not sure I understand what it is representing -- lines of code that tests did not cover, I guess. :)

However, the diff for that file in the PR, at https://github.com/twisted/twisted/pull/652/files#diff-32f19fc001798d7ea33686492428bdf2 does not touch any of the lines highlighted by the those codecov links.

Does this make any sense? (plain honest question, I'm not familiar with codecov other that using its output to guide missing test scenarios)

At some point, like I commented in https://github.com/twisted/twisted/pull/652#issuecomment-276334447, codecov tests were failing, complaining that a single non-changed line within the -3/+3 line diff boundaries wasn't being covered. I found that strange then.

One other thing I find strange is that trying to access any previous codecov test results from within that PR -- say the first run on Dec 30th -- ends up showing the same (or very similar?) report as the one you pointed out initially; in other words, complaining about lines that were not changed in any commit within that PR.

Again, I'm not aware of the details of codecov's operation, but it certainly is no longer reporting the same things it did back when the PR was being worked on. Whatever it is reporting now, at least with regards to PR 652, does not seem to match the associated diff.

Regards,
--
exvito

-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/twisted-python/attachments/20170227/7383be4f/attachment-0002.html>