Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

There can be circumstances in which the substituted operator is just as valid as the operator being substituted for, resulting in false positives.


How would that be a false positive? If the substituted operator does not cause a test to fail, then your tests don't cover it. If the function of the program is not changed by changing the operator, then the operator does nothing and should be removed.

That's the idea at least. All focus on high coverage, whether line coverage or mutation testing, creates an incentive to remove redundant robustness checks, and maybe that's not such a good idea after all. But that's a problem with all unit testing, and not just with mutation testing.


I was thinking that, for example, that a >= test is as valid as == in some cases. If == is actually used and is correct, substituting >= would not cause any valid test to fail. As I wrote this, however, I realized that if you substitute the inverse operator (!= for ==, < for >= etc.) and it is not caught, you can infer it is not covered. (edit: or that, in the context of the program as a whole, the result of the operation is irrelevant. I imagine this might legitimately happen, e.g. in the expansion of a macro or the instantiation of a template.)


Substituting == for >= should cause a test to fail. Either == is correct, or >=. They can't both be correct. Should the two things always be equal? Then they shouldn't be unequal. Is one allowed to be bigger than the other? Then that should be allowed, and not rejected.

If this change doesn't cause a test to fail, you're not testing edge cases for this comparison.

(Of course that's assuming you want perfect coverage.)


Counter-example:

  while ( j < JMAX) {
     i = callback(j);
     if( i >= FINAL) break;
     ...
If, in the specific circumstances of this use, callback(j) will always equal FINAL before it exceeds it, a test for equality here will not cause the program to behave differently.

The fallacy of your argument is that == is not an assertion that its arguments should always be equal; it is a test of whether they are at some specific point in the algorithm.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: