A blackboard to lay down ideas for the next version of SUnit Test Runner in 3.8. We want to have easier access to
testing information . And easier manipulation of the test.
That way we may end with a fully-tested Squeak :-).
Main axes :
(TODO : neatly recategorize all the items down there in the categories)
Ease Test-driven development
Use test/assertions as documentations. allow for an easy access
Test-specific refactorings
Ease test creation/test running/test feedback
All of this to encourage people to test things !
So here they are :
easily make a new test (BrowseUnitv4 does not generates tests for now)
make test-driven development/"programming in the debugger" easier :
automatically adds unknown ivars (I just sent a FIX to Lukas' code to the list for that)
automatically creates classes (sent an ENH for that)
color methods relative to unit tests : green if they are tested, red if they are not.
find the type of the method (return type and type arguments), using information in the tests. Show it in the annotation pane ?
I find that to be an important part which should be further developped. Having the type of the object at hand would ease navigation further (ie browse protocol of arguments/return type), and code understanding too. We just have to find a way to do it ...– Romain Robbes
find a way to cache things to make it faster (maybe using SystemEventNotification)
BrowseUnitTestCase : using a subclass of testcase might ease things, as we can override the run method to do interesting stuff (like type detection maybe)
Check Colin's OmniBrowser to see if it might be easier to extend than the System Browser. See how to add tabs in a Browser, to display additional information (tests, slint)
Test-specific refactorings : additional items in the RB while browsing a TestCase.
This seems an interesting path to explore.
extract to result (incorporate Markus Gaelli's one)
extract to resource (extract expression and append it to the setup method of the test ?)
extract to persistent resource (extract to a TestRessource instance)
594 of 617 sends of #assert: in a current image are sent in a unit test.
I think this is a bad thing, because
our program is abstract, why shouldn't our assertions be?
difficult to compose tests
One fault causes several tests to break, if you have independent broken tests you don't know with which test to start. Abstract assertions / pre- postconditions will on the other hand fail always first in the most specific situation.
Solution
Enhance the notion of a method test in the following way:
A method is tested by some unit test, if
the test executes the method and
has an assertion which only holds after this method has been called
or, even better, the method has a postcondition
If performance is really degenerated too much because of assertions, one could easily make them pluggable.
So, in short, you want to use pre and postconditions instead or in addition of standard unit tests ?
In addition. I mean that is what "both sides" get wrong: The design by contract -people overrate the abstract and the XPers the concrete part. Truth is like always somewhere inbetween ;-)
We need the concrete scenarios (method examples) as inputs for the abstract assertions. How should the-damned-to-be-concrete unit tests otherwise cope with the more and more abstract programs?
I don't follow you very well on your abstract vs concrete debate. Could you explain further, possibly giving "explanation examples" ;-) ? – Romain Robbes
That's quite a drastic change of focus ... how does that fit with method examples ?
No - and very well I think. A method example is a test, if
it includes a concrete assertion at the end
or if its method under test includes a post condition
There's some stuff about implementing assertions by modifying the compiler there : http://www.smalltalkchronicles.net/edition2-1/st_compiler.htm. – Romain Robbes
But we already have the assert: in Object>>assert:
So one could easily modify this to be executed on a per package basis, and add some global triggers. – Sorrow, OOPAL and Lispy Forth
That could be the first version. Vassily's approach (I reread quickly the article) says that it can impose no overhead at all, so it would be interesting in the long run. – Romain Robbes
By the way I checked the code Vassily gives. It was from Squeak 2.8, so I checked it and maked it work on 3.7. I have now to repackage it correctly, maybe add tests, and post it to the list. So the assertion mechanism DO work, and effectively adds no overhead when it is disabled. – Romain Robbes
Great that you digged that out! When you send this, hopefully you can motivate people to write more pre- and postconditions. I just counted and got: 1024 unit tests but only 23 assertions sent in pre- or postconditions. Sorrow, OOPAL and Lispy Forth