Last updated at 1:03 pm UTC on 16 January 2006
Some work is now on SM, see SUnit Test Runner in 3.8's latest version.
A blackboard to lay down ideas for the next version of
SUnit Test Runner in 3.8. We want to have easier access to
testing information . And easier manipulation of the test.
That way we may end with a fully-tested Squeak :-).
Main axes :
(TODO : neatly recategorize all the items down there in the categories)
All of this to encourage people to test things !
- Ease Test-driven development
- Use test/assertions as documentations. allow for an easy access
- Test-specific refactorings
- Ease test creation/test running/test feedback
So here they are :
- easily make a new test (BrowseUnitv4 does not generates tests for now)
- make test-driven development/"programming in the debugger" easier :
- automatically adds unknown ivars (I just sent a FIX to Lukas' code to the list for that)
- automatically creates classes (sent an ENH for that)
- color methods relative to unit tests : green if they are tested, red if they are not.
- find the type of the method (return type and type arguments), using information in the tests. Show it in the annotation pane ?
- I find that to be an important part which should be further developped. Having the type of the object at hand would ease navigation further (ie browse protocol of arguments/return type), and code understanding too. We just have to find a way to do it ...– Romain Robbes
- find a way to cache things to make it faster (maybe using SystemEventNotification)
- BrowseUnitTestCase : using a subclass of testcase might ease things, as we can override the run method to do interesting stuff (like type detection maybe)
- Check Colin's OmniBrowser to see if it might be easier to extend than the System Browser. See how to add tabs in a Browser, to display additional information (tests, slint)
Test-specific refactorings : additional items in the RB while browsing a TestCase.
This seems an interesting path to explore.
- extract to result (incorporate Markus Gaelli's one)
- extract to resource (extract expression and append it to the setup method of the test ?)
- extract to persistent resource (extract to a TestRessource instance)
Some ideas for BrowseUnit
594 of 617 sends of #assert: in a current image are sent in a unit test.
I think this is a bad thing, because
- our program is abstract, why shouldn't our assertions be?
- difficult to compose tests
- One fault causes several tests to break, if you have independent broken tests you don't know with which test to start. Abstract assertions / pre- postconditions will on the other hand fail always first in the most specific situation.
Enhance the notion of a method test in the following way:
A method is tested by some unit test, if
(Sorrow, OOPAL and Lispy Forth)
- the test executes the method and
- has an assertion which only holds after this method has been called
- or, even better, the method has a postcondition
- If performance is really degenerated too much because of assertions, one could easily make them pluggable.
So, in short, you want to use pre and postconditions instead or in addition of standard unit tests ?
In addition. I mean that is what "both sides" get wrong: The design by contract -people overrate the abstract and the XPers the concrete part. Truth is like always somewhere inbetween ;-)
We need the concrete scenarios (method examples) as inputs for the abstract assertions. How should the-damned-to-be-concrete unit tests otherwise cope with the more and more abstract programs?
- I don't follow you very well on your abstract vs concrete debate. Could you explain further, possibly giving "explanation examples" ;-) ? – Romain Robbes
That's quite a drastic change of focus ... how does that fit with method examples ?
No - and very well I think. A method example is a test, if
- it includes a concrete assertion at the end
- or if its method under test includes a post condition
There's some stuff about implementing assertions by modifying the compiler there :
But we already have the assert: in Object>>assert:
So one could easily modify this to be executed on a per package basis, and add some global triggers. – Sorrow, OOPAL and Lispy Forth
That could be the first version. Vassily's approach (I reread quickly the article) says that it can impose no overhead at all, so it would be interesting in the long run. – Romain Robbes
- By the way I checked the code Vassily gives. It was from Squeak 2.8, so I checked it and maked it work on 3.7. I have now to repackage it correctly, maybe add tests, and post it to the list. So the assertion mechanism DO work, and effectively adds no overhead when it is disabled. – Romain Robbes
- Great that you digged that out! When you send this, hopefully you can motivate people to write more pre- and postconditions. I just counted and got: 1024 unit tests but only 23 assertions sent in pre- or postconditions. Sorrow, OOPAL and Lispy Forth