Distribution of characters in Squeak code
Last updated at 11:34 am UTC on 24 June 2016
Tobias Pape (Squeak Mailing list Wed, Jun 22, 2016)
I was curious about the relative distribution of characters in Squeak Code.
I sampled the source code and drew a histogram (Attached)
" Uses the new HistogramMorph"
| characterFrequency |
CurrentReadOnlySourceFiles cacheDuring: [
characterFrequency := ((CompiledMethod allInstances select:
[:method | (method allLiterals detectSum:
[:lit | lit isCollection ifFalse: [0] ifTrue: [lit size]]) 1500])
gather: [:method | method getSource
reject: [:c |c isSeparator]]) asBag].
(HistogramMorph on: characterFrequency)
labelBlock: [:c | c codePoint > 32 ifTrue:[c asString] ifFalse: [c printString]];
openInWorld.
((characterFrequency sortedCounts collect: [:ea | ea value]) first: 90) join.
Result
- The most frequent (printable) characters are in order
etarsoinl:
and more detailed, the 90 most frequent characters:
etarsoinl:cdfumhpg.ybwSv"=1CT'x][0F)(k2ANPI|M^B4O7D6R3598#EL-,zWVjU;H+q/>G@KX${}YQZJ\~?!
- This is quit close to actual English:
etaonishrlducmwyfgpbvkjxqz
Observations:
- The most frequent punctuation is : and . follows quite long after.
- Cascading is comparatively rare. We have more blocks and equality/identity comparisons than ;
- Blocks are more common than parenthesis and literal arrays
- You cannot spell ifTrue or ifFalse with the 20 most common characters
- ifTrue: is far more common than ifFalse:
- The most frequent uppercase Character is S. I have no conjecture here, tho.
Comparison:
C, sampling the Linux kernel:
et_risancodlupfm,);(0hvgb-E=x>ITRSACkNL.P1O/wD2My"{}UF&3GB4q86HV5:X#[]+zK7W9Y|%\!jQZ'
- under_score_case vs. camelCase is rather obvious.
- (not displayed but tab and newline are amog the 6 most frequent characters!)
- Punctuation starts much earlier.
- The beginning differs a lot, the ending not so much.
- 0 is far more important than 1
- : is unimportant
Ruby, sampling Rails:
etsaonridl_cupmh.f:,"gb')(=y#vw/kq>ATx01R[]@S{}CE|2?-zjDMIPN+BO\F3L5!HU%&498GW6;YV7J`X
- underscore shows, but not so much as in C.
- The : is (like in Smalltalk) more important
- Uppercase is more uncommon than in both C and Smalltalk.
