DOC's Demo's - FTSE100 LeaderBoard
Last updated at 11:06 pm UTC on 8 November 2006
This is part of a (loosely connected) set of beginners tutorials done by a beginner (shock, horror). The entry point for these tutorials is DOC's Demo's
1. BackgroundThe FTSE100 is one of several "market indices" used in the UK as a sort of weather vane for the state of the UK financial market as a whole. It is an index continuously computed from gains/loses in the shares of the top 100 UK companies. In actual fact the number of companies may be 100 +/- a few, the calculation is not so straight forward as it may first seem and the use of any index as anything other than a momentary indicator is dubious at best. A "LeaderBoard" is a presentation tool that provides a bit more insight into what is happening by displaying the individual constituents of an index in a spreadsheet-like format. Why don't I just show you and then explain further...
On the left image is the results of this tutorial, a LeaderBoard for the FTSE100 constituents on a particular date at a specific time. Each cell represents one company designated by it's EPIC code (LSE's unique abbreviation of the company name) and shows the percentage increase of it's share price compared to the price when the market opened on that day. For (very effective) visual clues each cell background is either WHITE (no change), RED (falling) or BLUE (rising). Hence, even at a distance one can tell if, for example, it is generally a "bad day on the markets" (mostly red) or alternatively just bad for a few companies. Note that by itself an index won't convey this distinction because some companies carry more "weight" than others when computing the index.
The right image above shows a leaderboard in a 3D scene in a simulated big screen monitor. The leaderboard data is used in the dynamic configuration of the circular table in the foreground. In fact the raw data provided me with the original impetus to learn 3D programming and attempt to emulate an effect I had seen on tv (successfully at a fraction of the cost!).
So there you have it, a hopefully concise and clear description of a FTSE100 LeaderBoard. Easy peasy! Except to create one we have to source the data, download/store it, then display it. Oh and it would be nice to do this continuously, in real-time and for free. Whoa there! Let's keep this practical! "Continuously"... ok-ish. "Real-time and free"... not! You can get real-time market data but it will cost and if you can afford it then you won't need my flimsy leaderboard ;-) Mere mortals, ie, individual small-holding investors, AKA: the scummy underclass of "investors", have to be more realistic in our requirements. We can get "T15" data for free, ie, real (but not "real-time") data delayed by 15 minutes (supposedly so we don't upset the pin-stripe boys while they play their game in the "fair and open free-market economy" - yeah right (sarcasm, if you don't recognise it)). You can get "continuous" T15 data but again it will probably cost you. In practise, as is the case here, we use T15 data updated anything between 5 and 15 minutes. In conclusion, we are dealing with delayed data updated every 10 minutes or so, and if your requirements are more immediate then look elsewhere. This is after all only a tutorial!
NB: The fact that I choose the FTSE100 LeaderBoard as an example is because it is a relatively simple concept and one that I am familiar with after implementing them in Excel and other (non-smalltalk) languages, and as the basis for a learning 3D programming. I am in no way suggesting anyone invest money in the stock market and even if you do then I would advise against using my efforts as the basis for any investment decisions. I'm sure you wouldn't anyway, but just to be clear on this point this is my disclaimer, okaaaaaay?
2. The Tutorial Begins... At Last!I wish! Or rather you do? Whatever. First we need to scope out our requirements. Didn't we already do this above? Nooooope. As a beginner and especially as "procedural" programmer I now have to give up my shady past and begin thinking in "Object Oriented" terms. Well okay then! I close my eyes and enter a MATRIX-like world where I can easily reconfigure my mental faculties... it turns out I have been dealing with "objects" all my life and (tounge-in-cheek) simply have to dumb down my thinking. I open my eyes and begin: we have three objects (or "classes" if you want to be pedantic)...
2.1 ClassesA) MarketPrice: This is the class instances of which will represent the price information for specific shares.
B) MarketSnapshot: This is the class instances of which will represent prices on a specific date at a specific time.
C) FTSE100LeaderBoard: This is the class representing the display as seen in the lefthand image above.
As this is the first version I am shamelessly taking some shortcuts in my design. First my MarketPrice objects will be relatively dumb, really just storage devices. I will only have one MarketSnapshot object but it will be responsible for getting real data. So not so dumb as MarketPrice but on the other hand not itself very smart, eg, it won't keep a history of prices (this may not be so dumb later on). The FTSE100LeaderBoard object will create itself a Marketsnapshot object and if the snapshot is successfully updated then feed data into some grid-like display widget. If this seems a little too specific/vague then so be it - remember I'm a beginner but with a little hindsight because I have already implemented the focus of this tutorial. Hopefully the way I present the information will help guide other beginners.
We can go ahead and create the classes. You should have a Class/System Browser up with a new category created and selected. If you are lost at this stage then smack yourself on the forehead while shouting "DOH!" and then see step 5) on DOC's Demo's. Notice in the bottom pane of the browser is a class template with first line "Object subclass: #NameOfSubclass" (it's the "#NameOfSubclass" part that gives the game away). Go ahead and replace "NameOfSubclass" with "MarketPrice" and notice the pane now has a red border indicating you have made changes. To store the change, ie, create the class, either right-click (in the lower pane) and select "accept" or use key combo "alt-s" with the mouse pointer over that area(?). You should now see the class entry in the upper part of the browser (second listbox from the left). Now for the "MarketSnapshot" class: simply replace "MarketPrice" with "MarketSnapshot" and accept again. Easy! So do do the same for "FTSE100LeaderBoard" remembering to accept. All three classes should now be listed.
2.2 MarketPriceFor a moment I slip into old frowned-upon procedural mode and remember something about data dictating actual program design. So focusing on the data for a moment here is what we are using. The site http://www.moneyextra.com (no affiliation what so ever) kindly provides free T15 data in an easy to process tsv (Tab-Separated Values) text file. Here's a sample of the beginning of one file (titles may not display inline with data):
Data collected at 11:40, 13/10/06
Epic Name Mid Change-p Change-% Bid Offer Open High Low Close Volume
III 3i Grp Plc 989.00 5.00 0.50 987.50 990.00 981.00 990.00 976.00 984.00 577910
ABF AB Food 823.00 -3.00 -0.30 822.50 823.00 825.00 825.00 820.50 826.00 828303
Looking at this data we see the first line indicates when the data was collected, the second line is blank, the third line contains field names and actual prices follow in subsequent lines. Notice the field names and how they are almost perfect contenders for Instance Variable names for the MarketPrice class. We just have to get rid of the capitals, the tabs and abbreviate a few, ending up with the following:
epic name mid change changePct bid offer open high low close volume
Copy these, go back to the browser, select the MarketPrice class and paste them between the quotes on the line beginning "instanceVariableNames:" and accept the change. Note for now that we have preserved the order found in the data (relevent later on). We will return to MarketPrice if and when we need to but for now we consider it complete (I did say it was a dumb object). Here on things get a bit more involved so lets further refine the definition of MarketSnapshot.
2.3 MarketSnapshotMarketSnapshot's purpose is to provide a snapshot of prices. It has to know where to get the data from and once obtained has to create a collection of prices (MarketPrice instances to be specific). One more bit of useful information would be a date stamp of when the data was collected. In summary: using a source perform an update creating a prices collection and extracting a date-stamp. So instances of the MarketSnapshot class need an update method and the following instance variables: source dateStamp prices. In the browser select the MarketSnapshot class and enter the instance variables as per MarketPrice. Then click in the third listbox in the upper part of the browser and notice the lower pane now has a method template. Replace the first line with "update" (no quotes) and delete the lines below the comment and accept. At this stage you may get prompted for your initials so just enter them and accept. This is to do with tagging changes but is not relevent here. "update" should now be listed in the forth (right-most) list box and hightlighted.
A point of confusion occurred for me here: I tried to get back to the class definition by clicking on MarketSnapshot and nothing happened. So then I clicked on the "class" button at the bottom of the second list box but then my method list dissappears. My newbie-ness shows as what I am now looking at is the definition for the class itself not instances. I click on the "instance" and get to where I wanted to but the indirect route bugs me. Eventually I realised that de-selecting either the method or method category (third pane) drops me back to the instance definition - not at all obvious!
We will come back to "update" in a moment but first let's provide "accessors" for the instance variables. Accessors are the methods needed to get/set instance variables because instance variables are private to their class. We would like to be able to read all three but only "source" would be set from outside the class. Click in the method categories box again (third list) to get a template and enter the following and accept it:
Again no need to go back to a template for the other two, simply enter "dateStamp" and "prices" inplace of "source", accepting each before moving to the next one. That completes our "get" accessors. Now we just need a "setter" for "source" so click on "source" in the methods list (righthand list) and modify as follows:
source := aString
Did you remember to accept? Sure you did! Now would be a good time to save as well.
2.3.1 "Sample some Fudge" or "Fudge a Sample"Let the SOURCE be with you: It's an imperfect world but because this is a demo, my first no less, I am fudging my design by providing sample source data in the code itself. This allows to me to get things working without hitting the internet every time I test changes. To do this I used a normal web browser to directly download data from http://www.moneyextra.com/stocks/ftse100/ftse100.tsv. Then in MarketSnapshot I added a CLASS variable called "Sample" (note uppercase initial letter) in the line below instance variables. I want to set "Sample" before creating any instances so I use a CLASS initialization method. You can do this by hitting the "class" button in the class list then clicking in the methods category list as before to get a method template then enter and accept the following:
Now simply open up the data file in a text editor, copy all the contents, paste them between the single quotes in the code above and accept again.
Note: In recreating the code for the purpose of writing this tutorial I now had to explicitly initialise the class to get the data actually into the class variable "Sample". I don't remember having to do this in the original code? If you are a beginner following this then open a "workspace" via the World menu "open..." item, enter "MarketSnapshot initialize" and with the cursor on the same line hit the alt-d key combo.
2.3.2 "Update" MethodNow we can return to the "update" method. To do so you need to hit the "instance" button again to get back to instance methods and the select "update" in the methods list. Here is the code in full followed by an explanation:
"Update prices (but only if newer!)"
"Test using: MarketSnapshot new update; yourself; inspect"
| data newPrices lineCount newDateStamp |
"Download the data or use sample if no source..."
ifTrue: [data := Sample]
ifFalse: [data := (HTTPSocket httpGetDocument: source) content withSqueakLineEndings].
"Process the data..."
newPrices := OrderedCollection new. "for storing new prices in file order"
lineCount := 0.
data linesDo: [:dataLine |
lineCount := lineCount+1.
"First line contains date stamp so extract it..."
(lineCount = 1) ifTrue: [
newDateStamp := dataLine copyFrom: (dataLine size) - 14 to: dataLine size.
(newDateStamp = dateStamp) ifTrue: [^false]
"Lines 4 onwards contains prices..."
(lineCount > 3) ifTrue: [ | fieldCount newPrice |
newPrice := MarketPrice new.
"Use the fact that we have preserved field order to extract price info..."
fieldCount := 0.
dataLine tabDelimitedFieldsDo: [:dataField |
fieldCount := fieldCount + 1.
newPrice instVarAt: fieldCount put: dataField.
newPrices add: newPrice.
"If we get to this point without problems we have new data..."
dateStamp := newDateStamp.
prices := newPrices.
The code above should be clear enough, it downloads data (or uses the sample), then processes each line. It extracts the date stamp from the first line and does a simple comparison to the existing date stamp. No difference indicates same data and we bail out early from the method. If it does differ then the assumption is that this is newer data and we proceed to process other lines. We are now only interested in lines 4 onwards which contain actual price information. It is debateable if the next bit is good OO design but it definately is convenient. When we created the instance variables for the MarketPrice class we preserved the order of fields in the data. This now allows setting of instance variables by position rather than name. Finally, after processing is complete the actual instance variables of MarketSnapshot are replaced with new data. Experience tells me that there are several problems that may occur in the data and I will eventually have to account for them in my design, so it is prudent to leave any existing data intact until a full and complete update has been performed. The method returns "true" to indicate a successful update.
So does it work? If you look at the second comment in the code above it says: "Test using: MarketSnapshot new update; yourself; inspect". A great feature of Squeak is the ability to run code from almost anywhere. Select the comment starting at "MarketSnapshot" and upto but excluding the final double quote. Now right-click and select "do it" or simply do the alt-d key combo. You should get an "inspector" that lets you look into the contents of an object instance and clicking on "dateStamp" should show the date part of the first line of the sample data. Cool eh? Now onto the final section to get a working example.
2.4 FTSE100LeaderBoardThis class brings everything together but we are still missing an essential component, ie, a means to display data in a spreadsheet/grid-like format. In display terms we have to deal with Morphs (see Morph) but as this is an introductory level tutorial the subject of Morphs is outside its scope. I also hazzard a guess that this is such a common requirement that someone will have already created such a Morph. Sure enough a quick search on http://www.squeaksource.com using "grid" as the keyword reveals "SGrid" by John Pierce. The SqueakSource site is the repository for Squeaks package manager, which unsuprisingly is called "SqueakMap Package Loader" (typically just referred to as "SqueakMap"). Open one using the menu "World/open.../SqueakMap Package Loader". First thing to do is right-click in the upper of the two list boxes on the left and select "update map from net" (also do this periodically to get information on the latest packages/versions). When finished enter "sgrid" in the search box at the top left and hit the enter key. An "SGrid()" package entry should now be highlighted in red but note there is no number in the brackets (this is because it is not installed). Right-click on the entry and select "install". For now just select "yes" to the next two prompts. Successful install will be indicated by having a number in the brackets (this being the installed version number).
A Brief Rant: At this point I have to admit to knowing very little about morphs and the SGrid's examples did not reveal much about how to use it in the way I intended. So I had to make a guess based on SGrid's classes/methods and even by trawling the code itself a little. The general ethos in the Smalltalk community is that code is self-documenting. I come from an era that believed code only ever documents "what you did" not "what you intended to do". Also without examples covering complete functionality one is almost forced to start disecting a classes innards and this doesn't gel well with OOP's fundamental concept of "encapsulation". Arh well, I suppose I should be grateful for the prepackaged excuse for not documenting my own efforts... but then I realise what I'm writing... I slap myself on the forehead and shout "DOH!"... my dog runs and hides under the table (I don't know why. I swear I only kicked him the once, long ago).
So back to the FTSE100LeaderBoard class, speeding up a little and cutting some corners (which I will repair at a later date). No instance variables required but I do want to change the background colour of each cell. At this stage we don't know about "class extensions" (do we?) so, like a magician I conjour this method and add it to my FTSE100LeaderBoard class:
cell: aCell color: aColor
"Set background colour of cell"
aCell contentMorph backgroundColor: aColor.
FTSE100LeaderBoard needs to know two things about a share price: summary information and percentage change in price. It is the MarketPrice object's responsibility to supply these so we go back to MarketPrice and add these two methods:
Now we can add an "instance" intialisation method to FTSE100LeaderBoard:
"Display a leader board of the FTSE100"
| snapshot grid cols r c cell colsign |
"Get a snapshot..."
snapshot := MarketSnapshot new.
(snapshot update) ifFalse: [^ nil].
"Create and prepare the grid..."
grid := GridMorph new: 20. "create with 20 rows"
grid title: 'Leader Board (FTSE100 - ',snapshot dateStamp,')'.
cols := 1 + (snapshot prices size // 20).
1 to: cols do: [:i | grid addColumn: ('EPIC PCT -',i asString)].
"Populate the grid..."
r := c := 1.
snapshot prices do: [:price |
cell := ((grid at: r) at: c).
colsign := price changePct sign.
(colsign = -1)
ifTrue: [self cell: cell color: Color red]
(colsign = 0)
ifTrue: [self cell: cell color: Color white]
ifFalse: [self cell: cell color: Color blue]
cell contents: price summary.
r := r + 1. (r = 21) ifTrue: [c := c + 1. r := 1].
"Display the grid..."
First we create a new instance of a Marketsnapshot and update it. If the update failed we bail out early. Next a new gridMorph is created and initialised with twenty rows and enough columns for to display data for around a hundred shares. The only thing to note here is each column header is initialised with some text even though the header row is already hidden. This is done to ensure enough space in each column for the share price summary. There may be a better or more interactive way to do this but it seemed sufficient from my short investigation of SGrid.
Next we populate the grid in column-first order, setting the background colour according to the "sign" of the price change and cell contents to whatever the price "summary" method returns. Finally we open the grid in the Squeak world.
3 Final ThoughtsWIP: notes, todo
- a thought
- I don't have a dog and wouldn't kick it if I did ;-)