CurlPlugin


	links to this page:

CurlPlugin

Last updated at 8:04 pm UTC on 11 August 2009

Plugin code and the Smalltalk interface to the libcurl library: http://curl.haxx.se/ It is probably worth to follow that link to familiarize yourself with a subject.
A short quote from there:

libcurl is a free and easy-to-use client-side URL transfer library, supporting FTP, FTPS, TFTP, HTTP, HTTPS, TELNET, DICT, FILE and LDAP. libcurl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, Kerberos4), file transfer resume, http proxy tunneling and more!
…
libcurl is free, thread-safe, IPv6 compatible, feature rich, well supported, fast, thoroughly documented and is already used by many known, big and successful companies and numerous applications.

If you wish you can track the plugin development on the squeaksource: http://www.squeaksource.com/CurlPlugin/
Instructions to build CurlPlugin are here: CurlPluginBuilding

The use is straightforward in most cases, like this:

    Curl new getContentsUrl: 'http://www.squeak.org/'

However, before using the plugin, it is worth to understand how plugin operates and its relation to the libcurl. A short introduction is below.
User interacts with libcurl by means of Curl instance:

	curl := Curl new.

Curl instance can be thought as an object that has four important variables:

1. curlHandle – this is an ID for plugin’s C-structure. Don’t touch it, just know it is there.
This structure is an external resource and it is quite fat. When Curl instance is being garbage-collected it is freed by finalization. However, it is possible to free resources explicitly, for those who like me doesn't completely trust automatic way (curl instance will be unusable after this certainly):

	curl destroy.

Note, that if you save an image with a transfer running and then open it again, transfer will not resume and the curl instance will not be valid (in rare cases when it is possible it is very difficult to achieve such resuming and it can not be done in a general case). You'll see a walkback hence.

2. url. This is self explanatory – the url for the transfer. It is a string and it supports a variety of formats (http, https, ftp, tftp with username/passwords embedded).

	curl url: 'http://www.squeak.org/'.

3. contents – these are bytes user is downloading or uploading. You will not see this var in inspector because it is being maintained by the plugin side. But again know – it is there.

 curl contents

-> nil for the fresh instance, there is nothing yet in the buffer.

 curl contents: 'hi, there'; contents

-> ‘hi, there’

 curl clearContents; contents

-> nil

4. idleBlock – before the transfer you can define the code to execute on idle. Curl instance will pass itself to this block. Useful when you want to know how it goes for example.

Putting it all together, here is a workspace snippet:

    | curl |
     Transcript open.
     curl := Curl new.
     curl url: 'http://squeak.org'.
     curl idleBlock: [:c | Transcript nextPutAll: (c dataSize printString , ' bytes read.\ ' withCRs); endEntry].
     curl download.
     curl contents openInWorkspaceWithTitle: curl url

Or using shortcuts:

    Transcript open.
    (Curl new
	    idleBlock: [:c | Transcript nextPutAll: (c dataSize printString , ' bytes read.\ ' withCRs); endEntry];
	    getContentsUrl: 'http://squeak.org')  
               openInWorkspaceWithTitle: 'squeak page'

So operation flow is like this:

get new Curl instance
setup it for the transfer (by setting an url, at least)
perform the transfer, catch errors (that do have meaningful text descriptions). You can cancel it and cancel is not an error. BTW, HTTP 404, etc are not errors too, use Curl>>infoResponseCode to track or Curl>>onFailOnError to make them errors
read the contents and the information about the transfer.
free resources.

Most of the above happens transparently.
You can 'copy' between urls without fetching data into smalltalk image. Here is a snippet for getting SqueakV3 sources tarball into your squeak dir:

    Curl new
		getContentsUrl: 'http://ftp.squeak.org/sources_files/SqueakV3.sources.gz'
		putContentsUrl: 'file://' , (FileDirectory default fullNameFor: 'SqueakV3.sources.gz').

Data still visits plugin buffer in the case above.
Here is a more advanced example, when you are running download in the background while being able to cancel it:

| button curl |

	button := ScriptableButton new.

	curl := Curl new
		idleBlock: [:c | button label:
			 (c dataSize printString  , ' of ' , c url , ' downloaded, press me to cancel')];
		yourself.

	button
		actionSelector: #cancel;
		target: curl;
		openInHand.
	

 	[curl
		getContentsUrl: 'http://ftp.squeak.org/sources_files/SqueakV3.sources.gz'
		putContentsUrl: 'file://' , (FileDirectory default fullNameFor: 'SqueakV3.sources.gz').
		WorldState addDeferredUIMessage: [button delete] ] forkAt: Processor userBackgroundPriority

Please note, that although network transfers of the plugin are non-blocking, it does block on 'file//' url operations, so beware.

CurlPlugin strives to provide an easy to use interface for libcurl, but in the same time not to stay on the way between user and library. You can browse libcurl docs when you need an additional information about what is possible and how to achieve that: http://curl.haxx.se/libcurl/c/ I tried to keep corresponding Smalltalk api close to the libcurl one.

Most but not all of the functionality offered by libcurl is available in plugin, and in addition to the first 'but' it is not fully tested (because of number of features).

Some things are left because they are not portable.
Some things are not implemented because they don’t seem worth an effort (resuming, ftp-downloads directly to disk).
Some things are not fully implemented because I don’t have an idea of their typical usage and hence how user api should look like – I need your feedback here! (cookies, http multipart uploads and header manipulations).
Some features just are waiting to be added (ftp pre/postamble and commands).

"Using SSL, point to the certificate file or disable certificate check http://curl.haxx.se/docs/sslcerts.html:"

(Crl new
	caInfo: 'e:\squeak\cacert.pem';
      "offSSLVerifyPeer;" "-this disables CA check"
	getContentsUrl: 'https://www.google.com/index.html')

Both Windows and Linux packs are build against 7.16.0-20060930 and http://gnuwin32.sourceforge.net/packages/openssl.htm 0.9.7c:
CurlPlugin.zip

Windows Zip includes dll-s for openssl, libcurl, the plugin (unpack it into the Squeak folder), SSL certifacate bundle and fileouts:

CurlPlugin.st Plugin code generator, use it with VMMaker
Curl.st - An interface class
CurlFtpTests.st - 'Live' tests for ftp, you need to setup a ftp server to run them
CurlTests.st - user api protocol tests

Linux plugin is currently available as a SAR installer (install it from a file list menu).
It is linked dynamically, so much needed .so libraries should be present for plugin operation (it seems that every modern linux distro has libcurl and companions installed).
SAR installer tries to write CurlPlugin into a VM-plugin dir. If it fails - help it to get there :)
It is for 32bit linux: CurlPlugin-linux32-1.sar