The great UDP tip of the century

TIP:		
Title:		Datagram I/O in Tcl
Author:		Alexandre Ferrieux, Colin McCormack
Type:		Project
Tcl-Version:	8.7
Created:	17-Aug-2012

~ Abstract

This TIP  adds support  for Datagram I/O  in Tcl (starting  with UDP),
with  a  pure  event  approach  to packet  reception  (as  opposed  to
hijacking the existing channel infrastructure).

~ Rationale

UDP support  is a  long-awaited feature in  Tcl. Some  extensions have
traditionally supported it through the channel abstraction. This is of
course doable,  but there is  a nonnegligible cost in  both complexity
and  performance due  to the  impedence mismatch  between  the "fluid"
model behind channels and the "granular" world of packets or datagrams
(where  boundaries  are  significant).   Another  discrepancy  is  the
existence of  (per-packet) metadata (like the source  address and port
of  an  incoming  UDP  packet),  which  does not  fit  nicely  in  the
(per-connection) options of the channel API (via fconfigure).

Once this  mismatch is acknowledged, it  is easy to  identify a better
home for packet I/O  in Tcl: let it be a direct  event source (for the
receive side), just like Tk or Snack.

Indeed, hooking a callback for  packet reception is a natural fit with
Tcl's event-driven  tradition, while preserving  packet boundaries and
easily exposing per-packet metadata.

Sending  is trivially  handled by  a direct  per-packet call  (but not
disguised  as a [puts]).   Again, this  naturally allows  for boundary
preservation and metadata specification.

The  script API  put  forth in  this  TIP is  also  designed for  easy
generalization  to other  kinds of  packet  I/O.  Though  only UDP  is
supported  in the  reference  implementation, other  protocols may  be
registered,  and  selected with  the  ''protocol'' paramater.   Future
additions may include:

    - raw IP sockets
    - USB
    - Bluetooth

~ Overall Specification

The new '''dgram''' command creates a "datagram endpoint" of the given
protocol:

 > '''dgram''' ''protocol'' ?''options''?

Following the  traditions of Tk and  Snack, it returns  a Tcl command,
which takes subcommands implementing  the needed verbs. An endpoint is
thus created by:

 > set d ['''dgram''' ''protocol'' ?-''option'' ''value''? ...]

The returned  command lives by default in  the ::tcl::dgram namespace;
however, the  generic ''-name'' option  allows it to take  a (possibly
fully qualified) user-provided name:

 > dgram ''protocol'' -name foo ...

creates (and returns) the command ::foo to hold the endpoint.

Once created, its configuration can be tweaked by:

 > $d configure -''option'' ''value''

and retrieved with

 > set value [$d configure -''option'']

To close the endpoint, use:

 > $d close

or, as in Tk, destroy the command:

 > rename $d {}

or the whole namespace.

Other    verbs,   and    the   detailed    list   of    options,   are
protocol-specific. From now on,  the specification concentrates on the
UDP case.   The rationale for  *not* generalizing much is  the feeling
that  the costs of  parameter variability  would largely  outweigh the
tiny amount of code sharing.

~ The UDP case

With UDP, addresses and ports may only be specified on creation:

 > set d ['''dgram udp''' ?-localaddr ''address''? ?-localport ''port''?]

They default  to INADDR_ANY and 0,  respectively.  In case  of port 0,
after creation of the endpoint the actual port chosen by the OS can be
retrieved with [$d configure]:

 > set d [dgram udp]
 > puts "Local port is: [$d configure -localport]"

One can also specify remote address and port, meaning to connect() the
underlying UDP socket:

 > set d ['''dgram udp''' ?-remoteaddr ''address'' -remoteport ''port''?]

These options suffer no default, and must be simultaneously present or
absent. The semantics, as is well  known, is to tell the kernel (a) to
forbid  sending to  any  other  destination, and  (b)  to discard  all
incoming packets sent by another source.

All  the  other  options  may  be specified  on  creation  or  through
[configure].

Sending is done with, unsurprisingly, the 'send' verb:

 > $d '''send''' ''payload'' ?''destaddr'' ''destport''?

Its  blocking  semantics  is  that  of  the  underlying  protocol:  it
typically returns immediately, though the hardware may buffer the data
for some time, and delivery is not guaranteed.

The  destination parameters  can,  unsurprisingly, be  omitted if  the
endpoint has been created with the -remoteaddr/port options (connected
mode).

Receiving is done with the -listener callback:

 > $d '''configure -listener''' ''command_prefix''

Subsequently, when an incoming packet  hits Tcl in the event loop, the
''command_prefix'' is called with the endpoint identifier, payload and
metadata:

 > ''command_prefix'' $d ''payload'' ''metadata_dict''

where  ''payload''  is  the  byte   array  of  the  UDP  payload,  and
''metadata_dict'' is a dict containing at least three options:

 >   '''-remoteaddr''' ''address''
 >   '''-remoteport''' ''port''
 >   '''-localaddr''' ''address''

When  ''command_prefix'' is the  empty string,  the notifier  gives up
interest in  the underlying UDP socket;  this allows to  keep the port
bound  while letting  the  OS buffer  any  incoming packets  (up to  a
configurable limit)  without any script-level  handling, while leaving
the event loop active.  This is similar to setting a fileevent to {}.

~ Advanced UDP options

(TODO: multicast, ttl, buffersize ...)

~ Copyright

This document has been placed in the public domain.