Copas - Coroutine Oriented Portable Asynchronous Services for Lua

Copas
Coroutine Oriented Portable Asynchronous Services for Lua

## Installing

You can install Copas using LuaRocks:

    luarocks install copas

Note: LuaSec is not automatically installed as a dependency. If you want to use ssl with Copas, you need to manually install LuaSec as well.

## Runtime

Copas can either be used as a regular Lua library, or as a runtime. A command line script that acts as a runtime engine is included. When using the runtime, the library is available as a global (`copas`), and the scheduler will automatically be started. For example:

      #!/usr/bin/env copas

      local count = 0
      copas.timer.new {
        delay = 1,
        recurring = true,
        callback = function(self)
          count = count + 1
          print('hello world ' .. count)
          if count >= 5 then
            self:cancel()
          end
        end
      }

## Introduction to Copas

Copas is a dispatcher that can help a lot in the creation of servers based on LuaSocket. Here we present a quick introduction to Copas and how to implement a server with it.

Assuming you know how to implement the desired server protocol, the first thing you have to do in order to create a Copas based server is create a server socket to receive the client connections. To do this you have to bind a host and a port using LuaSocket:

    server = socket.bind(host, port)

Then you have to create a handler function that implements the server protocol. The handler function will be called with a socket for each client connection and you can use `copas.send()` and `copas.receive()` on that socket to exchange data with the client.

For example, a simple echo handler would be:

    function echoHandler(skt)
      while true do
        local data = copas.receive(skt)
        if data == "quit" then
          break
        end
        copas.send(skt, data)
      end
    end

You may alternatively use `copas.wrap()` to let your code more close to a standard LuaSocket use:

    function echoHandler(skt)
      skt = copas.wrap(skt)
      while true do
        local data = skt:receive()
        if data == "quit" then
          break
        end
        skt:send(data)
      end
    end

To register the server socket with Copas and associate it with the corresponding handler we do:

    copas.addserver(server, echoHandler)

Finally, to start Copas and all the registered servers we just call:

    copas()

As long as every handler uses Copas's `send` and `receive`, simultaneous connections will be handled transparently by Copas for every registered server.

Since Copas is coroutine based, using it within a Lua `pcall` or `xpcall` context does not work with Lua 5.1 yielding. If you need to use any of those functions in your handler we strongly suggest using coxpcall, a coroutine safe version of the Lua 5.1 protected calls. For an example of this usage please check Xavante.

## Why use Copas?

For those who already have a server implemented, here is an explanation of why and how to migrate to Copas. In a typical LuaSocket server usually there is a dispatcher loop like the one below:

    server = socket.bind(host, port)
    while true do
      skt = server:accept()
      handle(skt)
    end

Here `handle` is a function that implements the server protocol using LuaSocket's socket functions:

    function handle(skt)
      ...
      -- gets some data from the client - "the request"
      reqdata = skt:receive(pattern)
      ...
      -- sends some data to the client - "the response"
      skt:send(respdata)
      ...
    end

The problem with that approach is that the dispatcher loop is doing a busy wait and can handle just one connection at a time. To solve the busy waiting we can use LuaSocket's `socket.select()`, like in:

    server = socket.bind(host, port)
    reading = {server}
    while true do
      input = socket.select(reading)
      skt = input:accept()
      handle(skt)
    end

While this helps our CPU usage, the server is still accepting only one client connection at a time. To handle more than one client the server must be able to multitask, and the solution usually involves some kind of threads.

The dispatcher loop then becomes something like:

    server = socket.bind(host, port)
    reading = {server}
    while true do
      input = socket.select(reading)
      skt = input:accept()
      newthread(handle(skt))
    end

where `newthread` is able to create a new thread that executes independently the handler function.

The use of threads in the new loop solves the multitasking problem but may create another. Some platforms does not offer multithreading or maybe you don't want to use threads at all.

If that is the case, using Lua's coroutines may help a lot, and that's exactly what Copas does. Copas implements the dispatcher loop using coroutines so the handlers can multitask without the use of threads.

## Using Copas with an existing server

If you already have a running server using some dispatcher like the previous example, migrating to Copas is quite simple, usually consisting of just three steps.

First each server socket and its corresponding handler function have to be registered with Copas:

    server = socket.bind(host, port)
    copas.addserver(server, handle)

Secondly the server handler has to be adapted to use Copas. One solution is to use Copas `send` and `receive` functions to receive and send data to the client:

    function handle(skt)
      ...
      -- gets some data from the client - "the request"
      reqdata = copas.receive(skt, pattern)
      ...
      -- sends some data to the client - "the response"
      copas.send(skt, respdata)
       ...
    end

The other alternative is to wrap the socket in a Copas socket. This allows your handler code to remain basically the same:

    function handle(skt)
      -- this line may suffice for your handler to work with Copas
      skt = copas.wrap(skt)   -- or... skip this line and wrap `handle` using copas.handler()
      -- now skt behaves like a LuaSocket socket but uses Copas'
      ...
      -- gets some data from the client - "the request"
      reqdata = skt:receive(pattern)
      ...
      -- sends some data to the client - "the response"
      skt:send(respdata)
       ...
    end

Note that by default Copas might return different timeout errors than the traditional Lua libraries. Checkout `copas.useSocketTimeoutErrors()` for more information.

Finally, to run the dispatcher loop you just call:

    copas()

During the loop Copas' dispatcher accepts connections from clients and automatically calls the corresponding handler functions.

## Using UDP servers

Copas may also be used for UDP servers. Here is an example;

    local port = 51034
    local server = socket.udp()
    server:setsockname("*",port)

    function handler(skt)
      skt = copas.wrap(skt)
      print("UDP connection handler")

      while true do
        local s, err
        print("receiving...")
        s, err = skt:receive(2048)
        if not s then
          print("Receive error: ", err)
          return
        end
        print("Received data, bytes:" , #s)
      end
    end

    copas.addserver(server, handler, 1)
    copas()

For UDP sockets the `receivefrom()` and `sendto()` methods are available, both for copas and when the socket is wrapped. These methods cannot be used on TCP sockets.

IMPORTANT: UDP sockets do not have the notion of master and client sockets, so where a handler function can close the client socket for a TCP connection, a handler should never close a UDP socket, because the socket is the same as the server socket, hence closing it destroys the server.

NOTE: When using the `copas.receive([size])` method on a UDP socket, the `size` parameter is NOT optional as with regular luasocket UDP sockets. This limitation is removed when the socket is wrapped (it then defaults to 8192, the max UDP datagram size luasocket supports).

## Adding tasks

Additional threads may be added to the scheduler, as long as they use the Copas `send`, `receive` or `sleep` methods. Below an example of a thread being added to create an outgoing TCP connection using Copas;

    local socket = require("socket")
    local copas = require("copas")

    local host = "127.0.0.1"
    local port = 10000

    local skt = socket.connect(host, port)
    skt:settimeout(0)  -- important: make it non-blocking

    copas.addthread(function()
       while true do
          print("receiving...")
          local resp = copas.receive(skt, 6)
          print("received:", resp or "nil")
          if resp and resp:sub(1,4) == "quit" then
             skt:close()
             break
          end
       end
    end)

    copas()

The example connects, echoes whatever it receives and exits upon receiving 'quit'. For an example passing arguments to a task, see the async http example below.

## Creating timers

Timers can be created using the `copas.timer` module. Below an example of a timer;

    local copas = require("copas")

    copas(function()
       copas.timer.new({
         delay = 1,                        -- delay in seconds
         recurring = true,                 -- make the timer repeat
         params = "hello world",
         callback = function(timer_obj, params)
           print(params)                   -- prints "hello world"
           timer_obj:cancel()              -- cancel the timer after 1 occurence
         end
       })
    end)

The example simply prints a message once every second, but gets cancelled right after the first one.

## Synchronization primitives

Since Copas allows to asynchroneously schedule tasks, synchronization might be required to protect resources from concurrent access. In this case the `copas.lock` and `copas.semaphore` classes can be used. The lock/semaphore will ensure that the coroutine running will be yielded until the protected resource becomes available, without blocking other threads.

    local copas = require("copas")

    local lock = copas.lock.new()

    local function some_func()
      local ok, err, wait = lock:get()
      if not ok then
        return nil, "we got error '" .. err .. "' after " .. wait .. " seconds"
      end

      print("doing something on my own")
      copas.pause()  -- allow to yield, while inside the lock
      print("after " .. ok .. " seconds waiting")

      lock:release()
    end

The `some_func` function may now be called and the 2 lines will be printed together because of the lock.

## Ssl support

LuaSec is transparently integrated in the Copas scheduler (though must be installed separately when using LuaRocks).

Here's an example for an incoming connection in a server scenario;

    function handler(skt)
      skt = copas.wrap(skt):dohandshake(sslparams)
      -- skt = copas.wrap(skt, sslparams):dohandshake()  -- would be identical

      while true do
        -- perform the regular reading/writing ops on skt
      end
    end

A simpler handler would wrap the handler function to do the wrapping and handshake before the handler gets called;

    function handle(skt)
      -- by now `skt` is copas wrapped, and has its handshake already completed

      while true do
        -- perform the regular reading/writing ops on skt
      end
    end
    handle = copas.handler(handle, sslparams)  -- wraps the handler to auto wrap and handshake

Here's an example for an outgoing request;

    copas.addthread(function()
      local skt = copas.wrap(socket.tcp(), sslparams)
      skt:connect(host, port)  -- connecting will also perform the handshake on a wrapped socket

      while true do

        -- perform the regular reading/writing ops on skt

      end
    end

## High level requests

For creating high level requests; http(s), ftp or smtp versions of the methods are available that handle them async. As opposed to the original LuaSocket and LuaSec implementations.

Below an example that schedules a number of http requests, then starts the Copas loop to execute them. The loop exits when it's done.

    local copas = require("copas")
    local asynchttp = require("copas.http").request

    local list = {
      "http://www.google.com",
      "http://www.microsoft.com",
      "http://www.apple.com",
      "http://www.facebook.com",
      "http://www.yahoo.com",
    }

    local handler = function(host)
      res, err = asynchttp(host)
      print("Host done: "..host)
    end

    for _, host in ipairs(list) do copas.addthread(handler, host) end
    copas()

## Controlling Copas

If you do not want copas to simply enter an infinite loop (maybe you have to respond to events from other sources, such as an user interface), you should have your own loop and just call `copas.step()` at each iteration of the loop:

    while condition do
      copas.step()
      -- processing for other events from your system here
    end

When using your own main loop, you should consider manually setting the `copas.running` flag.

Valid XHTML 1.0!

$Id: manual.html,v 1.19 2009/03/24 22:04:26 carregal Exp $