• Logistics:
    • One more homework; will be released next Monday.

Control-flow

  • The next module for this course will be on control flow
  • So far, all of the programming languages we have seen have had relatively simple control-flow structure. So far we have seen two primary forms of control flow:
    1. Function calls.
    2. If expressions/statements
  • Practical programming languages often have more interesting ways of modifying control flow, for example exceptions. Consider the following OCaml code:
exception Oops
    
let () = 
  try 
    let x = 10 in 
    if x < 4 then raise Oops else Format.printf "aaah"
  with 
    Oops -> Format.printf "ouch!"
  • In a similar vein, many languages (like Python, C, Java) support a return statement that immediately returns a value and ignores all other code in the function.
  • This code does not simply execute one expression after the other in sequence like all of our interpreters so far; it relies on the ability to jump from one place of execution to another.
  • This module is about implementing languages with richer control flow structure than the ones we have seen so far. To do this, we will rely heavily on an idea called a continuation, which we will see today.

Tail calls and control context

  • Thus far, we have seen how we can write programs that manipulate names and mutable state by enriching our interpreter with heaps and environments.
  • Now, we will enrich our interpreters with a notion of control context in order to implement interesting control-flow structure
  • How do we do this? We begin with a notion of control context
  • A control context tracks which operations remain to be performed in order to finish a computation. If you’re familiar with the notion, a control context is analogous to a call stack.
  • As an example, consider the following definition of a factorial function:
let rec fact n =
  if n = 0 then 1 else n * (fact (n - 1))
  • Suppose we execute fact 4 using substitution:
fact 4
--> 4 * (fact 3)
    ^^^ control context
--> 4 * 3 * (fact 2)
    ^^^^^^^ control context
--> 4 * 3 * 2 * (fact 1)
    ^^^^^^^^^^^ control context
--> 4 * 3 * 2 * 1 * (fact 0)
    ^^^^^^^^^^^^^^^ control context
--> 4 * 3 * 2 * 1 * 1
--> 24
  • Each time fact n is called, that comes with a promise that the eventual value returned will be multiplied by n
    • This “promise to do something with the return value of a funtion call” is stored in the control context, which is a control-flow feature of a processor that keeps track of the context in which functions are called. The control context of each call to fact is annotated above.
  • Observe how above the control context grows with n
    • This can result in practical performance problems: if you’ve ever hit a “stack overflow” error in your code, it is typically because of accidental unbounded control context.
  • There is an alternative way to write fact that does not grow the control context by using an accumulator acc:
let rec fact_acc (n:int) (acc : int) =
  if n = 0 then acc else (fact_acc (n - 1) (n * acc))

let fact_iter n = fact_iter_acc n 1
  • Let’s examine the how the control context of the above function call grows:
fact_acc 4 1
--> fact_acc 3 2
--> fact_acc 2 12
--> fact_acc 1 24
--> fact_acc 0 24
--> 24
  • Observe how instead of storing all the necessary data to finish the factorial computation in the control context, it is instead stored in the accumulator
  • Definition: A function that requires no additional control context in order to execute is said to exhibit iterative control behavior.
    • A function that does require control context is said to exhibit recursive control behavior
  • Q: How can you tell if a function has iterative or recursive control?
    • A: If a function invokes itself in operand position (i.e., it is the argument to its own recursive call), then it requires recursive control
    • Observe how in fact, the recursive call is invoked as an argument to *
  • A tail call is a function call performed as the final act of a function
    • Observe that fact_acc above only ever calls itself as a tail call. We say fact_acc is in tail form.
  • Tail calls enable a host of powerful compiler optimizations and are often essential for writing high-performance functional programs. Consequently, OCaml supports a tailcall annotation that raises a warning if a function is not invoked as a tail call:
let rec fact_iter_acc n acc =
  if n = 0 then acc else ((fact_iter_acc [@tailcall]) (n - 1) (n * acc))

Continuations and continuation-passing form

  • In the fact example above, we are able to capture all the context necessary to finish the computation using an accumulator. This is not always possible; sometimes you need to keep track of more complicated behavior in the control context. This brings us to the notion of a continuation.
  • Instead of using an accumulator, let’s instead use a function to keep track of the remaining steps of computation. We will call this function a continuation (it “continues the computation”)
  • Let’s see it:
let rec fact_cont_h (n:int) (cont: int -> int) =
  if n = 0 then cont 1 else 
    fact_cont_h (n-1) (fun result -> cont (n * result))

let fact_cont n = fact_cont_h n (fun result -> result)
  • The best way to understand this code is to step through it and see how it avoids using control context (read this carefully! the scoping rules for the continuation parameter k are very important):
fact_cont_h 2 (fun k -> k)
--> fact_cont_h 1 (fun k -> (fun k -> k) (2 * k))
--> fact_cont_h 0 (fun k -> (fun k -> (fun k -> k) (2 * k)) 1 * k)
--> 2
  • Notice how the continuation grows with each recursive call instead of the control context: it is keeping track of all the remaining computation to do once the recursive call finally hits its base case.

A more interesting continuation

  • Let’s consider a more complicated function than fact and try to write it in tail-form:
(* Fibonacci sequence:
   a0 = 0
   a1 = 1
   an = a_{n-1} + a_{n-2}

Example:
  a0 = 0
  a1 = 1
  a2 = 1
  a3 = 2
  a4 = 3
  a5 = 5
  ... *)
let rec fib (a:int) =
  if a = 0 then 0
  else if a = 1 then 1
  else fib (a-1) + fib (a-2)
  • It seems difficult to write fib in tail-form because there are two separate calls to fib, neither of which are tail calls!
  • The answer is to use first-class functions: we add a parameter called a continuation of type int -> int that takes as input the result of evaluating a fib call and “continues” the computation. This is best illustrated by example:
(* continuation-passing-style fibonacci *)
let rec fib_iter_h (a:int) (cont : int -> int) =
  if a = 0 then cont 0
  else if a = 1 then cont 1
  else fib_iter_h (a-1)
      (fun res -> fib_iter_h (a-2)
          (fun x -> cont (x + res)))

let fib_iter a = fib_iter_h a (fun result -> result)

A continuation-passing interpreter

  • Explicit representation of the continuation is a powerful tool for implementing control-flow operators in interpreters
  • First, let’s see how to implement an interpreter in continuation-passing form: this way, we have the control context explicitly available. Then, we will make use of this explicit control context in order to do interesting control flow operations.
  • Recall the usual recursive calculator interpreter:
type calc =
  Num of int
  | Add of calc * calc

let rec interp_recursive c =
  match c with
  | Num(n) -> n
  | Add(c1, c2) -> (interp_recursive c1) + (interp_recursive c2)
  • Observe that this interpreter is not in tail-form
  • Let’s write this interpreter in tail-form by introducing a continuation:
let rec interp_iter_h c cont =
  match c with
  | Num(n) -> cont n
  | Add(c1, c2) ->
    interp_iter_h c1
      (fun n1 -> interp_iter_h c2 (fun n2 -> cont (n1 + n2)))

let interp_iter c =
  interp_iter_h c (fun n -> n)
  • Now that we have this function in continuation-passing form, we can use this continuation to implement some interesting control-flow behavior!
  • Let’s see how to implement a simple control-flow construct by using the fact that we have an explicit continuation in the context: returning a value before we are done computing. This looks just like how return works in Python.
(* a continuation-passing interpreter for a tiny calculator language *)
type return_calc =
    Num of int
  | Add of return_calc * return_calc
  | Return of return_calc

(* continuation-passing form *)
let rec interp_return_h (c:return_calc) (cont: int -> int) : int =
  match c with
  | Num(n) -> cont n
  | Add(c1, c2) ->
    interp_return_h c1
      (fun n1 -> interp_return_h c2 (fun n2 -> cont (n1 + n2)))
  | Return (calc) ->
    (* discard the current continuation and simply run `calc` *)
    interp_return_h calc (fun x -> x)

let interp_return c =
  interp_return_h c (fun n -> n)

let () =
  let prog : return_calc = (Add(Num 10, Return(Num 40))) in
  assert (interp_return (Add(Num 10, Num 10)) = 20);
  assert (interp_return prog = 40);
  • This is very interesting! Notice that, in order to implement Return, we discard the current continuation: this causes our interpreter to disregard all pending computation and simply “return”
  • This is hard to follow: let’s see an example trace:
interp_return_h (Add (Num  10, Return (Num 10))) (fun x -> x)
--> interp_return_h (Num 10) 
      (fun n1 -> interp_return_h (Return (Num 10)) 
         (fun n2 -> (fun x -> x) (n1 + n2)))
--> interp_return_h (Return (Num 10)) 
      (fun n2 -> 
        (fun x -> x) (n1 + n2))
--> interp_return_h (Num 10) (fun x -> x) (* see how the control context was overridden! *)
--> 10