Dependently-typed languages allow precise types to be used during development, facilitating reasoning about programs. However, stronger types bring a disadvantage that it becomes increasingly difficult to write programs that are accepted by a type checker and additional proofs may have to be specified by a programmer.

Embedded domain-specific languages (EDSLs) can help address this problem by introducing a layer of abstraction over more precise underlying types, allowing domain-specific code to be written in a high-level language which uses dependent types to enforce invariants without imposing additional proof obligations on an application programmer.

In this paper, we apply this technique to web programming. Using the dependently typed programming language Idris, we introduce an EDSL to facilitate the creation and handling of statically checked web forms, reducing the scope for programmer error and attacks such as SQL injection. We also show how to enforce resource usage protocols associated with common web operations such as CGI, database access and session handling.

You can find the accepted draft here. A revised version will appear later.

]]>

Assuming you have an up-to-date Idris installed, install the vim Idris mode. This mode allows vim to communicate with a running Idris REPL instance. It provides the following basic commands:

`\t`

displays the type of the name under the cursor (or a bit more information about its context if it happens to be a metavariable)`\e`

prompts for an expression to evaluate`\r`

reloads and typechecks the file in the current buffer

More interestingly, it provides support for interactive editing:

`\c`

, if the cursor is over a variable in a pattern match clause, splits that variable into all the patterns which are well typed. This also works in`case`

expressions.`\w`

, if the cursor is over a pattern clause of the form`f x y z = ?rhs`

creates a new`with`

clause, i.e. the clause becomes:f x y z with (_) f x y z | with_pat = ?rhs

`\d`

, if the cursor is in a top level function type declaration (right of the`:`

), creates a new clause for that function.

(With all of these, the `\`

(backslash) is your local leader key. I have this rebound to `,`

(comma) but `\`

is the default.)

For example, start up `idris vadd.idr`

(this will create the file if it doesn’t already exist) edit it with `:e`

and type the following:

module vadd vadd : Num a => Vect n a -> Vect n a -> Vect n a

Press ESC, and before moving the cursor anywhere, hit `\d`

. You should see:

module vadd vadd : Num a => Vect n a -> Vect n a -> Vect n a vadd x1 x2 = ?vadd_rhs

Now move the cursor over `x1`

and hit `\c`

to split `x1`

into patterns for `[]`

and `::`

. You should see:

module vadd vadd : Num a => Vect n a -> Vect n a -> Vect n a vadd [] x2 = ?vadd_rhs_1 vadd (_ :: _) x2 = ?vadd_rhs_2

Note that it hasn’t attempted to invent any names for the patterns. This is because I have decided that making no choice is better than making a bad choice! We can type some in ourselves…

module vadd vadd : Num a => Vect n a -> Vect n a -> Vect n a vadd [] x2 = ?vadd_rhs_1 vadd (x :: xs) x2 = ?vadd_rhs_2

Now do the same over the first `x2`

. Move the cursor over it and hit `\c`

. You should see:

module vadd vadd : Num a => Vect n a -> Vect n a -> Vect n a vadd [] [] = ?vadd_rhs_3 vadd (x :: xs) x2 = ?vadd_rhs_2

Note that this has only produced one new pattern, because `vadd [] (_ :: _)`

would not be well typed. Finally, over the remaining `x2`

:

module vadd vadd : Num a => Vect n a -> Vect n a -> Vect n a vadd [] [] = ?vadd_rhs_3 vadd (x :: xs) (_ :: _) = ?vadd_rhs_1

If you move the cursor over the metavariable `?vadd_rhs_1`

and hit `\t`

, you will see the context and the required type of `?vadd_rhs_1`

:

a : Type c : Num a n : Nat x : a xs : Vect n a _ : a _ : Vect n a -------------------------------------- vadd_rhs_1 : Vect (S n) a

The interactive mode will work with any file in the working directory of your REPL instance, so you can edit multiple files at once.

`with`

ruleLet’s try completing the following definition, in a module `elem.idr`

:

module elem import Decidable.Equality using (xs : List a) data Elem : a -> List a -> Type where Here : Elem x (x :: xs) There : Elem x xs -> Elem x (y :: xs) isElem : (x : Nat) -> (xs : List Nat) -> Maybe (Elem x xs)

As before, get a clause for `isElem`

by hitting `\d`

somewhere after the `:`

on the line with `isElem`

. You’ll see

isElem : (x : Nat) -> (xs : List Nat) -> Maybe (Elem x xs) isElem x xs = ?isElem_rhs

Now expand `xs`

using `\c`

and give the pattern variables names:

isElem : (x : Nat) -> (xs : List Nat) -> Maybe (Elem x xs) isElem x [] = ?isElem_rhs_1 isElem x (y :: ys) = ?isElem_rhs_2

We’ll leave the `[]`

case for now and concentrate on the `y :: ys`

case. We’ll do this by attempting to build a proof that `x = y`

using

decEq : DecEq t => (x1 : t) -> (x2 : t) -> Dec (x1 = x2)

(This is why we imported `Decidable.Equality`

.) Move to the line with the `y :: ys`

case and hit `\w`

. This gives:

isElem : (x : Nat) -> (xs : List Nat) -> Maybe (Elem x xs) isElem x [] = ?isElem_rhs_1 isElem x (y :: ys) with (_) isElem x (y :: ys) | with_pat = ?isElem_rhs

The cursor is conveniently in the `with`

argument. Change that to `decEq x y`

, move the cursor over `with_pat`

and hit `\c`

again to split `with_pat`

:

isElem : (x : Nat) -> (xs : List Nat) -> Maybe (Elem x xs) isElem x [] = ?isElem_rhs_1 isElem x (y :: ys) with (decEq x y) isElem x (y :: ys) | (Yes _) = ?isElem_rhs_2 isElem x (y :: ys) | (No _) = ?isElem_rhs_3

The `Yes`

case’s parameter is a proof that `x = y`

. We’ll need to match on this to get a canonical proof. Change it to `Yes p`

, and case split `p`

. You’ll get:

isElem : (x : Nat) -> (xs : List Nat) -> Maybe (Elem x xs) isElem x [] = ?isElem_rhs_1 isElem x (y :: ys) with (decEq x y) isElem x (x :: ys) | (Yes refl) = ?isElem_rhs_4 isElem x (y :: ys) | (No _) = ?isElem_rhs_3

Now that you have the canonical proof, the left hand side has been updated so that `x`

and `y`

are unified to `x`

. I’ll leave the rest of this definition as an exercise — remember to use `\t`

to check the types of `isElem_rhs_3`

and `isElem_rhs_4`

.

When an Idris REPL starts, it also starts up a server running on port 4294 your local machine, responding to requests from a REPL client. This can be queried using the `--client`

flag to Idris. For example, from the shell:

$ idris --client '2+2' 4 : Integer $ idris --client 'vadd [1,2,3] [4,5,6]' [5,7,9] : Vect 3 Integer $ idris --client ':t vadd' vadd.vadd : Num a => (Vect n a) -> (Vect n a) -> Vect n a

Therefore, if the REPL provides appropriate commands for editing a file in place, all an editor has to do is invoke it and reload the file. The REPL now provides:

`:casesplit [line] [name]`

, abbreviated`:cs`

which displays the cases resulting when the variable`[name]`

on line`[line]`

is split, e.g.$ idris --client ':cs 4 x1' vadd [] x2 = ?vadd_rhs_1 vadd (_ :: _) x2 = ?vadd_rhs_2

There is also a destructive version

`:cs!`

which is the same, but updates the source file.`:addclause [line] [name]`

, abbreviated`:ac`

which displays a clause suitable for the function declaration for`[name]`

on line`[line]`

. Similarly,`:ac!`

updates the source file.`:makewith [line] [name]`

, abbreviated`:mw`

which adds a`with`

clause for the definition of`[name]`

on line`[line]`

. Again,`:mw!`

updates the source file.

The vim script is simply a thin wrapper for these functions. It also uses the REPL to invoke the evaluator, and to call `:r`

for quickly type checking a buffer.

Enjoy! If you prefer to use another editor, please consider contributing a similar script to support interactive development in your own editor. More features are planned, in particular automatically solving metavariables where possible.

]]>

Sequential decision problems, dependently typed solutions

Nicola Botta, Cezar Ionescu, Edwin Brady

We propose a dependently typed formalization for a simple class of sequential decision problems. For this class of problems, we implement a generic version of Bellman’s backwards induction algorithm and a machine checkable proof that the proposed implementation is correct. The formalization is generic. It is presented in Idris, but it can be easily translated to other dependently-typed programming languages. We conclude with an informal discussion of the problems we have faced in extending the formalization to generic monadic sequential decision problems.

You can find the full paper here.

]]>

Programming and Reasoning with Algebraic Effects and Dependent Types,

Edwin Brady

One often cited benefit of pure functional programming is that pure code is easier to test and reason about, both formally and informally. However, in order to be useful, programs must interact with the outside world. Haskell solves this problem using monads to capture details of possibly side effecting computations — it provides monads for capturing State, I/O, exceptions, non-determinism, libraries for practical purposes such as CGI and parsing, and many others, as well as monad transformers for combining multiple effects.

Unfortunately, useful as monads are, they do not compose very well. Monad transformers can quickly become unwieldy when there are lots of effects to manage, leading to a temptation in larger programs to combine everything into one coarse-grained state and exception monad. In this paper I describe an alternative approach based on handling algebraic effects, implemented in the Idris programming language. I show how to describe side effecting computations, how to write programs which compose multiple fine-grained effects, and how, using dependent types, we can use this approach to reason about states in effectful programs.

You can find the draft here. As usual, comments are very welcome.

]]>

- Lecture 1, Introduction, slides, video
- Lecture 2, Embedded DSLs, slides, video
- Lecture 3, Effect management, slides, video
- Lecture 4, Implementing Idris, slides, video
- Example code
- Exercises
- Idris web page

]]>

I’m going to use “total” in a fairly informal sense here. By “total function” I mean either:

- A function which terminates for possible inputs, or
- A function which is guaranteed to produce some output before making a recursive call

In particular, this means you can still write an interpreter or operating system, for example. That is, you can write programs which run forever, but at least they achieve something while doing so.

It is a good idea to aim to make all of your functions total, as David Turner argues – when functions are total, you really can believe that “well-typed programs don’t go wrong” because you know you’re going to get an answer eventually (or in the case of coinductive functions, you know you’re going to keep making progress). And, indeed, dependent types can help you achieve that goal because you can use the extra structure the types give you as additional evidence for the termination checker. Here is a nice example.

Then, of course, some programs are really proofs, that you never actually run but which do demonstrate some properties of other functions. In this case, it’s vitally important that they are total, to guarantee that you haven’t missed any cases, or relied on circular reasoning, and so every possible input leads to a proof object.

Having said all that, Idris still supports partial functions, for two main reasons.

Firstly, you can divide programs in general into two categories:

- Programs you haven’t finished writing yet
- Programs which are complete

The first category is significantly larger than the second. Even more so if you replace “Programs” with “Applications”.

In the case that your program isn’t finished yet, you don’t want your type checker shouting at you that it’s not finished, because you already know, but you still might want to test it (sometimes we like to run our programs, as well as read and write them, after all). You might want to try running a program even with an incomplete proof, to help you develop an intuition for how the proof might work, or even to see if there are test cases which might invalidate your hypothesis. Alternatively, you might have a nice natural recursive algorithm which is not obviously structurally recursive (the functional version of quicksort is an obvious example of this) and not want to hold up the development of the rest of your application while you develop the termination proof.

The fact that programs spend most of their lives in an unfinished state is one reason people are interested in gradual types, and why a recent GHC extension allows you to run programs with type errors. You don’t want to release software with type errors (or, in the case of an Idris program with an associated correctness proof, release software with an incomplete proof) but it might be convenient during development.

Secondly, I don’t believe it’s a language’s job to tell a program what level of precision their programs’ types should have. Rather, a language and its features are there to help a programmer do their job. If I want the language to help me write a totally correct program with a precise specification, then I certainly want a totality checker, but if I’m just writing a quick script to do a simple task, maybe I’d rather just get the job done. I would like the library code the script invokes to be total, but I might not be so bothered about the script itself, at first.

Related to this, remember that there’s more you can do with dependent types than prove functional correctness. Just being able to give types to more programs is already a benefit which doesn’t require programs to be total. Dependent types can help you write generic libraries – the Eff library is a developing example of this, as are embedded domain specific languages in general – where the library user is making use of dependent types without necessarily being aware of it.

Idris aims to be a general purpose programming language, rather than a theorem prover, so aims to support all uses of dependent types, or even to support programming with conventional types if that’s all you need, so the default is to support partial functions. If the goal was primarily to be a theorem prover, I think the opposite choice would have been appropriate. Whatever happens, though, functions are always checked for totality, and you can make partial functions a compile-time error by default with the “–total” flag.

If you want complete safety, you certainly have to work for it, and depending on your application that may be important. Sometimes, though, you just want to get things done. In that case, Idris will let you – but at least the type system and totality checker make it clear what assumptions you have made to do so.

]]>

Many components of a dependently typed programming language are by now well understood, for example the underlying type theory, type checking, unification and evaluation. How to combine these components into a realistic and usable high level language is, however, folklore, discovered anew by successive language implementations. In this paper, I describe the implementation of a new dependently typed functional programming language, IDRIS. IDRIS is intended to be a general purpose programming language and as such provides high level concepts such as implicit syntax, type classes and do notation. I describe the high level language and the underlying type theory, and present a method for elaborating concrete high level syntax with implicit arguments and type classes into a fully explicit type theory. Furthermore, I show how this method, based on a domain specific language embedded in Haskell, facilitates the implementation of new high level language constructs.

]]>

Of course, the slides won’t make as much sense as with the accompanying commentary, but at least you’ll see what I talked about, and maybe you’ll get something from them anyway. In them, you’ll find:

- An introduction to Idris
- Some examples: verified binary arithmetic, lightweight verification of a virtual machine, and

my plan to never have to write a monad transformer again. - Some notes on compilation issues

]]>

- Various operations are only valid if the stack is in the right form (e.g. arithmetic requires at least two items at the top of the stack, and leaves at least one item). We can often statically check that the stack is in the right form before executing such instructions, and at least statically check that the necessary dynamic checks are carried out.
- We need to consider how to represent the stack – a
`Stack n`

is the type of stacks with*at least*elements.`n`

- There’s a lot of bounds checking of stack references, and as much of this as possible is made static.
- A jump is only valid if the given label is defined. We can statically check this, too, and guarantee that a program we execute has well-defined labels.
- Although whitespace is Turing complete, we can still use the totality checker to verify that executing individual instructions terminates.

If nothing else, we get to see how to use dependent types to make *assumptions* and *invariants* explicit in the code. Even if those invariants aren’t used to guarantee total correctness, we at least get the type checker to tell us where we’ve violated them. In fact, I avoided a few errors this way and found a couple of missing or badly specified parts of the whitespace tutorial… Perhaps I’ll write more on this at some point…

]]>

Implementation of a General Purpose Programming Language with Dependent Types, abstract as follows:

Many components of a dependently typed programming language are by now well understood, for example the underlying type theory, type checking, unification and evaluation. How to combine these components into a realistic and usable high level language is, however, folklore, discovered anew by succesive language implementations. In this paper, I describe the implementation of a new dependently typed functional programming language, Idris. Idris is intended to be a general purpose programming language and as such provides high level concepts such as implicit syntax, type classes and do notation. I describe the high level language and the underlying type theory, and present a method for elaborating concrete high level syntax with implicit arguments and type classes into a fully explicit type theory. Furthermore, I show how this method, based on a domain specific language embedded in Haskell, facilitates the implementation of new high level language constructs.

There’s still some polishing to do, but comments on the content, form, and so on, would be most welcome. Enjoy!

]]>