This post is a kind of sequel to my previous article on type classes and generic derivation, although unfortunately there's a lot of intermediate content that should go between there and here that I haven't written yet. This post introduces a new feature in circe that I'm pretty excited about, though, so I'm not going to worry about skipping over that stuff for now.

It's not unusual in Scala to want to take a collection with items of some algebraic data type and partition its elements by their constructors. In this Stack Overflow question, for example, we're given a type for fruits:

sealed trait Fruit

case class Apple(id: Int, sweetness: Int) extends Fruit
case class Pear(id: Int, color: String) extends Fruit

The goal is to be able to take a collection of fruits and split it into two collections—one of apples and one of pairs.

def partitionFruits(fruits: List[Fruit]): (List[Apple], List[Pear]) = ???

It's pretty easy to use collect to solve this problem for particular cases. It's a little trickier when we start thinking about what a more generic version of such a method would look like—we want to take a collection of items of some arbitrary algebraic data type and return a n-tuple whose elements are collections of items of each of that ADT's constructors (and let's require them to be typed as specifically as possible, since this is Scala). It's not too hard to imagine how you could write a macro that would perform this operation, but it'd be messy and would probably end up feeling kind of ad-hoc (at least without a lot of additional work and thinking).

Fortunately we've got Shapeless 2.0, where Miles Sabin and co. have written the macros for us so we can keep our hands clean.

This post is another entry in my series on stuff you should never do with macros in Scala, but that you could do with macros in Scala, if you really wanted to, and if you'd picked up a bottle of Macallan on the way home from work and were willing to waste half an evening doing something ridiculously useless.

It's specifically a response to this Stack Overflow question, which asks if it's possible to specify explicitly that you want to use the default value of a constructor parameter in Scala.

So suppose we have a class like this:

class Foo(val x: String, val y: Int = 13, val z: Symbol = 'zzz)

The goal is to allow the following syntax:

val useDefaultZ = true

new Foo(x = "whatever", y = 1, z = if (useDefaultZ) default else 'whatever)

This is possible with macros, and it's not nearly as easy as you might think.

Scala provides lexicographic Ordering instances for TupleN types when all of the element types have Ordering instances. It doesn't provide the same instances for case classes, however—probably just because lexicographic order isn't what you want for case classes as often as it is for tuples.

Sometimes you actually do want a lexicographic ordering for your case classes, though, and Scala unfortunately doesn't provide any nice boilerplate-free way to create such orderings. This post will provide a quick sketch of two approaches to filling this gap: one using macros, and one using Shapeless 2.0's new Generic machinery and the TypeClass type class.

First for a case class to use as a running example, along with some instances:

case class Foo(x: Int, y: String)

val a = Foo(9, "x")
val b = Foo(1, "z")
val c = Foo(9, "w")

val foos = List(a, b, c)

Let's quickly confirm that there's no Ordering[Foo] already sitting around:

scala> foos.sorted
<console>:14: error: No implicit Ordering defined for Foo.
              foos.sorted
                   ^

Yep, we're going to have to take care of this ourselves.

This Stack Overflow question is interesting—it asks whether we can use Scala macros to create a value class for positive integers where the positiveness is checked at compile-time, and where it's not possible to create an invalid instance.

I'm pretty sure it's not. My first thought was to turn PosInt into a sealed universal trait with a private value class implementation in the PosInt companion object, but inheriting from a universal trait forces us to give up most (all?) of the advantages of value classes in this case, and of course it's not actually possible to make the value class private, anyway.

So I don't have an answer, but I do have a pretty neat trick involving vampire methods that gives us some of the benefits of value classes.

Quasiquotation is an old idea (Miles Sabin notes the term in a 1937 Quine paper, for example) that's now available in Scala (thanks to the efforts of Eugene Burmako and Den Shabalin), where it allows us to avoid the nightmarishly complicated and verbose code that's required to construct abstract syntax trees manually in our macros.

Quasiquotes are a little like reification, but much more flexible about what kinds of things can be "spliced" into the tree, and where they can be spliced. For example, we couldn't use reify in the following code, because there's no way to splice in the name of the type member:

def foo(name: String): Any = macro foo_impl
def foo_impl(c: Context)(name: c.Expr[String]) = {
  import c.universe._

  val memberName = name.tree match {
    case Literal(Constant(lit: String)) => newTypeName(lit)
    case _ => c.abort(c.enclosingPosition, "I need a literal!")
  }

  val anon = newTypeName(c.fresh)

  c.Expr(Block(
    ClassDef(
      Modifiers(Flag.FINAL), anon, Nil, Template(
        Nil, emptyValDef, List(
          constructor(c.universe),
          TypeDef(Modifiers(), memberName, Nil, TypeTree(typeOf[Int]))
        )
      )
    ),
    Apply(Select(New(Ident(anon)), nme.CONSTRUCTOR), Nil)
  ))
}

This is an unreadable mess, and it's not even a complete example—it depends on some additional utility code.

My attempt to sneak the terms vampire and zombie into the Scala vernacular seems to be succeeding, so here's a new one:

Potemkin definitions: definitions in a macro-constructed structural type that are intended only to make an expression passed as an argument to another macro method typecheck before that macro rewrites it.

I came up with the trick to support this horrible abuse of Scala syntax:

case class Car(var speed: Int, var color: String) { ... }
object Car { ... }

import Car.syntax._

val car = new Car(0, "blue")

car set {
  color = "red"
  speed = 10000
}

Here color and speed are definitions in a structural type that have the same signatures as the fields on the case class, but they don't actually do anything—if we call them we get a NotImplementedError. They only exist to allow the block expression we're passing to set to typecheck before the macro implementation replaces them with something useful.

I've written several times about vampire methods, which are macro methods inside a macro-defined type, where the macro method's implementation is provided in an annotation. Normally when we define a type in a def macro, it looks like a structural type to the outside world, and calling methods on a structural type involves reflective access in Scala. Vampire methods allow us to avoid that ugly bit of runtime reflection.

This trick (which was first discovered by Eugene Burmako) is useful because it makes it a little more practical to use def macros to approximate type providers, for example. It's also just really clever.

For methods with no parameters, the execution of the trick is pretty straightforward. It's a little more complicated when we do have parameters, as Eric Torreborre observes in a question here, since in that case the annotation will need to contain a function instead of just a simple constant of some kind.

When macros first showed up in Scala as an experimental language feature last year, many Scala developers responded with skepticism or distaste. They argued that macros were a distraction from work on more urgent problems with the language, that they would lead to even more complex and reader-unfriendly code, etc.

After a year and a half I think these arguments have less weight, as macros have proven extremely useful in a wide range of applications: string interpolation, serialization, type-level programming with singletons, numeric literals and faster loops, typed channels for actors, and so on. They've let me make many parts of my own code faster and safer in surprising ways.

This post is not about a useful application of macros. It's inspired by a couple of questions on Stack Overflow today, and is an example of exactly the kind of thing macros should not ever be used for. But it's Friday evening and I'm drinking beer in the office and I think this trick is pretty clever, so here we go.

I wish I could take credit for what I'm about to show you, because it's easily the cleverest thing I've seen all week, but it's Eugene Burmako's trick and I've only simplified his demonstration a bit and adapted it to work in Scala 2.10.

I like writing code. I also like not writing code, especially when I'm writing code. Type providers are a particularly nice way not to write code. They let you take some kind of schema (for a relational database, RDF vocabulary, etc.) and turn it directly into binding classes at compile time—with no worrying about managing generated code, etc.

I've wanted type providers in Scala for a long time (heck, I wanted type providers ten years ago when I was a Java programmer who had no idea what a type provider was but was dissatisfied with code generation for this kind of task). The type macros currently available in Macro Paradise will provide the real thing, but they're still (at least) months and months away from a stable Scala release.

In the meantime, you can get surprisingly good fake type providers with the def macros in Scala 2.10. In a previous post I outlined one set of macro-based approaches to the problem, with the most concise version involving structural types and therefore (unfortunately) reflective access. In this post I'll go into a bit more detail about the code involved, and will look at just how bad reflective access actually is performance-wise by comparing the structural-type approach to two alternatives: plain old code generation and macro-supported compile-time dynamic types.

It's sometimes useful in Scala to have a type with a single value. These are called singleton types, and they show up most easily in the context of Scala's objects. For example, if we have the following definition:

object foo {
  def whatever = 13
}

We can refer to a type foo.type that is the singleton type for foo—i.e., the type that contains nothing except foo. We can use this type to write a function that won't compile with any non-foo argument:

def fooIdentity(x: foo.type) = x

For example:

scala> fooIdentity(foo)
res1: foo.type = foo$@5da19724

scala> fooIdentity("foo")
<console>:14: error: type mismatch;
 found   : String("foo")
 required: foo.type
              fooIdentity("foo")
                          ^

Note that this error message doesn't just tell us that we provided a String when we needed a foo—it lists the type as String("foo"). This is because string literals—like all other literals in Scala (except function literals)—are also singletons in the sense that their most specific type is a singleton type.

Suppose we want to define an HListable trait in Scala that will add a members method returning an HList of member values to any case class that extends it. This would let us write the following, for example:

scala> case class User(first: String, last: String, age: Int) extends HListable
defined class User

scala> val foo = User("Foo", "McBar", 25)
foo: User = User(Foo,McBar,25)

scala> foo.members == "Foo" :: "McBar" :: 25 :: HNil
res0: Boolean = true

So we try the following, which looks reasonable at a glance:

We've recently started using the W3C's banana-rdf library at MITH, and it's allowing us to make a lot of our code for working with RDF graphs both simpler and less tightly coupled to a specific RDF store. It's a young library, but also very clever and well-designed, and it does an excellent job of exploiting advanced features of the Scala language to make its users' lives easier. Alexandre Bertails and his collaborators deserve a lot of credit for what they've accomplished in just a little over a year.

One of the least pleasant aspects of working with any RDF library is writing bindings for particular vocabularies. For example, if we wanted to use the Open Archives Initiative's Object Reuse and Exchange vocabulary in our banana-rdf application, we'd need to write something like the following: