Custom format in the Play Framework templating engine example

I have been looking into the Play Framework to do rapid development of web-services. The clean MVC separation and a simple templating system is very attractive to be able to focus on just what is needed, instead of spending a lot of time writing glue code and juggling concepts that should be in the framework.

The template engine in Play supports Html, Xml, Txt and Js. When writing web APIs, I usually want to have control over the content-type used, either for adding entirely new media-types, or for using with versioning of the API. I have been reading the documentation at http://playframework.com/documentation/2.2.x/ScalaCustomTemplateFormat that describes how to add a custom format to the template engine, and it was reasonably good. Just a few points requiring trial and error, because the exact files to put things into was not described, and the example used was how to add Html if Html was not supported, so it was not possible to just take the code and verify that it worked.

This post contains the actual files added and updated to a fresh Play 2.2.2 app to add CSV support. See the doc page referenced above for details and explanations.

Model classes

Although not strictly a part of the support for CSV, the model classes are used to demonstrate that things work, and to provide a record type for basing the concrete template example on:

app/models/Record.scala

package models

case class Record(val foo: String, val bar: String)

object Record {
  def sampleRecords = List(Record("f1","b1"), Record("f2","b2"), Record("f3","b3"))
}

Format definition

The meat of the new format is in the format implementation classes. I have put these in app/views/Csv.scala as that seemed appropriate.
The format contains the classes Csv and CsvFormat inheriting the proper classes as indicated by the official documentation.
The Csv object contains helpers for the CvsFormat object.

app/views/Csv.scala

package views

import play.api.http.ContentTypeOf
import play.api.mvc.Codec
import play.api.templates.BufferedContent
import play.templates.Format

class Csv(buffer: StringBuilder) extends BufferedContent[Csv](buffer) {
  val contentType = Csv.contentType
}

object Csv {
  val contentType = "text/csv"
  implicit def contentTypeCsv(implicit codec: Codec): ContentTypeOf[Csv] = ContentTypeOf[Csv](Some(Csv.contentType))

  def apply(text: String): Csv = new Csv(new StringBuilder(text))
  
  def empty: Csv = new Csv(new StringBuilder)
}

object CsvFormat extends Format[Csv] {
  def raw(text: String): Csv = Csv(text)
  def escape(text: String): Csv = {
    val sb = new StringBuilder(text.length)
    text.foreach {
      case '"' => sb.append("\"\"")
      case c => sb += c
    }
    new Csv(sb)
  }
}

Glue

To actually make the template compiler compile new templates, we need to define the file extension to trigger compilation. This is done by adding to the templateTypes at the end of the top-level build.sbt

build.sbt

name := "csv-play"

version := "1.0-SNAPSHOT"

libraryDependencies ++= Seq(
  jdbc,
  anorm,
  cache
)     

play.Project.playScalaSettings

templatesTypes += ("csv" -> "views.CsvFormat")

Putting it all together

The last part is to define an endpoint in the controller (in app/controllers/Application.scala), point a route at that endpoint (in conf/routes), and write a template to serve results (in app/views/records.scala.csv).

app/controllers/Application.scala

package controllers

import play.api._
import play.api.mvc._
import models.Record

object Application extends Controller {

  def index = Action {
    Ok(views.html.index("Your new application is ready."))
  }

  def records = Action {
    Ok(views.csv.records(Record.sampleRecords))
  }

}

conf/routes

# Routes
# This file defines all application routes (Higher priority routes first)
# ~~~~

# Home page
GET     /                           controllers.Application.index

GET     /records                    controllers.Application.records

# Map static resources from the /public folder to the /assets URL path
GET     /assets/*file               controllers.Assets.at(path="/public", file)

app/views/records.scala.csv

@(records: List[Record])"foo","bar"@for(r <- records) {
"@r.foo","@r.bar"}

The final result

Now everything is in place to test the new format. Run the app by doing play run in a separate terminal. Then you will observe the following result using curl as our HTTP client:

$ curl -i http://localhost:9000/records
HTTP/1.1 200 OK
Content-Type: text/csv
Content-Length: 41

"foo","bar"
"f1","b1"
"f2","b2"
"f3","b3"

Notice that the Content-Type is correctly set to the result rendered from the template.

Conclusion

This post shows a simple and straight-forward way to add a custom content-type to be used in the templating system. Using this approach, you get to write your format in the simple template language, and each endpoint in your controller does not do anything special to support the format.

There are other approaches that may be preferable in certain cases. If you have an XML-like format, you can use the Xml support in Play by having a app/views/foo.scala.xml in the template directory, but then set an explicitly different content-type in your controller, with something like:

...
  def records = Action {
    Ok(views.xml.records(Record.sampleRecords)).as("application/vnd.mycompany.api-v1+xml")
  }

This can of course also be used to abuse the Txt format in a similar way. The big downside to this approach is that you must remember to do that on every result produced by the application.

Good luck, and happy content typing!

Mysterious Problem with JUnit in IntelliJ IDEA – with Mysterious Fix!

Coming back to work after the holidays, I dutifully fired up my favourite IDE to get to work. Full of energy and determination, I added a lot of code, and felt quite good about myself. The only thing left was running the unit tests, to verify that I had not broken anything.
The tests completed fast; too fast. Looking closer, the only output was “Process finished with exit code 0”, and in fact no tests had been run. I tried it a few more times, but still nothing. Even after uninstalling IntelliJ, rebooting, and reinstalling a fresh copy of the IDE; still the same (non)result mocking me “Process finished with exit code 0”.

Completely stumped, I threw in my towel and searched the internet. Usually, I’m less than impressed with answers on stackoverflow.com, but in this case there was actually a hint at the solution!

arcane knowledgeApparently, there is something wrong with some cache somewhere. By going to the File menu and selecting “Invalidate Caches / Restart …”, testing should start working again.
And, as if by magic, it did! Right-clicking a package and selecting “Run Tests in …” started correctly, and lights started showing up beside each test-case. And they where all green. And it was good.

I still don’t understand what was wrong, and what I did to fix it, but now I at least have added another tool to my arsenal of arcane knowledge.

The referenced question at Stack Overflow can be found here:
http://stackoverflow.com/questions/13157815/intellij-idea-sudenly-wont-recognize-tests-in-test-folder

Java Constructor Anti-Pattern

I have never liked the typical java way of creating constructors for a typical “bean” class. The preferred way, by many, is to have the constructor arguments directly map to the fields in the class, and assign each in turn to the “this.” prefixed name.

I have always preferred to have the constructor arguments be short one-or-two-letter variables that you assign to the fields. This avoids the following problem I found today:

public class MetadataEntityDescriptor {
    private ContactPerson contactPerson;
    private IDPSSODescriptor idpSsoDescriptor;

    private MetadataEntityDescriptor(IDPSSODescriptor idpssoDescriptor,
                                     ContactPerson contactPerson) {
        this.idpSsoDescriptor = idpSsoDescriptor;
        this.contactPerson = contactPerson;
    }

    public static final class Builder {
        private IDPSSODescriptor idpssoDescriptor;
        private ContactPerson contactPerson;

        public Builder setIdpssoDescriptor(IDPSSODescriptor idpssoDescriptor) {
            this.idpssoDescriptor = idpssoDescriptor;
            return this;
        }

        public Builder setContactPerson(ContactPerson contactPerson) {
            this.contactPerson = contactPerson;
            return this;
        }

        public MetadataEntityDescriptor build() {
            return new MetadataEntityDescriptor(idpssoDescriptor, contactPerson);
        }
    }

    public Element asXml() {
        return new Element("EntityDescriptor", Constants.NAMESPACE_METADATA)
            .addContent(idpSsoDescriptor.asXml())
            .addContent(contactPerson.asXml());
    }
}

Why does the asXml() method here give a NullPointerException? The code looks good and even compiles. But notice the subtle capitalization bug in the first parameter of the constructor… In fact, the argument was never assigned to the field, and it thus stayed “null”. Fortunately my IDE caught it, but I did not see that until I had scratched my head a few times regarding the strange NPE.

Now, I will admit that an argument FOR having complete argument names is that it looks nicer in the IDE when you see the argument popup help. But to me that is not enough; I’d rather have shorter and less error prone code such as this:

public class MetadataEntityDescriptor {
    private ContactPerson contactPerson;
    private IDPSSODescriptor idpSsoDescriptor;

    private MetadataEntityDescriptor(IDPSSODescriptor isd,
                                     ContactPerson cp) {
        idpSsoDescriptor = isd;
        contactPerson = cp;
    }

    ...
}

And finally, this is of course a great argument for moving to Scala, where the equivalent, error proof code would be:

class MetadataEntityDescriptor(var contactPerson: ContactPerson,
                               var idpSsoDescriptor: IDPSSODescriptor) {
  def asXml = <EntityDescriptor>
      { idpSsoDescriptor.asXml }
      { contactPerson.asXml }
  </EntityDescriptor>
}

Akai LPD8 controller for Ableton Live 8

UPDATE: The following is a quote from the release notes for Ableton Live 8.2.2:

8.2.2 Release Notes. Improvements and feature changes: Added control surface support for Akai Professional LPD8.

So finally the contents of this article is not relevant anymore 🙂

The original article follows


This is how to set up Ableton Live to use the Akai LPD8 controller.

The Akai LPD8 is an ultra-portable pad controller. It has 8 sensitive trigger pads and 8 assignable knobs. It is plug’n’play for Windows and Mac, so in theory you can just connect it and start using it with your favourite music production software.

AKAI LPD8 MIDI pad controller

 

When you first connect the LPD8, it will show up in Live as a MIDI note input device enabled for triggering note events. So just load up a drum kit on a MIDI track, arm it, and start hitting those triggers.

There is no native support for the LPD8 knobs in Ableton Live, though. So a lot of the more powerful features of remote controllers, such as instant mapping and device locking, are not enabled by default.

To start using the LPD8 in “Manual Control Surface Setup” mode, you must go to the Preferences, MIDI Ports section, and enable the “Remote” button. Twiddling the knobs will now send MIDI data to Live, but nothing really happens just yet. The last step is to go to MIDI Learn mode, select the parameter you want to change, and move the corresponding knob to link the knob to the software control.

Now, the problem with the Manual Control and MIDI Learn mapping is that the link between the knob and the control is not context sensitive. This is the most interesting feature of natively supported controllers, because then you can use the same knobs to control all different instruments and sound parameters depending on what is currently selected.

So to get to this exalted level of controller support, we need to implement native support for the LPD8.

The official Ableton Live manual has nothing about how to do this, and searching around on the internet may lead you down a severely complicated path with multiple python scripts and internal APIs etc. But I found an easier way in an article called Ableton Live MIDI Remote Scripting How To: Custom Korg nanoSERIES Control. This tutorial was for setting up a similar (but inferiour…) controller, so I tweaked the scripts a little for the LPD8.

The key is to find the User Remote Scripts folder in your Preferences for Ableton Live. Here you will find actual instructions and a template for writing your own script. This script must be placed in a subfolder named after your controller.

So I created the folder Library/Preferences/Ableton/Live 8.1.5/User Remote Scripts/AKAI LPD8/. In this folder I put the following file, called UserConfiguration.txt

# Config File for User-defined Instant Mappings

# We assume that the controls on your MIDI controller
# send CCs (except for pads). All controls that do not have
# an explicit channel setting are expected to use the
# global channel. CCs & Notes are counted from 0-127
# and channels from 0-15.

[Globals]
# The channel that the controller should send on
GlobalChannel: 0
# If your controller is connected via USB, replace ControllerName
# with the name of the respective port. Live will then try to 
# recognize the ports for you when you select your Instant-Mappings
InputName: LPD8
OutputName: LPD8
# If your controller has pads that send notes, you can use them to
# play the visible pads in your DrumRacks. Just replace the -1 for
# the note (and channel) of the respective pad. The arrangement of
# the pads in the DrumRacks is as follows:
#   1     2     3     4
#   5     6     7     8
#   9    10    11    12  
#  13    14    15    16
# (If you leave the channel of a pad at -1, Live will assume that
#  the pad uses the global channel)
Pad1Note: -1
Pad2Note: -1
Pad3Note: -1
Pad4Note: -1
Pad5Note: -1
Pad6Note: -1
Pad7Note: -1
Pad8Note: -1
Pad9Note: 40
Pad10Note: 41
Pad11Note: 42
Pad12Note: 43
Pad13Note: 36
Pad14Note: 37
Pad15Note: 38
Pad16Note: 39
Pad1Channel: -1
Pad2Channel: -1
Pad3Channel: -1
Pad4Channel: -1
Pad5Channel: -1
Pad6Channel: -1
Pad7Channel: -1
Pad8Channel: -1
Pad9Channel: -1
Pad10Channel: -1
Pad11Channel: -1
Pad12Channel: -1
Pad13Channel: -1
Pad14Channel: -1
Pad15Channel: -1
Pad16Channel: -1

[DeviceControls]
# The Encoders will control the device parameters (you can also
# use knobs or sliders). Replace the -1's with the CCs sent by 
# the respective controls on your controller. You can also set
# the channel for each controller if it differs from the global
# channel (if you leave the channel of an encoder at -1, Live
# will assume that the encoder uses the global channel).
Encoder1: 1
Encoder2: 2
Encoder3: 3
Encoder4: 4
Encoder5: 5
Encoder6: 6
Encoder7: 7
Encoder8: 8
EncoderChannel1: -1
EncoderChannel2: -1
EncoderChannel3: -1
EncoderChannel4: -1
EncoderChannel5: -1
EncoderChannel6: -1
EncoderChannel7: -1
EncoderChannel8: -1
# Enter the respective map mode for the encoders here. The following
# map modes are available:
# - Absolute
# - Absolute14Bit
# - LinearSignedBit 
# - LinearSignedBit2
# - LinearTwoCompliment
# - LinearBinaryOffset
# - AccelSignedBit 
# - AccelSignedBit2
# - AccelTwoCompliment
# - AccelBinaryOffset
# Consult the controller's documentation to find out which mode to use.
EncoderMapMode: Absolute
# Buttons used here are expected to not be toggles (i.e., sending 
# value 0 every second time you press it).
Bank1Button: 9
Bank2Button: 10
Bank3Button: 11
Bank4Button: 12
Bank5Button: 13
Bank6Button: 14
Bank7Button: 15
Bank8Button: 16
NextBankButton: -1
PrevBankButton: -1
LockButton: -1

[MixerControls]
# Again enter the appropriate CCs for the respective controls.
# If all sliders use the global channel to send their data,
# you can leave the channels at -1. You can, of course, use
# encoders or knobs instead of sliders.
VolumeSlider1: -1
VolumeSlider2: -1
VolumeSlider3: -1
VolumeSlider4: -1
VolumeSlider5: -1
VolumeSlider6: -1
VolumeSlider7: -1
VolumeSlider8: -1
Slider1Channel: -1
Slider2Channel: -1
Slider3Channel: -1
Slider4Channel: -1
Slider5Channel: -1
Slider6Channel: -1
Slider7Channel: -1
Slider8Channel: -1
MasterVolumeSlider: -1
MasterSliderChannel: -1
Send1Knob1: -1
Send1Knob2: -1
Send1Knob3: -1
Send1Knob4: -1
Send1Knob5: -1
Send1Knob6: -1
Send1Knob7: -1
Send1Knob8: -1
Send2Knob1: -1
Send2Knob2: -1
Send2Knob3: -1
Send2Knob4: -1
Send2Knob5: -1
Send2Knob6: -1
Send2Knob7: -1
Send2Knob8: -1
TrackArmButton1: -1
TrackArmButton2: -1
TrackArmButton3: -1
TrackArmButton4: -1
TrackArmButton5: -1
TrackArmButton6: -1
TrackArmButton7: -1
TrackArmButton8: -1
VolumeMapMode: Absolute
SendsMapMode: Absolute

[TransportControls]
# The transport buttons are also expected not to be toggles.
StopButton: -1
PlayButton: -1
RecButton: -1
LoopButton: -1
RwdButton: -1
FfwdButton: -1

With this file in place, the next time you start Ableton Live, you will find AKAI LPD8 as a possible selection in the Preferences MIDI Control Surface drop-down box. Select this, and go back to your musical masterpiece.
If you now try to click on an instrument rack control, you will get the blue hand icon, and your LPD8 will control the parameters. A typical instrument rack has 8 virtual knob controls, neatly mapping into the 8 physical knobs on the LPD8.

For more complex controls, such as the Operator synth, the knobs will control individual detailed parameters. When there are more than 8 parameters, they will be assigned different “banks”. To switch banks using the LPD8, push the “CC” button, and use the trigger pads to select bank 1 (pad 1) through bank 8 (pad 8). The status bar at the bottom of Live’s interface will show you the name of the bank, such as “Operator Bank : Oscillator B” for pad 2, or “Operator Bank : LFO” for pad 5, etc.

Now, you can finally enjoy detailed control of all Live parameters using the Akai LPD8 controller.

PS: Akai also makes the fabulous Akai LPK25 ultra-portable MIDI keyboard, in the same style as the LPD8. They go very well together, and I recommend getting both to have a complete on-the-road MIDI setup for your laptop.

What’s on my iPhone – Part I: Most useful apps

In this series of post I will show what apps I currently have installed on my iPhone. I have tried a lot of apps, and have ended up with a set that is quite stable. My overall organization is divided into screens as follows:

  1. Most Used Apps – 2 screens
  2. Chinese and Oriental Languages Apps – 2 screens
  3. Games
  4. Artistic/Creative Apps
  5. Utilities
  6. Try-out apps
  7. Digital Clocks…

In this post, I will discuss my most frequently used apps.

Dock

iPhone Dock

iPhone Dock

In my dock, I have Phone, Mail, and iPod. The stock apps are, of course, some of my most used apps, and the backbone of the whole device. I have chosen to kick Safari from the standard dock, and replace it with Things. I have tried several todo/GTD apps (such as OmniFocus), and have found that Things is the easiest and quickest to use, while staying powerful enough for my needs. It syncs on the local wireless with a desktop client on my laptop.

 

Screen 1: Most Used Apps

Most Used Apps

Most Used Apps

This is the home screen, and it contain my most frequently used apps. The top two rows are all apps that shipped with the iPhone. SMS and Calendar I use a lot, and the Camera app is very handy to have in a top corner on the home screen for quick access (you do know that pushing the Home butten takes you back to the first screen, right?). Photos belongs with the Camera, and then the suite of Safari, Maps, Clock and Contacts nicely matches my familiar environment from the laptop mac.

The standard Calculator is very good (especially in landscape mode…), and I have coupled it with the best units converter, Units. Yr.no is a quick, no-frills UI for the Norwegian weather service.

The final row is communication apps. BeejiveIM is “expensive”, but the best IM client, in that it looks good, supports all my services, and keeps the connection while the app is not running, handing you chat messages in email instead (works best with push IMAP, such as my MobileMe email…). TwitterFon I have found is the best, quickest and slickest twitter app, and Brightkite is actually the most convenient for updating my location, i.e. I don’t use the social aspect much, but maybe some day? Colloquy is an IRC client, that perhaps most people these days will find completely foreign and useless…

Screen 2: Frequent Apps “spill over”

Frequent Apps - part deux

Frequent Apps - part deux

The second screen is for apps that are in frequent used, but have “spilled over” from the first screen.

I often find myself browsing the App Store and iTunes on the phone when I am on a wireless connection. So these are nice to have there.

Shopper is a shopping list app, for writing down, and carrying out (mainly grocery) shopping. I have tried a few others, but this is good looking, flexible, and does the job.

Spend Lite is the free/lite edition of Spend, but has enough features for me to track my bus-card usage. This is designed to track expenses in different categories, with either a periodic refill, or manual deposit. The lite only supports daily or weekly cycles; I have no cycles, so…

Line two has my connectivity part 2 apps; Facebook, LinkedIn, MySpace Mobile and WordPress. These are all stripped down compared to the websites, but works well for quick checking (haven’t written a blog post on the phone, yet, but…) And, they are free 🙂

The next apps are Keynote Remote and iTunes Remote. Using your phone to control a presentation is always cool, and since I have a set of Airport Express basestations at home with AirShare music streaming, the iTunes Remote gives me control of music around my house.

Then, my most expensive app, the Concise Oxford English Dictionary. It is comprehensive, and works off-line. Not as well integrated in the operating system as the dictionary on my mac, but…

Next, is the best news/RSS reader, Byline, offering tight integration with Google Reader, and a workable (but slow) off-line mode.

The page is rounded of with MobileFiles, which really is for accessing my MobileMe files, but I use it for storing/syncing files to my phone; WriteRoom, a better Notes with usable font, landscape mode and LAN sharing; and Classics, which is a Delicious Library like ebook reader with high quality visuals, and convenient handling for short stints of reading. It has a select set of public domain books. So in my spare time, I am re-reading “Alice in Wonderland”, “Dracula”, “the Metamorphosis”, “Call of the Wild” etc…

That concludes my list of most used apps. Stay tuned for more specialized apps…