Image credit: Toby Hudson

Day 29-30: On the internet, no-one knows your plumber is a magpie

(Note - It’s been a busy weekend, and this post is a little late: I did “all” the coding on the 26th though, so I’m totally going to pretend I met my self-imposed deadline 😀)

Often when I am bereft of ideas, I go browsing through the awesome R list in search of inspiration, marvelling at all the shiny, shiny packages 📦 that I wish I understood better than I do. Like an intrepid magpie 🐦 eyeing off a toddler’s unguarded lunch 👧 🥪, I spy the plumber package and its pretty pretty documentation

Plumber allows you to create a web API by merely decorating your existing R source code with special comments. Take a look at an example.

Ooh, decorating my code with pretties? Yes please! ✨ 🦋 ✨ 🎁

Okay, kid, if you insist.

I mean, I’m am pretty hungry and you don’t look like you’re eating that 🥪. I dunno though, the last time I checked out shiny web applications it turned out to be a whole big thing. This bird is a clever bird now 🦉. She ain’t gonna get fooled again.

One for sorrow

So, reading the documentation I can start with a short script histogram1.R that defines a simple function that draws a histogram, with a some “special” comments added that are prefaced with #*. Seems easy enough, but okay. I still think this sounds like a trap ☠️. But okay…

# histogram1.R

#* Plot a histogram
#* @png
#* @get /hist
function(){
  rand <- rnorm(100)
  hist(rand)
}

What next? According to the documentation 📚 all I have to do at the R console is type this?

library(plumber)           # package load
r <- plumb("histogram1.R") # path to the histogram1.R function
r$run(port=8000)           # set it running??
Starting server to listen on port 8000
Running the swagger UI at http://127.0.0.1:8000/__swagger__/

Okay, well that seems easy enough but I bet something is busted. Nothing ever works for a poor little bird like me the first time around. Expecting the worst, I open up my browser to http://127.0.0.1:8000/hist and I what I see is this…

Ooooooohhhhhh! 🎉 🎈

Two for joy

That’s awfully easy. Maybe next I’ll try to modify the function so that it takes inputs, allowing the user to customise the plot. The histogram2.R) file does that

# histogram2.R

#* Plot a histogram
#* @png
#* @get /hist
function(n=1000, mean=0, sd=1, breaks = 10){
  
  # coerce to numeric
  n <- as.numeric(n)
  mean <- as.numeric(mean)
  sd <- as.numeric(sd)
  breaks <- as.numeric(breaks)
  
  # plot
  hist(                                       # a histogram of...
    x = rnorm(n, mean, sd), breaks = breaks,  # ... normally distributed data,
    xlab = "", ylab = "", main = "",          # ... with no labelling,
    col = "#88398A", border = "white"         # ... coloured in purple
  )
}

Simple enough! As before I call r <- plumb("histogram2.R") and then start the server with r$run(port=8000). Just like last time it starts with no hassles.

Passing parameters to the plot is done through the URL query string, so if I want n = 10000 observations and set breaks = 50 for my histogram, the URL becomes

http://127.0.0.1:8000/hist?n=10000&breaks=50

The result is exactly what you’d expect!

This is too easy. This is definitely a trap. Here there be dragons 🐉, I just know it.

Three for a girl

So for my next step, I copy mindlessly from the documentation, and add a function that will just parrot 🦜 back whatever message gets passed to it. That gives me this script:

# twoendpoints.R

#* Echo back the input
#* @param msg The message to echo
#* @get /echo
function(msg=""){
  list(msg = paste0("The message is: '", msg, "'"))
}

#* Plot a histogram
#* @png
#* @get /hist
function(n=1000, mean=0, sd=1, breaks = 10){
  
  # coerce to numeric
  n <- as.numeric(n)
  mean <- as.numeric(mean)
  sd <- as.numeric(sd)
  breaks <- as.numeric(breaks)
  
  # plot
  hist(                                       # a histogram of...
    x = rnorm(n, mean, sd), breaks = breaks,  # ... normally distributed data,
    xlab = "", ylab = "", main = "",          # ... with no labelling,
    col = "#88398A", border = "white"         # ... coloured in purple
  )
}

It took me a moment to work out what was happening here. As I understand it, now my server has two “endpoints”, one of which is hosted at … actually, wait, if I’m mindlessly copying code I might as well mindlessly copy from the documentation too:

This file defines two Plumber “endpoints.” One is hosted at the path /echo and simply echoes the message passed in; the other is hosted at the path /plot and returns an image showing a simple R plot.

That makes sense.

Four for a boy

A parrot parrots again. More from the documentation:

In the previous example, you saw one endpoint that rendered into JSON and one that produced an image. Unless instructed otherwise, Plumber will attempt to render whatever your endpoint function returns as JSON. However, you can specify alternative “serializers” which instruct Plumber to render the output as some other format such as HTML (@html), PNG (@png), or JPEG (@jpeg).

Well, that explains what the commented sections of my plumber scripts are doing. For the histogram function these two lines are specifying the output format (i.e. use the @png serializer) and the path to the “endpoint” (i.e. /hist):

#* @png
#* @get /hist

I presume that the @get part is telling the server to expect a GET request rather than POST or whatever.

I admit that it is still a bit beyond my grasp of the internets to really understand the difference between how GET and POST work, but I’m sure at some point a wizard 🧙‍♂️ will come along and explain everything to me 😀, but for now I feel like I’ve worked out a few things.

Five for silver

I really like playing around with plumber. My usual process for learning any new programming skill is to start out by imitating whatever someone smart has done, and then gradually go back and forth between “trying something different” and “reading the documentation when it inevitably breaks”. Until now my only experience with web programming through are has been setting up Shiny apps for running psychology experiments. That’s really nice from the perspective of actually achieving things, but Shiny apps are complicated beasts, and it took me a long time even to get a feel for the basics. Using Shiny still feels like a kind of sympathetic magic to me, like I’m building an effigy or poppet; the power comes from the similarity between my code and the real thing, and I still don’t have a great sense of why I’m doing what I’m doing.

For me at least, getting past the “mere similarity” style of programming requires something more than just taking someone else’s beautiful thing and making small changes. I find it helps to try to approach the problem from a few different perspectives until it starts to hang together into something coherent. Seeing the similarities in superficially different things; trying to understand what is the same and what is different about a plumber API and a Shiny app; or what the server side code for these tools has in common with the very simplistic Python code I use to host experiments on Google App Engine; or whatever. It takes time, but piece by piece I feel like I’m starting build up some sense of how web programming actually works!

Of course, there’s still the small matter of deploying the code. I took a quick peek at the plumber::do_provision() function to deploy to DigitalOcean, and, well… 🐉 ☠️ ⚡ 💣 💥

Post script

One for sorrow,
Two for joy,
Three for a girl,
Four for a boy,
Five for silver,
Six for gold,
Seven for a secret,
Never to be told.
Eight for a wish
Nine for a kiss
Ten a surprise you should be careful not to miss
Eleven for health
Twelve for wealth
Thirteen beware it’s the devil himself.

(source)

Avatar
Danielle Navarro
Associate Professor of Cognitive Science

Related