ShinyPsych

Programming online experiments using R and Shiny

Nathaniel Phillips
Basel Research Colloquium - March 16, 2017

I want to program a study, what software can I use?

  • Lab-based study
    • Psychopy (Free, open source)
    • Presentation (Not free, not open source)
  • Online study
    • Survey software: Unipark, Qualtrics, Inquisit (Not Free, not open source)
    • Custom HTML / JavaScript

plot of chunk unnamed-chunk-1plot of chunk unnamed-chunk-1

Comparing online software

Software Time to Learn Cost Customizability Best Suited for
UniPark / Qualtrics Medium ? Low Questionnaires
Inquisit Medium / Low ? Medium (?) Questionnaires, Simple stimuli
Custom HTML / JavaScript High Free Very High Complex studies
Shiny Medium / High Free* High Complex, statistical stimuli

What is Shiny?

  • Shiny is a web application framework for R.
  • From the website: Turn your analyses into interactive web applications. No HTML, CSS, or JavaScript knowledge required.
  • Shiny applications are called "Apps" (aka, experiment)
  • Shiny Example Gallery: http://shiny.rstudio.com/gallery/

plot of chunk cookmap

What is Shiny?

  • Shiny was developed to let people easily make cool, interactive websites with data, not to create experiments.
  • However, just like psychologists have appropriated the Amazon mTurk for participant recruitment, I think we can also appropriate Shiny to create online experiments.
  • Over the past few months, I, along with Markus Steiner and Kevin Trutmann (also Janine Hoffart) have been 'hacking' Shiny to run experiments.

plot of chunk unnamed-chunk-3plot of chunk unnamed-chunk-3plot of chunk unnamed-chunk-3

Programming a Shiny "App"

  • Combination of standard R code, and special Shiny R functions that create and manage interactive elements.
  • Shiny has lots of 'widgets' programmed http://shiny.rstudio.com/gallery/widget-gallery.html
  • You can copy widget code, and assign the results to an R object.

Programming a Shiny "App"

sliderInput(inputId = "prob.est", 
            label = "What is the probability of X?", 
            min = 0, 
            max = 100)

Becomes

plot of chunk unnamed-chunk-5

Programming a Shiny "App"

  • Combination of standard R code, and special Shiny R functions that create and manage interactive elements.
h3("I don't have to do a lot of thinking")
radioButtons(inputId = "REI_q1", 
             label = "1 = Disagree, 5 = Agree",
             choices = list(1:5))

Becomes

plot of chunk unnamed-chunk-7

Programming a Shiny "App"

  • Once a participant completes an input, the result is stored as an r object you can then use any way you'd like
# Store result in prob.est
sliderInput(inputId = "prob.est", 
            label = "What is the probability of X?", 
            min = 0, 
            max = 100)

# Use prob.est in other code

true.value <- 33
error <- abs(input$prob.est - true.value)

h3("Your estimate was off by ", error)

Programming a Shiny "App"

  • You don't program explicit pages, rather, you update what's in the display based on some criteria:
# Definition of welcome page
if(CurrentValues$page == "welcome") {

  h3("Welcome to the experiment"),
   textInput(inputId = "workerid", 
                    label = "Please enter a unique ID that no one else would use", 
                    value = "", 
                    placeholder = "e.g.; Cat57Door"),
  # This displays the action putton Next.
  actionButton(inputId = "gt_Instructions", 
               label = "Continue to Instructions")

}

observeEvent(input$gt_Instructions, {CurrentValues$page <- "instructions"})

Programming a Shiny "App"

  • You don't program explicit pages, rather, you update what's in the display based on some criteria:
# Definition of instructions page
if(CurrentValues$page == "instructions") {

  h3("Instructions"),
  p("In this experiment, you will....."),
  p("On the next page, you will play a practice game"),
  # This displays the action putton Next.
  actionButton(inputId = "gt_Practice", 
               label = "Continue to Practice")
}

Example: A simple survey

Go to ShinySurvey

Programming a Shiny "App"

  • Randomizing stimuli order is no problem.
  • You can write 'pseudo-loops' to sequentially display questions or other stimuli automatically and then store sequential responses.
rei.survey <- list(
  "text" = c("I don't like to have to do a lot of thinking",
             "I try to avoid situations that require thinking in depth about something",
             "I prefer to do something that challenges my thinking abilities rather than something that requires little thought",
             "I prefer complex to simple problems",
             "Thinking hard and for a long time about something gives me little satisfaction",
             "I trust my initial feelings about people",
             "I believe in trusting my hunches",
             "My initial impressions of people are almost always right",
             "When it comes to trusting people, I can usually rely on my 'gut feelings'",
             "I can usually feel when a person is right or wrong even if I can't explain how I know"))

Programming a Shiny "App"

  • You can write 'pseudo-loops' to sequentially display questions or other stimuli automatically and then store sequential responses.
# P3) REI
if (CurrentValues$page == "rei") {
    list(
      h3(paste(rei.survey$text[current.question])),
      em(paste(rei.survey$description)),
      radioButtons(inputId = "rei_survey", 
                  choices = rei.survey$labels),
      actionButton(inputId = "nextq", label = "Continue"))
    }

observeEvent(input$nextq, {
    CurrentValues$question.num <- CurrentValues$question.num + 1
    CurrentValues$rei <- c(CurrentValues$rei, input$rei_survey)
  }})

3 examples

How to run a ShinyApp

  • In order for participants to use your App, it has to be hosted on a Shiny Server.
  • You can run up to 5 apps for free at RStudio. However, you are limited to 25 active hours per month
  • 3 options for advanced hosting: RStudio Standard ($1,100 / year), Amazon Web Services, or in-house ($4,500 / year).
  • If there is enough interest, our IT will set up an in-house server

Shiny Pros and Cons

Pros

  • You can integrate all of your data management using R code within the experiment.
    • Great for dynamic experiments where stimuli depend on responses.
  • Automatically export data in exactly the format you want to Dropbox.

Cons

  • Simple things you get for free in Survey software require extra code.
    • Required responses, checking for invalid responses.
  • Crashing is, unfortunately, somewhat common.
    • No more than 20 simultaneous users.
  • Response time measures are almost certainly not very precise
    • >500ms is probably ok, <500ms is probably not

What's next?

  • We are currently programming and documenting standard psychology experiments in Shiny and distributing them at http://www.github.com/ndphillips/ShinyPsych.

  • Check out the code, play around, let us know what works and what doesn't, and contribute!