Questionnaires with Likert Scales Using SoSciSurvey and GNU R

In a recent post, I discussed different online questionnaire tools. My personal preference is still SoSci Survey, because it’s support for complex Likert scale questions, the good options for data export, and the support for reuse. In this post, I focus on visualizing results from the System Usability Scale (SUS) question block that is available in SoSci Survey. For this, I am using GNU R and the likert package.

Step one: Get the data

In SoSci Survey you can download the data for GNU R together with an import script. This works fine, but you need some more work to actually make sense of the data in R. Store both files in a folder.

Step two: Load the data in R

Start your R command line (btw: I like this reference) in the folder where you saved the data and import script. The first line of the script reads

tmpImportFile <- file.choose()

In the command line, you now need to type in the name of the data file. The result of the rest of the import script is a data frame with all your questions, answers, and some metadata.

Step 3: Select data for the SUS question

Let’s say we are only interested in the usability questions. We can extract them as follows:

sus <- data[substr(names(data), 1, 4) == "SU01"]

(Optional) Step 3a: Clear text questions

The import script stores only the question and answer ids in the data frame. It adds the full text question only as a comment. In my case, I need to translate the questions from German (used in the questionnaire) to English (which I want to use in the report). I do this like this (more elegant ways are probably available, but I do not want to lose time here):

sue <- rename(sus, c(
  SU01_01 = "I think that I would like to use this program more often.", 
  SU01_02 = "I feel very confident while using this program.", 
  SU01_03 = "I think that this program is easy to use.", 
  SU01_04 = "I assume that most people will learn quickly to use this program.", 
  SU01_05 = "I think that the different functions of this program are well integrated.", 
  SU01_06 = "I think that this program is unnecessarily complex", 
  SU01_07 = "I think that I would need the support of an experienced person to use this program.", 
  SU01_08 = "I had to learn a lot before I could use this program.", 
  SU01_09 = "I think the program was very clumsy to use.", 
  SU01_10 = "I think that there are many inconsistencies in this program."

If you encounter a problem here: On my system, rename becomes available, when I load the likert package.


(Logical step 3b) Use the Likert package

If it was not for some small detail, we are basically done and can run some nice plots with the likert package:

lsus <- likert(sus)
Error in likert(sus) : 
  All items (columns) must have the same number of levels
In addition: Warning message:
In likert(sus) :
  items parameter contains non-factors. Will convert to factors

There is one error and one warning. The warning is that the likert package converts our data in factors (i.e. a discrete grouping of components [R-intro]). The error is a consequence of this and of the fact that I am using only a limited amount of answers for this walk-through.  For this reason, not every question has all possible replies and the levels (possible values the factor can assume) seemingly differ between the answers to SUS.

Step 4: Specify factors

So now we need to define the factors with the appropriate levels and labels. In this example I adjusted the original questions I reused in SoSci Survey to have a 6-point Likert scale (5-point Likert scale in the original).

i <- 1
while(i<=ncol(sue)) {
  sus[[i]] = factor(sus[[i]],labels = c("strong disagree","disagree","rather disagree","rather agree", "agree", "strong agree"), levels=c(1,2,3,4,5,6))
  i <- i + 1

Step 5: Apply the likert package

lsus  = likert(sus)


Or, if you have some other question in your questionnaire, you can use this to group the data:

lsusg = likert(tq4, grouping=data$A003)
plot(lsusg, type = 'density')

So, my main motivation for writing this down is that this work should not take more than 15 minutes, if you know how. I did not, so I spend quite some time figuring out the details. So this post, while hopefully useful to others, is mainly for my future reference. As is the following link on how to save plots in R.


Special thanks to Daniel Ott, Daimler AG, for feedback to this blog post as well as for an exciting research project which included a prototype with just enough usability (as can be seen in the examples above)


8 thoughts on “Questionnaires with Likert Scales Using SoSciSurvey and GNU R

    • First, my apologies for taking so long to answer and my thanks for sharing your thoughts.

      Good point. I am at the moment unable to try it out, but it seems that likert(sus) would produce the untranslated labels. I would be happy to share (a link) to a better / more comprehensive example here if it could be provided!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s