In a recent post, I discussed different online questionnaire tools. My personal preference is still SoSci Survey, because it’s support for complex Likert scale questions, the good options for data export, and the support for reuse. In this post, I focus on visualizing results from the System Usability Scale (SUS) question block that is available in SoSci Survey. For this, I am using GNU R and the likert package.
Questionnaires are a funny thing. Obviously, they are a great tool for researchers. They seem to be so simple. But it is really difficult to create a good one. Two years ago I tried an online questionnaire tool for the first time and I was impressed on how helpful it was.
First, the assistants and editors such tools provide allow to quickly create a questionnaire with a good layout of questions. Then, you get the results in a format that allow analysis even with little experience. You do not have to bother with setting up a webpage.
As a drawback, you have to find the right tool or service provider. And this proves to be difficult. There is all kinds of advertisements going on, service providers try to lure you with features you don’t need (in my case: Facebook integration) and make it almost impossible to discover that they do not offer features you do need as part of their basic plan, before you have created and posted your first questionnaire (in my case: export of the data as CSV or similar). Here are some of my experiences. Please let me know if there is more.
Further reading: How to questionnaire?
Wai-Ching Leung: How to design a questionnaire. These three pages are a great read. Leung provides a checklist with do’s and don’ts that quickly help to create and improve a first draft. It also includes some inspiration for longer thinking but might be to brief for real experts.
Idealware: A few good questionnaire tools. A good starting point for choosing a tool. I especially liked the discussion.
Where to do it
Google Docs: This is pretty straight forward. Just create a Google Form as a front-end to your Google spreadsheet. The questionnaire can be directly integrated into the email, it looks great and might be the best solution for a quick poll. From Google spreadsheets, you can do all the analysis you want to do and export the data in all possible formats. Maximum freedom, but limited support for more advanced types of questions.
Surveymokey: Google search gave me this tool as first result. It looks great. I love the question editor. I was able to create a nice questionnaire pretty quickly. What I do not like: I cannot export the data in the free plan and the restrictions (amount of responses, amount of questions) might actually be hurtful. I have to check if they offer free upgrades for education and not-for-profit research.
SurveyMoz: Extremely good price and lots of important things in the extreme plan. However, the data only exports to Microsoft Word in the free plan, which would be a no go. There is a free upgrade for educational and non-profit organizations, which mitigates this problem. Did not try to create a questionnaire yet, because apparently one cannot sign up with a Mac (Exception: Validation of viewstate MAC failed.)
SoGoSurvey: The new star among the questionnaire tools. It has a very reasonable free package and low prizes. My opinion: Has all the functionality I need, even in the free plan, but the editor is more complicated and not as straight forward as SurveyMonkey, without direct support for a set of questions with the same Likert-scale.
SoSciSurvey: This is the provider I made my good experience two years ago. Although this seems to be a German project, you can switch the language and create surveys in English, too. If I look on the page today, it looks a bit old-fashioned, but it is a solid tool, especially focused on the needs of university research. It has all the question types I need, excellent data export (e.g. to R), great examples. For example, you can integrate standard usability questions with one click. This is the tool I am currently recommending when asked by students.
Please let me know about your experiences!
Since a few month now, we are constantly observing the progress of an amazing project: Four adventurers of OAR Northwest are rowing(!) across the Atlantic Ocean from Dakar to Miami. The project aims at educating (young?) people towards a sustainable and healthy lifestyle. It also supports a high number of research projects.
The team allows us to take part in their adventure via social media and the partly dramatic accounts show that this project is not only a “metaphor for important life lessons” (Adam Kreek), but that it is literally an example for reasonable management of very limited resources.
It is also a great example of excellent team spirit, keeping a positive attitude, and stay motivated, despite overwhelming challenges and a seemingly endless task. And there is one particular picture, that shows this more than any words:
That is right, the crew spend almost two days on painting a heart with their trajectory, after being caused to leave their course for some reason. Despite the fact that there potentially is a longer and more profane story behind this, this picture has become a symbol for caring about beauty in your actions to me.
Learn more about this exciting project and lessons you can draw from it on http://oarnorthwest.com. And if you wonder, how this relates to Software Engineering, you can find a more specific answer here.
Yesterday, I participated in the workshop on empirical requirements engineering. My notes on the talks will follow shortly, but I want to highlight the excellent panel that concluded the workshop. The panelists where Nan Niu, Jörg Dörr and Neil Maiden. Note, that the panelists were required to be provocative in their statements. Also note that I will add more links and references to this article later. Continue reading