Postprocessing LimeSurvey results

If you have suggestions on how to improve this document, or find mistakes, please send them to labman.gw@nulluu.nl

Introduction

LimeSurvey can (of course) output the results of your survey to various formats. There’s possibilities to output your survey results to Microsoft Word and Excel, PDF, comma-seperated values (CSV), SPSS and even R.

However, LimeSurvey outputs all results per participant in one single line. In a typical experimental setup, with your stimuli defined as question groups, this is usually not what you want: you would like a line per question group. This how-to will describe in detail how to “normalize” your data, with the help of a Python script. Do note that this will of course only work if you have the same number of questions in each questions group (of course, some introductionary questions are allowed).

Preliminaries

Downloading the results

You can extract your results from LimeSurvey per survey using the Responses & statistics tab. Choose Export results to application, and review the options. Make sure to output all columns (standard, only the first 255 columns are selected: this is the maximum column size for Excel 2003) and to select CSV (comma-seperated values) as format. If you have more than one survey with the same layout, you can download the results from that one as well, the script will merge these results into one file. 

Installing Python

If you’re working on a lab PC, Python will already be installed, and you can skip this step. If not, download and install Python from here (note: there’s also a 3.x version, but the 2.7.x version will suffice for this manual). If you’re on a Solis-PC under Windows (which we’ll safely assume below), make sure to choose an installation directory in which you actually have write rights, so for example your “My Documents” folder. 

Running the script

Create a file process.py in your installation directory (actually, it can be any directory if you have enough rights to alter the PATH variable or are on one of the lab PC’s). Copy the following text into it: 

import csv
   
   # Constants for header and group sizes (to split up .csv-file)
   HEADER_SIZE = 9
   GROUP_SIZE  = 7
   
   # Writes the results to the output file per group
   def run(out, filename):
       with open(filename, 'rb') as csv_file:
           # Create reader for the input file
           reader = csv.reader(csv_file, dialect='excel', delimiter=',')
           next(reader, None)  # skip the header
           
           # Loop over input file
           for row in reader: 
               # Write the results to the output file
               for n, item in enumerate(row):
                   start_ok = n > GROUP_SIZE
                   modulo_ok = (n-HEADER_SIZE+1) % GROUP_SIZE == 0
                   end_ok = n + GROUP_SIZE < len(row)
                   
                   if modulo_ok and start_ok and end_ok:
                       out.writerow(row[:HEADER_SIZE] + row[n+1:n+GROUP_SIZE+1])
                       
   # Main loop of the program. Calls the run method above.
   def main():
       with open('out.csv', 'wb') as out_csv:
           out = csv.writer(out_csv, delimiter=',', quoting=csv.QUOTE_MINIMAL)
           out.writerow(['dcol1', 'dcol2', 'dcol3', 'dcol4', 'dcol5', 
                       'hcol1', 'hcol2', 'hcol3', 'hcol4', 
                       'col1', 'col2', 'col3', 'col4', 'col5', 'col6', 'col7'])
                       
           run(out, 'results-1.csv')
           run(out, 'results-2.csv')
           
   if __name__ ==  '__main__':
       main()

Move the result file(s) into the same folder. Open process.py again and make sure that you refer to your result file(s) in the run() function near the end of the script. Check whether the HEADER_SIZE and GROUP_SIZE are set to the correct values. The HEADER_SIZE designates the standard LimeSurvey output columns, including (if you have that) some introductionary questions (e.g. age, gender, etc.). The GROUP_SIZE designates the number of questions per question group. 

Make sure to also rename the columns dcol1, dcol2 … dcol5 (default LimeSurvey output), hcol1 … hcol4 (introductionary questions) and col1 … col7 (result columns per question group), or add/delete them if your header or group size is different. 

Now open a command prompt (or terminal) and move to the directory of the script and result file(s). Write python process.py as your command and push <enter>. The result file(s) should now be processed. If you did everything correctly, you should now have a merged, normalized result file called out.csv in the same directory. 

If you seem to have any problems, be sure to contact lab support. They’re happy to help!