-
Notifications
You must be signed in to change notification settings - Fork 1
Home
Collector Documentation Documentation by Dr. Anthony Haffey on behalf of Scientific Open Solutions. If you cannot click on a button due to your screen size, you can zoom out using CTRL Minus(-) (Windows) CMD Minus(-) (Mac) If you identify bugs or errors please try to save CTRL S (Windows and Mac) CMD S (Mac) and refresh the page: CTRL-R (Windows) CMD-R (Mac) Please do report any bugs or errors at: https://github.com/scientific-open-solutions/collector/issues If you have suggested clarifications, please go to https://github.com/scientific-open-solutions/collector/discussions The most up to date version of this documentation can be found at: https://docs.google.com/document/d/1SKYIJF1dAjMDS6EHUIwfZm2KQVOzx17S6L bU_oSGxdE/edit?usp=sharing License Collector (Garcia, Kornell, Kerr, Blake & Haffey) A program for running projects on the web Copyright 2012-2016 Mikey Garcia & Nate Kornell This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License version 3 as published by the Free Software Foundation. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program. If not, see http://www.gnu.org/licenses/ Kitten release (2019-21) author: Dr. Anthony Haffey ([email protected]) Table of Contents Collector Documentation 1 License 1 Table of Contents 2 Installing Collector on your machine 5 Advantages 5 Creating github account (and organization) 5 Windows 6 Mac 7 Linux 8 Developer mode (Node Js) 10 Once you have a copy of Collector on your computer 11 Linking Collector to github (for installer version only) 11 Creating projects 16 Conditions sheet 16 age_check 17 buffer 17 download_at_end 17 end_message 17 fullscreen 18 forward_at_end 18 mobile 18 name 18 notes 18 participant_id 18 procedure 18 start_message 19 stimuli 19 Phase editor 19 Graphic editor 19 How it looks 22 Timeline 24 Capturing response with keyboard 26 Capturing the response with the mouse 27 Changing the canvas size 27 Code editor 28 Saving responses 28 Default javascript and css packages 28 Collector functions 29 Development variables for previewing 30 Procedure sheet 31 Shuffle 1, 2, 3 etc. 31 shuffle 2, 3, 4 etc. 32 Phase type 32 Item 33 Max time 33 Text 33 timer_style 33 Weight/freq/frequency/repeat - useful for piloting your task 34 Stimuli sheet 34 List Stimuli button 34 Cell editor 35 Helper sidebar 38 Downloading and Uploading projects 38 Running your study 39 Save! 39 Click the “run” button 39 Publishing your task 40 Saving for installed version only 42 Have multiple studies in a row 42 Surveys 43 Columns 44 answers 44 btn_width 44 feedback 44 feedback_color 44 item_name 44 no_text 44 optional 45 score:[your score name] 45 shuffle 45 side_text 45 side_by_side 45 text 46 type 46 values 47 Including a survey into an project 47 Data 48 Setting up a server for Saving and emailing data 48 Telling Collector where your saving/emailing script is 52 Decrypting your data (and understanding it) 53 Decrypting your data within the “Data” tab 55 Understanding your data 56 FAQ/General questions 57 I need to write an ethics application, do you have some template text for how the data storage is handled? 57 How do I make a copy of an project? 57 Can I use Collector if I am not a programmer? 57 What language is Collector written in? 58 Do I need to ask anyone if I want to use Collector? (short answer: no) 58 How do I cite Collector? 58 Troubleshooting 58 Further notes 58 I’m a participant, how is my data encrypted? 58 What happens with my password for encryption and decryption 59 What happens with my password for Collector 59 Advice/Tips 61 Collaborating 61 Collaborating with dropbox version 61 Collaborating with github/installed version 61 Contributing 62 Original contributors 62 Mikey Garcia 62 Tyson Kerr 62 Victor Sungkhasettee 62 Adam Blake 62 Nate Kornell 62 Installing Collector on your machine Advantages There are some advantages of installing Collector on your machine and then pushing the changes to a github repository where your site can be hosted (all for free):
- You don’t need to wait for files to be saved on Dropbox before seeing changes to your task. This can save time with previewing.
- Your task might run a little smoother if you are using images, videos or audio files. This is because you can upload these media files onto your github repository, so there is no cross-site delay between github and dropbox when requesting for the images, videos etc. However, hopefully the buffering that Collector does should avoid or minimise any delay caused by calling in files from dropbox into your project. Creating github account (and organization) You will need a github account. Please go to: https://github.com/ and login or create an account. Once you have an account, you will need to create an “organization”. This will allow you to collaborate with other researchers if you choose to share your repository with them later. This step cannot be skipped. Just click on the + next to your icon in the top right corner and select New organization Select the “Join for Free” option when offered different plans. The remaining options you can answer how you like or skip in some cases. Windows Go to https://github.com/scientific-open-solutions/collector/releases/ to download installers. Then please go to the most recent version. Download the “Collector.Setup.VERSION.exe”. Once it’s downloaded you should see the following (if using google chrome): To sidestep that message, click on the arrow next to “Discard” and then select “Keep”. You should then see: And thus be ready to install Collector by clicking on the file. You then should see the following: If you then click on more info then you’ll be able to click on “Run anyway”. You can go with the default options which will make Collector software that is just for you as a user on the computer, rather than all users on the computer. Assuming that all went smoothly, you’re ready to go to Linking Collector to github. Mac If you have security problems at any point, go to the final paragraph at the end of this subsection. Go to https://github.com/scientific-open-solutions/collector/releases/ to download installers. Then please go to the most recent version. Download the the Collector-version.dmg file. Once it’s downloaded, you should be able to click on it (this is what it looks like in Google Chrome): Then you can just drag Collector into “Applications” to install it. You might get a warning at some point about using apps that are not from the app store or recognised developers. To sidestep this issue, go to system preferences then security and privacy. Open the general panel (as opposed to the FileVault, Firewall and Privacy panels). If you’ve tried to install Collector then you might have an option to approve Collector specifically. If not, then you may need to click on the lock at the bottom to make changes. Once you’ve done that you should be able to change your settings for Allow apps downloaded from to “App Store”. You should now be able to install Collector. Assuming that all went smoothly, you’re ready to go to Linking Collector to github. Linux Go to https://github.com/scientific-open-solutions/collector/releases/ to download installers. Then please go to the most recent version. The linux version ends with .AppImage. Download that, and then relocate it to somewhere you would like to keep it. You need to give Collector permission to run. One way of doing this is to right click on Collector and select properties: Then you need to go into the permissions tab and select “Allow executing file as program”: If you’re unable to do the above, you should be able to give permission to Collector using the terminal with the following command (once you’ve navigated to the correct directory in the terminal): chmod a+x Collector-VERSION.AppImage You could then open it in the terminal with ./Collector-VERSION.AppImage (or double click on it within the normal finder) Assuming that all went smoothly, you’re ready to go to Linking Collector to github. Developer mode (Node Js) Developer mode can be useful if you’d like to make changes to how Collector works when running your projects. This could be part of you either customising Collector for your needs, or to start giving suggested improvements to Collector. To use Collector in developer mode, you will need to download and install Node-js. Windows 7 users: You will need to download Node JS 13 rather than the most recent version of node-js. To do this, please go to: https://nodejs.org/en/download/releases/ Everyone else (including Windows 10 users): you can download the most recent version of node-js from https://nodejs.org/en/. If you just want to use Collector, but don’t want to make suggested improvements, download if from the main repository: https://github.com/scientific-open-solutions/collector . You can skip the following instructions about contributing to Collector’s development by clicking here. If you would like to make suggested improvements to Collector, then please create a github account if you don’t have one yet at https://github.com. Once you have an account then you’ll be able to fork the developmental repository at https://github.com/scientific-open-solutions/collector-dev using the fork button: Once you’ve forked it, you’re ready to clone Collector onto your computer, which can be done with github desktop (https://desktop.github.com/) on Mac and Windows machines. Or you can use the command line in the terminal (or powershell for Windows):
syntax git clone {{url of your forked version of Collector}} Once you have a copy of Collector on your computer To open Collector in this developer mode, you’ll need to open terminal (if Linux or Mac) or powershell (if Windows) and navigate to the “web” subfolder within the Collector folder. Once you’re there, if you have node-js installed you should be able to type npm install This should help install all the packages you need for Collector. You may also need to specifically install electron, which you can do with the following command: npm install electron Assuming all the above has gone well, you can start Collector with: npm start Assuming that all went smoothly, you’re ready to go to Linking Collector to github. Linking Collector to github (for installer version only) Once you’ve installed Collector, the first time you run it you should see: You can ignore this for now, so click on “Cancel”. You now need to either create a repository, or clone a repository that already exists. In either case you need to click on the github icon in the top right corner: Which will give you the following interface: You will need to have created a github account and be a member of an organization to proceed. If this is not the case yet, please go back to creating github account before proceeding. You first need to create and add an authentication token. You’ll only need to do this once (unless you lose your token when reinstalling Collector). To create an authentication token you need to be in github, and then click on the arrow by your profile picture in the top right: Click on Settings from this list, and then Developer settings on the list on the left on the next page. Then click on Personal access tokens and then Generate new token. You then may need to put in your password. When you see the following, replace where I’ve written “example_token” with any name that you like (or use example_token), and then click on the “write:packages” checkbox. That will automatically tick the other checkboxes you need and you should see something like: You now need to scroll to the bottom of the page to click on the Generate Token button. You should then see a token, click on the clipboard icon next to it to copy it, and then go back to Collector. Back in Collector, you now need to add your new token, so please click on Add token: You can then either press CTRL-V to paste the code into the input, or right click on the text input and select “paste” to paste in your access token. Once you’ve successfully added the token click “Okay”. Now you’re ready to either create a new repository to allow you to backup and push your projects online, or clone in an existing repository. First, click on the Add button on the row with Organisation on the far left. Add the name of your organization (you will need to have done this during creating github account (and organization) if you have not been added to an organization by another user). Hopefully your organization is all lowercase, but if not make sure that you get the cases correct. Once you have added the organization, you can add a repository. If the repository already exists in this organization, it will then Clone onto your computer, and you will be able to make changes to it on your computer and push those changes to the online repository. If the repository doesn’t exist yet, then it’ll need to be created on your computer, and then created online, before you start working on it. In either case you should see the following: Once it’s done it’ll automatically restart Collector, and then you’re ready to start using it! Creating projects For the purposes of this documentation, I’ll be using the example of creating a stroop task called “stroop_example”. To create a new project just click on the “New” button at the top left of the screen: If you’re developing your project online rather than with an installation, give it a few seconds for the project creation to finish. You’ll know it’s ready because the new name will be where “Select a study” was before: Conditions sheet (Can be edited directly as a spreadsheet, e.g. using Excel, in the installed version of Collector - go to the User folder) The conditions sheet controls some of the settings of the project. For example, if you want the user to have a randomised id rather than what they typed, then change the cell just below “participant_id” from “on” to “random”: Have a look around at the different settings (notice how the “Help” sidebar tells you about each of them as you click across different columns). If you want to have different participants experience different conditions, then describe the differences in the different rows. This will result in you creating slightly different links for each version of the task. The differences between the different conditions will largely be due to differences between the different procedure and stimuli sheets. To create a new procedure or stimuli sheet, click on the “New Sheet” buttons in their respective sections. However, here is what you can specify in the conditions sheet: age_check Whether you want to restrict your study to participants aged 18 and above By default the project will tell participants to not participate if they are under 18. You can turn this off with the value “no”. buffer Number of trials you want buffered By default, Collector will try to have 5 trials in advance loaded to try to make the project run as smoothly as possible. You can change this number to whatever number of trials you want to have in advance download_at_end Can participants download at the end? By default, participants will be able to download their data at the end of the project. This can be turned off if you don't want this. BUT YOU ALMOST CERTAINLY WANT THIS! end_message A message at the very end This will replace the default message at the end (which includes their completion code). If you want the participant to see their completion code, do NOT have an end_message. Instead, have an instruct trial at the end of your procedure with whatever you want your last message to be to them, and let them proceed to the default end message and completion code. fullscreen On or off? By default, the project will NOT be in Fullscreen mode. Change this to on if you want to request the participant to go into fullscreen mode. forward_at_end Where should the participant go next? By default, the participant will not be prompted to complete any other links. However, if you want the participant to complete another link after your task has saved all their data then just write the link you want them to go to next in this column. This could be useful for sending participants to urls on Prolific academic and SONA to confirm they’ve completed your study. mobile Whether the user is allowed to use a mobile device By default the user is allowed to use mobile devices - but you can add this column and write “no” and this will prevent them completing the study on one. name Name of Condition Please don't put any spaces in this name. You can use underscores _ instead though notes Just for you This doesn't do anything in the project, but you might find it useful for adding any notes you need for each condition. participant_id Participants give their ID By default, participants will give a participant id (e.g. a Prolific Academic or MTurk ID) at the start of the project. Change this setting to either random (recommended) to generate a unique and randomised id for the user, or off (not recommended) for the participant to have the id 'notUnique'. The participant_id influences the name of their data file. If the participant does not put in an ID, they will skip the start_message. procedure Which Procedure sheet? By default, your project will use procedure_1.csv, which is a list of the order of trials and settings for each Phase. If you want different participants to have different procedures, you will need to create a new procedure sheet, and have separate rows in the condition sheet for each procedure. If you are developing your projects online (e.g. on open-collector.org or ocollector.org) ignore the following. If you are developing the project on your computer, you should your procedure sheet in the web/User/projects/YOUR project folder. start_message A message at the very start This will replace the default message at the start of the project. If participant_id is on off or random then this message will be skipped. In this case, start your project with an instruct trial with the message you want to be the first thing the participant will see. stimuli Which Stimuli sheet? Stimuli sheets can used as a list of words or media files (images, audio, pictures) that your project will use. If you want different lists of stimuli, you need to create a new stimuli sheet. If you are developing your projects online (e.g. on open-collector.org or ocollector.org) ignore the following. If you are developing the project on your computer, you should find your stimuli sheet in the web/User/projects/YOUR project folder. Phase editor Graphic editor Phase types are types of trials the participant will experience. To create a new phasetype just click on the “New” button towards the bottom of the page: And then you’ll be asked for a name and whether you want to create the phasetype using code or the graphic editor. Let’s start by talking through how to use the graphic editor. So let’s click on “Using Graphics”: You may have noticed that I’ve given exactly the same name to the project and the phasetype “stroop_example”. I find this helpful as they can never get confused with each other. Once you’ve created the new phasetype you should see the the graphic editor: For a simple stroop project, we’ll start by adding some text. Just click on the Text button: And then click where you want it. I’m going to try to put this pretty close to the center of the page: Let’s unpack what we now have. In the top right corner we have the element_id “canvas_element_0”: This can be useful for making the project interactive, but we don’t need to worry about that for now. Just below that we have the actual text in this element: As it stands, every time you use the trial “stroop_example” you will get the text “canvas_element_0 html” - which is not particularly helpful. Let’s change that to just say “hello world” to see exactly how this works: So you could theoretically create a separate trial with a different word or series of words for each Phase. However, that would be very time consuming, so we’re going to embed a variable instead. Variables change for each trial depending on what you have in your procedure and stimuli spreadsheets. You just write the column title in the {{}} brackets. For a stroop task, we’ll make use of the fact that stimuli sheets already have a cue column. So let’s just refer to that column (we’ll edit the stimuli sheet later). We’ll replace “hello world” with “{{cue}}”. How it looks Play around with the “How it Looks” panel to see the different sizes and colors you can have. You might find that the code editor is in the way of the lower options: You can hide the code editor by clicking on the “Code” button at the above the graphic editor: You might need to click it a couple of times before the code editor disappears.You can click it again to make it come back. This can be a nice way to learn code - as you edit the project using the graphic editor, the code is written for the trial type. But let’s get back to making our stroop task. Being a stroop task, color is something that will also have to be a variable that changes from trial to Phase. So I’m going to replace “black” with “{{letter_color}}” which is a column I’m going to need in my stimuli sheet later. I’m also going to make the font-size 30px; and the width 300px: It’s important to have an appropriate width; if your width is too small your text will run over multiple lines. You might have noticed this in the earlier example with “hello world”. Timeline At the top of the graphic editor, we get a sense of what happens when. In our case, let’s:
- Add a fixation cross at the start for 500ms
- Add a 500ms gap following this by changing the start time of the word to 1000ms To add a fixation cross, we just add another text element as described above. I’m going to make the font-size 100px and put it 250px from the top and left. This should put it in the center of a 600 x 600 pixel Phase. To make it only be present for 500ms I’m going to scroll down to the interactivity panel, and change the hide(ms) value to 500ms: (Note that you can click on how it looks or interactivity headers to hide and show these interfaces). Now that we’ve changed the hide time for the fixation, our timeline at the top looks like this: The dotted blue line means that at least one of the elements has no stop or hide time (i.e. will display until the end of the trial). Pay attention to how the timeline changes as we change the show and hide times to the stroop word. But first, let’s change the element names so that the timeline makes more sense. Just click on the element name: And you’ll be asked for a new element id. I’m going to change the fixation cross element’s name to “fixation_cross”: I’m also going to change the stroop word (“{{cue}}”) element id to “cue_word”. To switch between elements, either click on them in the timeline, or click on them on the canvas. The timeline is a bit easier to read now: The next step is to have a 500ms gap between the fixation cross end and the start of the cue word. So let’s just show the cue word at 1000(ms). This option is just above the hide input we just updated: Let’s change that to 1000 and see what it does to our timeline: That seems sensible enough. Now we need to actually capture the participant’s response! Capturing response with keyboard For stroop tasks, you typically need to capture a participant pressing on a key that represents a color. For simplicity’s sake, I’m going to accept keyboard responses of “a” and “b”. To be able to set the valid keyboard responses click on the canvas anywhere that there isn’t an element. You should see the following: I’m just going to add the letters “g” for green and “b” for blue to keys. Notice also the End trial on key press checkbox. This is what it says - if you have this ticked then as soon as the participant presses any of the keys you put in the keys input, the trial will end. Capturing the response with the mouse This is less relevant for a stroop task, but if you want to see what the participant clicked on in the trial, then select that element, go to the interactivity pane and change the name and value. The name will be related to the column header in your data sheet. So I’m going to make the name “clicked_element”. The value will be what value you will put in that column if the participant clicks on that element. In this case I’m going to change the value to “fixation”. I’m also going to tick the End trial on click option. If you do the same you should see the following: Do the same for the cue_word element, but use the value “cue”. Changing the canvas size The canvas, i.e. the space that stores your phasetype, will by default be 600x600 pixels. If you want to change that, just click anywhere on the canvas where there isn’t an element, and change the trial width (px) and trial height (px) settings. Now we’re ready to use this phasetype in our project. Code editor (Phase type html files can be edited directly using editors like Atom and Notepad++ in the installed version of Collector - go to the User folder) Saving responses To save responses in Collector, just give the input with the response you want to save a “name”. For example:
s: If you’ve copied and pasted a bunch of text from another file (e.g. word), you may need to click on this to make sure that the text is spread across different lines of text in the way that you would like them to be. For example, if I copied: This is the first line of text This is the second line of text This is the third line of text I would get: Which is not what I want. However, when I press Add
s: (note the
s that have been added at the end of each line) B(old), I(talics), U(nderline): These don’t work quite the way as a normal text editor, but will generate the code needed to allow you to start writing in bold or italics or underline, or convert whatever you had highlighted accordingly. If you want to stop writing something in bold, for example, just press the right arrow until you’ve moved past the code that is making the text bold (e.g. YOUR TEXT). This can be a nice way to learn some basic HTML and css code. H(eader): Like the above, this will create a Header using an tag. Play around with the h numbers (e.g.
etc.) to see how big you want your header to be. This just allows you to change the color of your text, similar to the above. This allows you to change the size of your text. By default, it’ll assume you mean “px” or pixels, but you can explicitly say this, e.g. “12px”. This is on by default, so that you get live updates as you edit the cell contents. However, if you have a really really large cell, you might want to turn this off if your browser is slowing down. (I haven’t experienced this yet though…) Auto-preview gives a simplified preview, but doesn’t use the default libraries that Collector uses. Click on Full Preview to see what the content should look like in practice. (Having this on all the time is much slower than the light auto-preview). Will revert all changes you’ve made since opening the cell editor. If you click it accidentally, you should be able to press CTRL-Z in windows to go back to what you had before clicking the Undo All button. Helper sidebar On the right hand side there is a “helper” sidebar. You’ll notice that the advice it gives you changes depending on what you’re focusing on. For example, when I’m looking at the “participant_id” column in the conditions sheet I get the following help: Downloading and Uploading projects On either side of the project name, there’s an up arrow for uploading an project and a down arrow for downloading the selected project: This can be an easy way to duplicate an project. Just download it, and then upload it and give it a different name! If your project included phasetypes that were written using the Code editor (as opposed to the Graphic editor), Collector should ask you if you want to upload those phasetypes also. Similarly, any Surveys that were included in the original project should be downloaded and uploaded accordingly. Running your study Save! Make sure you’ve saved! You usually can just press CTRL-S to save. The best way to save all your progress is click on the save button in the top left of the studies interface: There are some differences between the online version (e.g. on https://ocollector.org or https://some-open-solutions.github.io/ocollector) compared to when you are running Collector locally on your computer after having installed it. Click the “run” button When you click on the run button: Run - this will run the study and save data on your machine. Preview Local - this will run the study, skip the quality checks at the beginning, and not save your data Preview Online - this will run the study as if you are running it online. This can help you check online data storage pipelines. You can change which condition you’re running using the dropdown list at the top. The link you’ll need to run the online version (which you’ll give to participants) will change when you change the condition, so you’ll need multiple links at the moment. Publishing your task After clicking on run → publish or just the icon on the top right of the screen, you should see something like: If you are a member of one of the following universities, you are automatically able to have your data e-mailed to you: - University of Reading (i.e. have an @reading.ac.uk e-mail address) - University of Reading Malaysia (i.e. have an @reading.edu.my e-mail address) - University of California, Los Angeles (i.e. have an @g.ucla.edu e-mail address) - University of DeMontford (i.e. have an @dmu.ac.uk e-mail address) If you are not a member of one of the above institutions you will need to complete the Saving data with google drive step before proceeding. To get your data e-mailed to you and backed up, put in your e-mail and password and then click on create account for the server you want to register with. Once you have successfully registered with that account, you can then Publish your projects to that server. Once you’ve successfully published your project, you’re ready to start data collection! Note, if you publish your project with multiple servers Collector will use some of the servers as back-ups in case the first server fails. Once you’ve reached here with the stroop task (and have successfully registered your project), close this interface and click on the run button as described above. You can now click on the online button to see how your task looks (and allow us to look at your data in a bit). Saving for installed version only You don’t need to read the following if you are using Collector purely online, e.g. on https://ocollector.org or https://some-open-solutions.github.io/ocollector. If you have installed Collector and are running it on your machine, you will need to push your changes onto the Collector Repository you set up. Have multiple studies in a row You might find that trying to have everything your participant does in one project is a bit too complicated to fit into one procedure sheet and stimuli sheet. A way to deal with this is to break your research up into multiple projects. To make the participant move from one project to another, add a “forward_at_end” column to your conditions sheet, and then put the URL of the next part of your project in there. To find out the URL, open the project in question and run it, selecting the “online” option. Surveys To create and edit surveys, go to the Survey tab at the top: You cannot create a survey from scratch, you need to select a previously existing survey to base the new survey on. I’m going to use demographics just because it has a lot of different types of questions. Just select “demographics.csv” and then click on new survey: The helper bar on the right should tell you a lot about each column depending on where you click in the survey. So I’m just going to highlight the fact that after each change to the survey you can preview you changes by clicking on the preview button: If you want to go back to editing, just click on the Spreadsheet button to the left of the preview button: Columns answers Answers the participant can choose A single answer or list of answers the participant can choose. Put a pipe character ( | ) between each possible answer. If you're having problems finding the pipe key, you can highlight the pipe in the brackets and then copy and paste it. btn_width How wide each button will be (likert scale) This is for likert questions only - you can set it so that each button has the same width as each other by setting a specific width (e.g. 200px). feedback Tell the participant about each answer You can list what feedback you associate with each response. Put a pipe | between each feedback feedback_color What color is each feedback The colors you want to associate with each feedback (e.g. green for the correct ones, and red for the wrong ones. Put a pipe | between each feedback color item_name Column name in data sheet The item_name will be what the column header is in your data sheet. So make sure it's unique! no_text No text on left side of survey If you have an item, in which you want the content that would normally be on the right side of the survey to take who whole width of it, put 'on' in this column. optional Can a participant ignore this item? The default assumption is that a participant doesn't need to respond to any particular item in your survey. If they need to reply to an item then put a no in this column. score:[your score name] What is the scale or subscale you want this questionnaire to contribute to To get a score for a questionnaire, you need to have a column header that is formatted with “score:[name of your scale or subscale here]”. Inside this column you can add a “1” or “r1” for each question you want to contribute to this scale (see answers and values above”). 1 will add the question to the score as described in the answers and values columns and “r1” will do the same but with the values reversed. shuffle Randomise the order of items In this column, if you want to randomise the order of questions, you can either simply write 'on' in the items you want randomised, or you can use a different words for different groups of items you want randomised together. For example, if you wanted the first and second half of your items shuffled, you could write 'first' in the shuffle column for each of the first half of the items, and 'second' for each of the second half of the items. The order of items is preserved otherwise, so any items without a word in the shuffle column will not be shuffled. If you write 'off' in this column, the item will not be shuffled. side_text Text on either end of a likert scale What does each extreme of the likert scale mean? For example, a scale of 1 to 5 might range from 'strongly disagree' to 'strongly agree'. In this case, you would write 'strongly disagree|strongly agree' side_by_side Likert questions only - will the question be above or to the side of the response buttons By default, this is set to “no”, and your question will be in a row above the response buttons. However, if you set this to “yes”, then the responses will be presented in the same row as the question. The default survey autism quotient is an example of how it looks by default, but the info sheet has an example of what “yes” looks like. text Question text This will be presented on the left side of the survey. This is useful if you have a question with responses for this item. If you want to have no text for this row of the survey, create a column no_text in your sheet, and turn it on. If you want the text to take up the whole row, then under the type header, choose 'instruct' type There are a lot of different types of survey question you can have. If you click on any cell in the “type” column you should see a summary of each type of question: checkbox If you want the participant to be able to select multiple options, use a checkbox. If you only want them to be able to select one option, use a 'Radio', date This allows standardised storage of date. Note that this works better in Chrome than firefox. dropdown A standard dropdown list. instruct Allows you to type in instructions. likert Participants can click on one of the options that they want. number Only allows participants to type in a number para Gives the participant a large text box to write in their answer. ", radio Allows the participant to only select one option in a list (unlike checkboxes) shortAnswer Gives participant space for a small paragraph for their response slider The participant can slide between values you specify text Gives the participant a text box to write in their answer values What each answer is worth Values work in parallel to answers, in which you associate a score for each answer. For example, if the answers the participant could choose were 'strongly agree|agree|disagree|strongly disagree' and the values were '3|2|1|0', then the value of strongly agree would be 3, agree would be 2, disagree would be 1, and strongly disagree would be 0. These value of the participant's response is stored by itself in the data, and can also be used when adding together the score across multiple items. To get the score across multiple items, create a column with ‘score:[your score name]’. Once you've created this column, click on any cell in that column to get more guidance. Including a survey into an project To include a survey, you need to add a “survey” column to an project: If you have the helper on, when you are editing or selecting a cell within the new survey column, you should see a shortlist of your available surveys: Note that there are some default surveys, and also some user surveys if you’ve created any surveys yourself. Make sure to type the survey you want with “.csv” into the procedure sheet to make sure Collector knows which survey you’re referring to. Data If you are one of the institutions supported by Scientific-Open-Solutions, then you can skip the “Saving data with google drive” step to store your data. The institutions currently supported are: - University of Reading (UK and its Malaysia campus) - University of DeMontford (UK) - University of California: Los Angeles Setting up a server for Saving and emailing data If you are part of our network (e.g. University of Reading, University of California: Los Angeles, De Montfort University, Swansea University) then you can already use servers at ‘ocollector.org’ and ‘open-collector.org’ for managing your data. If you would like to join our network then please contact us at [email protected] and we’ll discuss how to proceed. If you are not a member of the network or would like to set up another server as a backup, the following instructions will tell you how to create your own server. You will need to have CPanel hosting for this, which your organisation may already provide (so ask your IT services) or you may need to purchase. This can be inexpensive - for example we use ecowebhosting (https://www.ecowebhosting.co.uk/). Once you have CPanel hosting set up, follow these steps: 1) Download the collector-server repository (https://github.com/some-open-solutions/collector-server) and unzip the files to somewhere convenient if you’ve downloaded the repository as a zip file. 2) Open the Files Manager - should look something like: 3) Then open the folder public_html 4) Then upload all the files from the collector-server repository to the current folder (not as a zip, if you. It should now look something like: 5) You’ll want to restrict who can use this server. We do this by changing the “ValidExtension.csv” to only allow certain e-mail extensions. So let’s open this now: Ugly, isn’t it? You can keep editing this as a text file, but some of you might (understandably) prefer editing it as an spreadsheet on your computer using software like excel, and reuploading the updated version. I’m going to edit it as a text file for simplicity. If you only wanted people from the University of Reading to be able to collect data with this server, you would add “@reading.ac.uk” to the second row. Your file should look like this: Each comma says that you’re starting a new cell. This means we’re ignoring the “data_gb”, “script” and “admin” columns for now. If you were to open this file in excel, it would look like: 6) Now you’ve set who you want to be able to register projects, let’s set up the rest of the server. The next thing to do is set-up the mysql databases. Go back to the main page of your CPanel, and select the “MySQLDatabases” button: 7) Decide on a name for your database (this isn’t too important) and click “Create Database” 8) Go back to the main CPanel page, but this time go to phpMyAdmin interface: and then click on “Log in” for the database you’ve just created. 9) You can now simply import the “collector_database” file that’s part of the “collector-server” repository that you downloaded. Click on “import”: 10) And then click on the “Choose File” button above, select the “collector_database.sql” file and click “Go” Telling Collector where your saving/emailing script is If you have set up a script for saving your data as described above in Saving data with google drive or Saving and emailing data with a server, then you need to tell Collector where to find these scripts. To do this, click on the Collector Icon in the top-right corner of the screen to get the following: Click on the Add server button to add your script. You’ll first be asked for a script to save the participant data, where you put the script described in Saving data with google drive or Saving and emailing data with a server. You will then be asked if you want to add a script for registration. This will only be relevant if you completed the steps described in Saving and emailing data with a server. Either add in a script for users to register with, or Decrypting your data (and understanding it) I’m going to include some clarifications about actually running your task in this section, so feel free to skip to decrypting your data within the “Data” tab if you already have been e-mailed or downloaded data. If you’re working through the stroop task exercise, you’ll see something like the following when you run your project: I’m just going to run the project as “example_p”. Assuming everything went okay, you’ll get through the task until you reach: After a while, the data should be saved, and you’ll see: It seems that the lag between completing an project and the researcher’s e-mail acknowledging it vary quite a lot. However, if your participant sees the second screen with the map the data is on the way. If they get stuck on: ...for 5 or more minutes then something has gone wrong. You may need to ask them to press CTRL-S to save the data so that they can email it to you directly if this occurs. Decrypting your data within the “Data” tab Once your data has arrived in your email (you might need to check your spam folder for the first few participants), download it, and then go to the data tab at the top of the Collector interface: Once there you can click on the “Decrypt files” button, select your file in your downloads folder (or wherever you saved it). You’ll need the password you created originally to decrypt it. Once you’ve typed it in you should be presented with your file! Understanding your data If you’ve been working through the stroop example, your data should look something like this: You can see in column s “keyboard_response” that I pressed g then b then g. This seems to be correct as “b” represents me thinking the letters were blue, and “g” represents me thinking the letters were green (which matches column R “letter_color”). Had I clicked on the fixation cross or the word, then column t “clicked_element” would have included the words “fixation” or “cue”. FAQ/General questions I need to write an ethics application, do you have some template text for how the data storage is handled? This is some text that has been used in a successful ethics application relating to data storage. Please adapt it to your particular usage: As this study will be online, there could be issues about data security. Collector encrypts and e-mails data directly to the researcher, who then needs to use their password to decrypt this data and download it directly onto their own personal computer. This process protects the data from being accessed by anyone other than the researcher. The servers encrypting and emailing the data are based in the UK, consistent with GDPR regulations requiring participant data to be stored in the EU. The encrypted data will also be backed up on European Servers. An important consideration in your ethics application is if you can minimise the collection of personally identifiable information in the first place. For example, can you use your institution’s infrastructure to collect sensitive personal information using their survey software, and thus only use Collector for anonymised data? How do I make a copy of an project? To make a copy of an project, you can simply download the project using the download button, and then upload it using the upload button: Can I use Collector if I am not a programmer? YES! This program was designed to allow researchers to work with a format they're very familiar with (spreadsheets) to create interactive projects. As you will see in the tutorial Collector is almost a completely programming free solution. Of course there are times when being able to code will make your life easier but we have tried to minimize user coding wherever possible. What language is Collector written in? Collector is written mostly in javascript/Jquery. Formatting of the presentations is controlled mostly by HTML5 and CSS2. Do I need to ask anyone if I want to use Collector? (short answer: no) No. Collector is distributed under the GNU GPLv3 license (full text can be found in the Web/Admin folder). You are free to use, modify, and distribute this program as you wish. If you distribute a modified version of this program you are required to make your modified version available to the public under the same GNU GPLv3 license of the original program. How do I cite Collector? If you are publishing papers that have used Collector projects we ask that you cite the following paper: https://psyarxiv.com/u3saf/. If you build trial types that you think others would find useful please go to https://www.collectalk.com to share. Troubleshooting My changes aren't being reflected in the project. Why? 1. Save all your files before running the project. 90% of the time when you are having this problem it is because you forgot to save the changes you've made. Seems silly but I run into this all the time. 2. Clear your cache. There are extensions for chrome (http://goo.gl/r961j) and firefox (http://goo.gl/r3Zky) that make this process very quick 3. Are you sure you're editing the right project/folder? (this is for if you edit the project files directly, not within the Collector.py interface) Further notes I’m a participant, how is my data encrypted? A public key is used to encrypt data, whereas a private key is used to decrypt data. The researcher will have created a public key that is paired with a private key. This public key can be seen by anyone, including you, and is used to encrypt your data on your device. This means that no-one can steal your data while you are sending it, because it’s already encrypted before you send the data anywhere. The researcher has used a password that only they (and perhaps other researchers on the project) know to encrypt their private key. This means that only people who know the password will be able to use the private key, and this is how the researcher is the only person who can decrypt your data. What happens with my password for encryption and decryption Upon creating a Collector folder in your dropbox account, a public and private key are generated using tweet nacl library by dchest available at https://cdnjs.cloudflare.com/ajax/libs/tweetnacl/1.0.1/nacl.min.js . The public key is used for encrypting data, the private key is used for decrypting data. Your private key is encrypted using Crpyto-Js available at https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.9-1/crypto-js.min.js This means that your private key can only be used when you type in your password, and thus the data can only be decrypted by someone who knows your password. What happens with my password for Collector When creating an account on Collector you are asked for a password. This password is hashed using a randomly generated salt and pepper at the start and end of your password using PhP: $salt = create_random_code(20); $pepper = create_random_code(20); $prehashed_code = create_random_code(20); $hashed_password = password_hash($salt.$user_password.$pepper, PASSWORD_BCRYPT); $hashed_code = password_hash($prehashed_code, PASSWORD_BCRYPT); This means that we don’t store your password in bare form on the server. Next time you log in your password is compared to the hashed version, and if they match then we accept it is you, and keep you logged in for 30 minutes. If you don’t do anything for 30 minutes then we’ll ask you to log in again. In case you’re curious - here’s how we generate random codes: function create_random_code($length){ $characters = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'; $charactersLength = strlen($characters); $new_code= ''; for ($i = 0; $i < $length; $i++) { $new_code .= $characters[rand(0, $charactersLength - 1)]; } return $new_code; } Advice/Tips Collaborating Collaborating with dropbox version If you are working with other researchers, or plan to work with other researchers I would strongly suggest you create a new gmail and dropbox account for your collaboration. This is because you don’t want to have to change your passwords, which could mess with previous projects you’ve run, such as the decryption of the encrypted data. You also presumably don’t want to share the password(s) you came up with before deciding to collaborate with other people. Note that you will want to have clear communication between you to make sure that you don’t overwrite each other’s edits by working at the same time. Collaborating with github/installed version In some ways, this is simpler, just tell your collaborator to clone the repository for your project. Note that you will want to have clear communication between you to make sure that you don’t create conflicting commits by working at the same time. Contributing Any contributions to Collector would be very welcome! The idea when it was initially registered as open-source software was to make it easy to do online research for free if possible, or as cheaply as possible. There are three key ways to contribute: - Highlight any problems you find with collector or its documentation on https://collectalk.com - Add your own phasetypes (and exemplar projects) on https://collectalk.com - Contribute to the code for Collector itself! A good starting point is to fork the repository at https://github.com/scientific-open-solutions/collector-dev and drop me an e-mail ([email protected]). Original contributors Mikey Garcia Collector was initially created by Mikey Garcia as software that should allow researchers to conduct online studies for free. The below is taken from the original repository he created: Tyson Kerr "Many of the best ideas/solutions in Collector have come from the mind of Tyson. Tyson's contributions are so wide that it is hard to think of a piece of the code he hasn't been involved with at this point. Despite his broad contributions, I think Tyson would agree with me that his real baby is the getdata functionality in /Data/. Every time you use those slick menus to check participant completion, exclude flagged users, or download all your precious data into one clean sheet you have Tyson to thank." Victor Sungkhasettee Figured out how to implement the Audio trial type Adam Blake "Completely reorganized the collector.js code to be object oriented. Is responsible for the current look of Collector because he redid nearly all of the CSS to make it much prettier than I initially could." Nate Kornell "Without Nate there would be no Collector. Many years ago Nate taught me how to use the tool he had created for himself, Generic, and that code inspired me to write Collector. Most of the core ideas and design decision at the heart of this project are either directly lifted from Nate's program or were based on adaptations of what he had created."
Hi there
Ho there