Skip to content

Tutorial 2. Getting started with LSL multiple streams

Jump to bottom
Chadwick Boulay edited this page May 10, 2021 · 3 revisions

In this tutorial, you learn to record two streams simultaneously, one data stream (Audio), and one marker stream (key presses).

For this tutorial you need:

  • Start LabRecorder binary (LabRecorder.exe)
  • Start AudioCapture binary (AudioCapture.exe)
  • Start Keyboard.exe

Now you conduct a little experiment in which you record button presses and microphone input.

  • Start a new recording in the LabRecorder: Click 'Start'

  • Repeat the following steps for all letters on the middle row of the keyboard (from ‘a’ to ‘l’, on a QWERTY keyboard)

    1. Press down a key (e.g. 'A') on the keyboard
    2. Pronounce the letter (e.g. say 'a')
    3. Release the key (make sure that you release the key once you are finished pronouncing this letter).
    4. GOTO 1.

  • Stop the recording

Load XDF file into MATLAB

  • To load the data into MATLAB, change to the directory of the MATLAB Importer
    • Load the .xdf file you just recorded: data = load_xdf(‘C:\yourfilepath\’)

The load_xdf function provides several advanced options for time synchronization, support for breaks in the data, as well as other defects (see help load_xdf).

  • You should see a cell array with two entries
   
1×2 cell array

    {1×1 struct}    {1×1 struct}

type data{1} to see the first stream. The info structure contains additional information that describes this stream in more detail (see below), the time series (i.e. the actual audio data, here a mono recording) and the corresponding time stamps (i.e. for each data sample you have the exact time when this sample was acquired).

    info: [1×1 struct]
    segments: [1×1 struct]
    time_series: [1×667233 single]
    time_stamps: [1×667233 double]

Here the first stream is the Audio (NOTE: the order of streams could be different for your experiment). Type data{1}.info to see more details. You see the name and type of the stream, the nominal_srate (i.e. the sampling rate the device is intending to send), the source_id etc.

  struct with fields:

               name: 'AudioCaptureWin'
               type: 'Audio'
      channel_count: '1'
      nominal_srate: '44100'
     channel_format: 'float32'
          source_id: 'DESKTOP-I9R6TCB{0.0.1.00000000}.{9349f85e-b1ac-40d9-a83f-3c3cf9edbd91}'
            version: '1.1000000000000001'
         created_at: '437615.75132321799'
                uid: 'fdb68f1e-c192-486c-a289-871506575d60'
         session_id: 'default'
           hostname: 'DESKTOP-I9R6TCB'
          v4address: [1×1 struct]
        v4data_port: '16573'
     v4service_port: '16579'
          v6address: [1×1 struct]
        v6data_port: '16573'
     v6service_port: '16579'
               desc: [1×1 struct]
      clock_offsets: [1×1 struct]
    first_timestamp: '438262.4887584534'
     last_timestamp: '438277.6188940192'
       sample_count: '667233'
    effective_srate: 4.4099e+04

Type data{2} to see the second stream. The second stream is the keyboard marker stream. In time_series you find which key was used and whether it was pressed or released. In time_stamps you find the corresponding time steps of when the key was pressed or released. This is an irregular marker stream.

    info: [1×1 struct]
    time_series: {1×20 cell}
    time_stamps: [1×20 double]

Type data{2}.info to see more details.

  struct with fields:

               name: 'Keyboard'
               type: 'Markers'
      channel_count: '1'
      nominal_srate: '0'
     channel_format: 'string'
          source_id: 'KeyboardCapture_DESKTOP-I9R6TCB'
            version: '1.1000000000000001'
         created_at: '437621.11354082898'
                uid: 'ea8b74eb-7101-4543-87f3-1337ecabea6b'
         session_id: 'default'
           hostname: 'DESKTOP-I9R6TCB'
          v4address: [1×1 struct]
        v4data_port: '16574'
     v4service_port: '16581'
          v6address: [1×1 struct]
        v6data_port: '16574'
     v6service_port: '16581'
               desc: [1×1 struct]
      clock_offsets: [1×1 struct]
    first_timestamp: '438265.853400029'
     last_timestamp: '438276.141606314'
       sample_count: '20'

Check the first 6 entries of your marker stream by typing data{2}.time_series(1:6) you should see something like:


  1×6 cell array

    {'A pressed'}    {'A released'}    {'S pressed'}    {'S released'}    {'D pressed'}    {'D released'}

To visualize the results you can type:

figure;
plot(data{1}.time_stamps, data{1}.time_series)
hold all
plot(data{2}.time_stamps, zeros(size(data{2}.time_stamps)),'o', 'LineWidth',2)

(Note: the order of streams might be different in your case.)

The audio data is shown in blue. You can nicely see the moments when a letter was pronounced. The orange markers indicate key presses and releases (not differentiated here). The key was pressed before the utterance of the letter and released directly afterwards.

In the following, we make the plot a bit nicer. To get an easier interpretable x-axis we subtract from both time stamp vectors the first value so that the time vector starts at zero. Note, we have to subtract the same value from both so that their respective time axis match (in this case we use the first time stamp of the audio stream). To differentiate key presses and releases more easily we plot them with different symbols. It is save to assume here that the odd entries in our vector correspond to the button presses and the even values to the releases.

figure;
plot(data{1}.time_stamps-data{1}.time_stamps(1), data{1}.time_series)
hold all
plot(data{1}.time_stamps(1:2:end)-data{1}.time_stamps(1), zeros(size(data{2}.time_stamps(1:2:end)))-0.1,'o', 'LineWidth',2)
plot(data{2}.time_stamps(2:2:end)-data{1}.time_stamps(1), zeros(size(data{2}.time_stamps(1:2:end)))+0.1,'v', 'LineWidth',2)
legend({'Audio', 'Key press','Key release'})
xlabel('Time in seconds')

For EEGlab users there is an xdf_import plugin.

  • Start EEGlab
  • You can load the data either from the GUI or the command line
  • Specify in this case that the stream has the type ‘Audio’
  • EEG=pop_loadxdf(filepath,’streamtype’, ‘Audio’)
  • With the option ‘exclude_markerstreams’ you can select to exclude some marker streams if needed
  • You now can use the standard functionality of EEGLab such as data visualization or epoching
Clone this wiki locally