12  Multimedia

learning goals
  • Understand the importance of preloading multimedia assets
  • Implement image, audio, and video stimuli using appropriate jsPsych plugins
  • Embed multimedia content in HTML instructions
  • Troubleshoot common multimedia issues in online experiments

12.1 Introduction to Multimedia in Online Experiments

Multimedia content (e.g., images, audio, and video) opens up many possibilities for psychological research. But here’s the thing: presenting multimedia in web-based experiments is fundamentally different from traditional lab software like E-Prime or PsychoPy. When participants access your experiment through a web browser, multimedia files must be downloaded from your server to their device before they can be displayed. This creates potential timing issues, loading delays, and compatibility problems that could seriously compromise your data quality if not handled properly.

Think about a simple reaction time study where participants respond to visual stimuli as quickly as possible. If your image takes 200ms to load on one trial but 500ms on another, your reaction time measurements become meaningless—the variability from inconsistent loading times would completely swamp any cognitive effects you’re trying to measure. Similarly, if audio files fail to play on certain browsers, you’ll lose data from those participants in ways that might systematically bias your results.

What makes this even worse, in terms of data collection, is that this could happen and we would never know if the image was delayed or by how much. If we wanted to know how fast someone took to respond to an image, our data would become useless because we wouldn’t know when exactly when the image appeared.

The good news is that jsPsych provides excellent tools for managing these challenges. By understanding how to properly preload multimedia assets, choose appropriate plugins, and handle cross-platform compatibility, you can create experiments that maintain the precision required for psychological research while taking advantage of the accessibility and scalability of web-based testing.

12.2 Image Plugins

Let’s start with the most commonly used multimedia plugin: jsPsychImageKeyboardResponse. This plugin displays an image and waits for a keyboard response, making it perfect for tasks where participants need to make quick categorical judgments.

12.2.1 Basic Image Presentation

Here’s a straightforward example of how to present an image and collect a keyboard response:

const stroop_trial = {
 type: jsPsychImageKeyboardResponse,
 stimulus: 'images/stroop_red_word.png',
 choices: ['r', 'g', 'b', 'y'],
 prompt: '<p>Press the first letter of the <strong>color</strong> the word is written in.</p>',
 stimulus_width: 400
};

The key parameters here are pretty intuitive and similar to those we’ve already discussed. For example, choices defines which keys participants can press, and prompt lets you add instructions below the image.

The stimulus parameter points to the location of your image file. It’s important to point it to the proper location so the browser can find it and load it. If the stimulus was simply stroop_red_word.png it would look for the file in the same folder as your experiment files. Like this:

📂 my_experiment
-- 📄 index.html
-- 📄 exp.js
-- 📄 style.css
-- 🖼 stroop_red_word.png
-- 📂 jspsych

In this case, it is images/stroop_red_word.png which indicates that it is inside a folder called image, so it expects it to be inside an additional folder called images:

📂 my_experiment
-- 📄 index.html
-- 📄 exp.js
-- 📄 style.css
-- 📂 images
    -- 🖼 stroop_red_word.png
-- 📂 jspsych

Again, this organization is important, because if it cannot find the image file, it will not load!

There are other parameters available. In the previous example, we also used stimulus_width (stimulus_height is also available). You only need to set one and it will keep the original aspect ratio to automatically adjust the other.

We won’t review all of the other parameters available, however I suggest reviewing the documentation on your own.

12.2.2 Other Image Response Types

While keyboard responses work great for most experiments, sometimes you need different response methods. jsPsych provides two main alternatives:

Button Responses (jsPsychImageButtonResponse): Perfect for rating scales, multiple choice questions, or when you want a more intuitive interface. Participants click on labeled buttons instead of remembering key mappings.

Slider Responses (jsPsychImageSliderResponse): Ideal for continuous ratings like attractiveness, confidence, or emotional intensity. Instead of forcing responses into discrete categories, participants can position a slider along a continuum.

Both of these plugins work very similarly to the keyboard version and they just change how responses are collected. You can find detailed examples and parameter lists in the documentation for each: https://www.jspsych.org/v8/plugins/image-button-response/ and https://www.jspsych.org/v8/plugins/image-slider-response/.

12.3 Audio Plugins

Audio stimuli are essential for language research, auditory perception studies, and providing feedback to participants. The jsPsychAudioKeyboardResponse plugin (https://www.jspsych.org/v8/plugins/audio-keyboard-response/) handles audio playback while collecting keyboard responses.

12.3.1 Basic Audio Presentation

Here’s how to present an audio stimulus and collect a response:

<!DOCTYPE html>
<html>
<head>
 <title>Demo</title>
 <!-- jsPsych -->
 <script src="jspsych/jspsych.js"></script>
 <link href="jspsych/jspsych.css" rel="stylesheet" type="text/css" />
 
 <!-- jPsych plugins -->
 <script src="jspsych/plugin-audio-keyboard-response.js"></script>
 <script src="jspsych/plugin-html-keyboard-response.js"></script>
 <script src="jspsych/plugin-preload.js"></script> 
</head>
<body>
 <!-- custom JS -->
 <script src="exp.js"></script>
</body>
</html>
const jsPsych = initJsPsych();

const preload = {
 type: jsPsychPreload,
 audio: ['media/speech_green.mp3']
}

const welcome = {
   type: jsPsychHtmlKeyboardResponse,
   stimulus: `
    <div style='text-align: center;'>
      <h2>Welcome to the Auditory Study</h2>
      <p>This experiment includes audio content.</p>
      <p>Please ensure your speakers or headphones are connected.</p>
      <p><strong>Press the spacebar to begin.</strong></p>
    </div>`,
   choices: 'ALL_KEYS', 
   post_trial_gap: 500
}

const word_recognition_trial = {
  type: jsPsychAudioKeyboardResponse,
  stimulus: 'media/speech_green.mp3',
  choices: ['1', '2', '3', '4'],
  prompt: '<p>How many syllables did you hear?</p>',
  response_ends_trial: true,
  trial_ends_after_audio: false,
  response_allowed_while_playing: false
};

jsPsych.run([
   preload,
   welcome,
   word_recognition_trial
]);
Live JsPsych Demo Click inside the demo to activate demo

12.3.2 Handling Browser Audio Policies

Here’s something that may trip you up: modern browsers won’t play audio automatically until the user has interacted with the page. This is a security feature to prevent websites from blasting unwanted sounds, but it can disrupt your experiment if not handled properly.

The solution is simple: make sure participants interact with your experiment before any audio trials begin. In the previous example, I had a “Welcome” screen appear first. This required you to interact with the site before the first trial.

This initial interaction satisfies the browser’s requirement while also giving you a chance to remind participants about audio settings.

12.3.3 Audio Format Compatibility

Different browsers support different audio formats, though MP3 files work almost everywhere. For maximum compatibility, you can provide multiple formats and let the browser pick the first one it supports:

const cross_platform_audio = {
    type: jsPsychAudioKeyboardResponse,
    stimulus: [
        'audio/word.mp3',  // Primary format
        'audio/word.wav'   // Fallback option
    ],
    choices: ['y', 'n']
};

If, for some reason, the participant’s browser does not support the .mp3 format, it will try the next option which is a .wav format. Keep in mind these different files need to be present in your folder in order for this to work!

12.3.4 Other Audio Response Types

Like the image plugins, audio also has button and slider response variants (jsPsychAudioButtonResponse and jsPsychAudioSliderResponse) that work the same way but collect different types of responses.

You can read more about those in the documentation: https://www.jspsych.org/v8/plugins/audio-button-response/ and https://www.jspsych.org/v8/plugins/canvas-slider-response/.

12.4 Video Plugins

The jsPsychVideoKeyboardResponse plugin provides comprehensive video playback control.

12.4.1 Basic Video Presentation

Here’s a basic video trial:

<!DOCTYPE html>
<html>
<head>
 <title>Demo</title>
 <!-- jsPsych -->
 <script src="jspsych/jspsych.js"></script>
 <link href="jspsych/jspsych.css" rel="stylesheet" type="text/css" />
 
 <!-- jPsych plugins -->
 <script src="jspsych/plugin-video-keyboard-response.js"></script>
 <script src="jspsych/plugin-html-keyboard-response.js"></script>
 <script src="jspsych/plugin-preload.js"></script>
</head>
<body>
 <!-- custom JS -->
 <script src="exp.js"></script>
</body>
</html>
const jsPsych = initJsPsych();

const preload = {
 type: jsPsychPreload,
 video: ['media/starfield.mp4']
}

const welcome = {
  type: jsPsychHtmlKeyboardResponse,
  stimulus: `
    <div style='text-align: center;'>
      <h2>Welcome to the Video Study</h2>
      <p>This experiment includes video content.</p>
      <p>Please ensure your speakers or headphones are connected.</p>
      <p><strong>Press the spacebar to begin.</strong></p>
    </div>`,
  choices: 'ALL_KEYS', 
  post_trial_gap: 500
}

const video_trial = {
  type: jsPsychVideoKeyboardResponse,
  stimulus: ['media/starfield.mp4'],
  choices: ['r', 'g', 'b'],
  prompt: `<p>What is the dominant color? (R = red, G = green B = blue).</p>`,
  width: 320,
  height: 240,
  autoplay: true,
  controls: false, // Hide video controls from participants
};

jsPsych.run([
 preload,
 welcome,
 video_trial
]);
Live JsPsych Demo Click inside the demo to activate demo

12.4.2 Advanced Video Control

Videos offer more sophisticated timing options than images or audio. You can show specific segments, adjust playback speed, and control exactly when participants can respond:

<!DOCTYPE html>
<html>
<head>
 <title>Demo</title>
 <!-- jsPsych -->
 <script src="jspsych/jspsych.js"></script>
 <link href="jspsych/jspsych.css" rel="stylesheet" type="text/css" />
 
 <!-- jPsych plugins -->
 <script src="jspsych/plugin-video-keyboard-response.js"></script>
 <script src="jspsych/plugin-html-keyboard-response.js"></script>
 <script src="jspsych/plugin-preload.js"></script>
</head>
<body>
 <!-- custom JS -->
 <script src="exp.js"></script>
</body>
</html>
const jsPsych = initJsPsych();

const preload = {
 type: jsPsychPreload,
 video: ['media/starfield_long.mp4']
}

const welcome = {
  type: jsPsychHtmlKeyboardResponse,
  stimulus: `
    <div style='text-align: center;'>
      <h2>Welcome to the Video Study</h2>
      <p>This experiment includes video content.</p>
      <p>Please ensure your speakers or headphones are connected.</p>
      <p><strong>Press the spacebar to begin.</strong></p>
    </div>`,
  choices: 'ALL_KEYS', 
  post_trial_gap: 500
}

const video_trial = {
  type: jsPsychVideoKeyboardResponse,
  stimulus: ['media/starfield_long.mp4'],
  choices: ['r', 'g', 'b'],
  prompt: `<p>What is the dominant color? (R = red, G = green B = blue).</p>`,
  width: 320,
  height: 240,
  autoplay: true,
  controls: false, // Hide video controls from participants
  
  start: 15, // Start 15 seconds into video
  stop: 25,  // Stop at 25 seconds (10-second clip)
  rate: 1.2, // Play at 1.2x speed

  response_allowed_while_playing: true,
  response_ends_trial: true
};

jsPsych.run([
 preload,
 welcome,
 video_trial
]);
Live JsPsych Demo Click inside the demo to activate demo

12.4.3 Video Optimization

Video files are typically the largest multimedia assets in your experiment. MP4 files with H.264 encoding provide the best browser compatibility, while WebM offers a good open-source alternative. Like with the images plugin, we can provide fallback formats if we’re concerned about browser compatibility:

const optimized_video_trial = {
    type: jsPsychVideoKeyboardResponse,
    sources: [
        'video/interaction.mp4',    // Primary format
        'video/interaction.webm'    // Fallback format
    ],
    choices: ['1', '2', '3', '4', '5']
};

12.4.4 Other Video Response Types

As with other multimedia plugins, video also has button and slider response variants available in the jsPsych documentation.

You can read more about those in the documentation: https://www.jspsych.org/v8/plugins/video-button-response/ and https://www.jspsych.org/v8/plugins/video-slider-response/.

12.5 The Importance of Preloading

Now that you understand how to present multimedia content, let’s talk about why preloading is incredibly important for any experiment using images, audio, or video.

12.5.1 What Happens Without Preloading

Imagine this scenario: you’ve created a beautiful reaction time experiment, but you haven’t preloaded all of your images. Take a look at this example below, that simulates what could happen if you preloaded your first trial image, but did NOT preload the second trial image.

<!DOCTYPE html>
<html>
<head>
 <title>Demo</title>
 <!-- jsPsych -->
 <script src="jspsych/jspsych.js"></script>
 <link href="jspsych/jspsych.css" rel="stylesheet" type="text/css" />
 
 <!-- jPsych plugins -->
 <script src="jspsych/plugin-audio-keyboard-response.js"></script>
 <script src="jspsych/plugin-html-keyboard-response.js"></script>
 <script src="jspsych/plugin-preload.js"></script>
</head>
<body>
 <!-- custom JS -->
 <script src="exp.js"></script>
</body>
</html>
const jsPsych = initJsPsych();

const welcome = {
 type: jsPsychHtmlKeyboardResponse,
 stimulus: `Press any key to begin the first trial!`,
 choices: 'ALL_KEYS', 
 post_trial_gap: 500
}

const preload = {
 type: jsPsychPreload,
 images: ['media/noodle_cropped.jpg']
}

const image_1 = {
 type: jsPsychImageKeyboardResponse,
 stimulus: 'media/noodle_cropped.jpg',
 stimulus_width: 150,
 prompt: `<p>Is this cat happy or sad? Press 'e' for happy and 'i' for sad.</p>`,
 choices: ['e', 'i'],
 post_trial_gap: 500
}

const welcome_2 = {
 type: jsPsychHtmlKeyboardResponse,
 stimulus: `Press any key to begin the second trial!`,
 choices: 'ALL_KEYS', 
 post_trial_gap: 500
}

const image_2 = {
 type: jsPsychImageKeyboardResponse,
 stimulus: 'media/gatsby_cropped.jpg',
 stimulus_width: 150,
 prompt: `<p>Is this cat happy or sad? Press 'e' for happy and 'i' for sad.</p>`,
 choices: ['e', 'i'], 
 post_trial_gap: 500

}
// 3. Run jsPsych with our trials
jsPsych.run([
 preload,
 welcome,
 image_1,
 welcome_2,
 image_2
]);
Live JsPsych Demo Click inside the demo to activate demo

You’ll notice that in the first trial, the image immediately appeared simultaneously with the prompt below it. But in the second trial, the prompt appeared first, then the image appeared. This is not good! You may have also noticed a ‘layout shift’. Because the image wasn’t loaded to take up space, the prompt appeared in a different location and then moved when the image loaded.

When this trial starts, the browser has to download the image from your server. During this download:

  • The trial timing becomes completely unpredictable
  • Participants might see a broken image icon or blank space
  • Response timing measurements start before they can actually see the stimulus
  • The trial might fail entirely if the network is slow

This variability would make your reaction time data essentially meaningless. One participant might have 50ms loading delays while another has 300ms delays, and you’d have no way to separate these technical artifacts from the cognitive processes you’re trying to study.

12.5.2 How Preloading Solves the Problem

Preloading downloads all your multimedia files before any experimental trials begin, storing them in the browser’s memory. When trials later reference these files, they display instantly because they’re already available locally.

The modern way to implement preloading in jsPsych uses the jsPsychPreload plugin with automatic detection. This plugin works just like any other plugin, in that we have to load it in the <head> and place it into our experiment timeline. Wherever we place it is when it will occur.

jsPsych can do some automatic preloading for us. In this first example, I’ll call the preload trial right at the beginning of the experiment, so that everything loads before the participant sees anything.

const preload = {
    type: jsPsychPreload,
    auto_preload: true
};

// These files will be automatically detected and preloaded
const image_trial = {
    type: jsPsychImageKeyboardResponse,
    stimulus: 'images/face1.jpg',
    choices: ['y', 'n']
};

const audio_trial = {
    type: jsPsychAudioKeyboardResponse,
    stimulus: 'audio/word.mp3',
    choices: ['space']
};

const timeline = [preload, image_trial, audio_trial];

The auto_preload: true setting tells jsPsych to automatically scan your timeline and preload any multimedia files it finds. This is incredibly convenient and works for most standard use cases.

12.5.3 Manual Preloading

However, often we need to manually set our files to preload because we’re using timeline variables in our experiment or embedding our stimulus in HTML (we’ll do this later in the book). For this reason, I almost always just use manual preloading because I want to always avoid missing the preload.

For the manual preload, we provide the filepaths for all of our stimuli that need to be preloaded. Different media require different strategies for preloading, so we need to provide separate lists for our images, audio, and video files:

const manual_preload = {
    type: jsPsychPreload,
    images: ['images/instruction_example.jpg', 'images/feedback_happy.png'],
    audio: ['audio/instructions.mp3', 'audio/beep.wav'],
    video: ['video/demonstration.mp4']
};

// This image is embedded in HTML, so it needs manual preloading
const instructions = {
    type: jsPsychHtmlKeyboardResponse,
    stimulus: `
        <h2>Task Instructions</h2>
        <img src="images/instruction_example.jpg" width="200">
        <p>You will see faces like the one above...</p>
    `,
    choices: [' ']
};

12.5.4 Preloading Options and User Experience

Like all other plugins, there other parameters we can set. The preload plugin offers several options to improve the user experience:

const user_friendly_preload = {
    type: jsPsychPreload,
    auto_preload: true,

    show_progress_bar: true,
    message: 'Loading experiment materials... This may take a moment.',

    max_load_time: 60000, // 60 seconds timeout
    error_message: 'The experiment failed to load. Please refresh and try again.',

    continue_after_error: false // Stop if loading fails
};

The progress bar gives participants feedback about loading progress, the custom message explains what’s happening, and the timeout prevents indefinite waiting if something goes wrong.

12.5.5 Batch Loading for Large Experiments

For experiments with many files, you might want to load content in smaller batches throughout the experiment rather than everything at once:

const block_1 = {
    timeline: [
        {
            type: jsPsychImageKeyboardResponse,
            stimulus: 'img/file1.png'
        },
        {
            type: jsPsychImageKeyboardResponse,
            stimulus: 'img/file2.png'
        }
    ]
}

const block_2 = {
    timeline: [
        {
            type: jsPsychImageKeyboardResponse,
            stimulus: 'img/file3.png'
        },
        {
            type: jsPsychImageKeyboardResponse,
            stimulus: 'img/file4.png'
        }
    ]
}

const preload_1 = {
    type: jsPsychPreload,
    images: ['img/file1.png', 'img/file2.png'] 
}

const preload_2 = {
    type: jsPsychPreload,
    images: [ 'img/file3.png', 'img/file4.png'] 
}

jsPsych.run(
    // add each preload trial onto the timeline before the appropriate trial block
    [preload_1, block_1, preload_2, block_2]
);

This approach can be helpful for very large experiments or when working with participants who have limited bandwidth.

12.6 Embedding Multimedia in HTML Instructions

Sometimes you need to include multimedia content within instruction screens rather than as the primary stimulus. This is different from using dedicated multimedia plugins because the content is embedded within HTML.

To accomplish this, we rely on the HTML and CSS we learned in the earlier chapters. However, we can still preload the multimedia in the same way to prevent lags in loading.

12.6.1 Images

Visual examples can make instructions much clearer. Here is an example of using <img> tag to embed an image in our instructions:

<!DOCTYPE html>
<html>
<head>
 <title>Demo</title>
 <!-- jsPsych -->
 <script src="jspsych/jspsych.js"></script>
 <link href="jspsych/jspsych.css" rel="stylesheet" type="text/css" />
 
 <!-- jPsych plugins -->
 <script src="jspsych/plugin-html-keyboard-response.js"></script>
 <script src="jspsych/plugin-preload.js"></script>
</head>
<body>
 <!-- custom JS -->
 <script src="exp.js"></script>
</body>
</html>
const jsPsych = initJsPsych();

// 2. Define our trials
const preload = {
 type: jsPsychPreload,
 images: ['media/noodle_cropped.jpg']
}

const task_instructions = {
    type: jsPsychHtmlKeyboardResponse,
    stimulus: `
        <div style='max-width: 800px; margin: 0 auto; text-align: center;'>
            <h2>Categorization Task</h2>

            <p>You will see cats and categorize them as either happy or sad:</p>

            <div style='display: flex; justify-content: space-around; margin: 20px 0;'>
                <div>
                    <img src='media/noodle_cropped.jpg' width='100' 
                         style='border: 2px solid #333; border-radius: 5px;'>
                </div>
            </div>

            <p><strong>Press spacebar when ready to begin.</strong></p>
        </div>
    `,
    choices: [' ']
};



// 3. Run jsPsych with our trials
jsPsych.run([
 preload,
 task_instructions
]);
Live JsPsych Demo Click inside the demo to activate demo

12.6.2 Audio

You can also embed audio in the HTML using the <audio> tag (see here for info). This can be useful if you need the participant to test their audio before beginning the experiment or providing examples.

<!DOCTYPE html>
<html>
<head>
 <title>Demo</title>
 <!-- jsPsych -->
 <script src="jspsych/jspsych.js"></script>
 <link href="jspsych/jspsych.css" rel="stylesheet" type="text/css" />
 
 <!-- jPsych plugins -->
 <script src="jspsych/plugin-audio-keyboard-response.js"></script>
 <script src="jspsych/plugin-html-keyboard-response.js"></script>
 <script src="jspsych/plugin-preload.js"></script> 
</head>
<body>
 <!-- custom JS -->
 <script src="exp.js"></script>
</body>
</html>
const jsPsych = initJsPsych();

const preload = {
 type: jsPsychPreload,
 audio: ['media/speech_green.mp3']
}

const welcome = {
   type: jsPsychHtmlKeyboardResponse,
   stimulus: `
    <div style='text-align: center;'>
      <h2>Welcome to the Auditory Study</h2>
      <p>This experiment includes audio content.</p>
      <p>Please ensure your speakers or headphones are connected.</p>
      <p><strong>Press the spacebar to begin.</strong></p>
    </div>`,
   choices: 'ALL_KEYS', 
   post_trial_gap: 500
}

const audio_instructions = {
    type: jsPsychHtmlKeyboardResponse,
    stimulus: `
         <div style='max-width: 600px; text-align: left'>
            <h3>Instructions</h3>
                <p>Listen to each word carefully. Press 'R' for real words, 'N' for nonsense words. Respond as quickly as possible</p>
            <div>
                <p><strong>Listen to an example:</strong></p>
                <audio controls style='width: 100%;'>
                    <source src='media/speech_green.mp3' type='audio/mpeg'>
                    Your browser does not support audio.
                </audio>
            </div>
            <p><strong>Press spacebar when ready.</strong></p>
        </div>
    `,
    choices: [' ']
};

jsPsych.run([
   preload,
   welcome,
   audio_instructions
]);
Live JsPsych Demo Click inside the demo to activate demo

12.6.3 Video

Video demonstrations are great for showing complex procedures or providing examples;

<!DOCTYPE html>
<html>
<head>
 <title>Demo</title>
 <!-- jsPsych -->
 <script src="jspsych/jspsych.js"></script>
 <link href="jspsych/jspsych.css" rel="stylesheet" type="text/css" />
 
 <!-- jPsych plugins -->
 <script src="jspsych/plugin-video-keyboard-response.js"></script>
 <script src="jspsych/plugin-html-keyboard-response.js"></script>
 <script src="jspsych/plugin-preload.js"></script>
</head>
<body>
 <!-- custom JS -->
 <script src="exp.js"></script>
</body>
</html>
const jsPsych = initJsPsych();

const preload = {
 type: jsPsychPreload,
 video: ['media/starfield.mp4']
}

const welcome = {
  type: jsPsychHtmlKeyboardResponse,
  stimulus: `
    <div style='text-align: center;'>
      <h2>Welcome to the Video Study</h2>
      <p>This experiment includes video content.</p>
      <p>Please ensure your speakers or headphones are connected.</p>
      <p><strong>Press the spacebar to begin.</strong></p>
    </div>`,
  choices: 'ALL_KEYS', 
  post_trial_gap: 500
}

const video_instructions = {
    type: jsPsychHtmlKeyboardResponse,
    stimulus: `
        <div style="max-width: 800px;">
            <h2>How to Complete the Task</h2>

            <p>Watch this example:</p>

            <video width="300"controls>
                <source src="media/starfield.mp4" type="video/mp4">
                Your browser does not support video.
            </video>
            <p><strong>Press spacebar to continue.</strong></p>
        </div>
    `,
    choices: [' ']
};

jsPsych.run([
 preload,
 welcome,
 video_instructions
]);
Live JsPsych Demo Click inside the demo to activate demo

12.7 Understanding File Paths

Getting the file paths right is obviously important for our experiments; if we don’t point the browser to the right location, the file will not load. Let’s review how to reference files from different locations using a typical experiment structure.

📂 my_experiment  ← Reference point (where index.html lives)
├── 📄 index.html  ← This file loads your experiment
├── 📄 exp.js      ← This file contains your jsPsych code
├── 📄 style.css
├── 🖼️ stroop_red_word.png
├── 📂 jspsych
│   └── 📄 jspsych.js
├── 📂 images
│   ├── 🖼️ happy_face.jpg
│   └── 🖼️ sad_face.jpg
└── 📂 audio
    └── 🔊 beep.wav

When your JavaScript code tries to load a file, it looks for that file relative to the location of the HTML file that’s running the experiment (typically index.html). This location serves as your reference point for all file paths.

12.7.1 Files in the Same Directory

When files are in the same folder as your index.html file, just use the filename. For example:

📂 my_experiment
├── 📄 index.html
├── 📄 exp.js
├── 📄 style.css
├── 🖼️ stroop_red_word.png
├── 🖼️ stroop_blue_word.png
└── 📂 jspsych

If we wanted to load these image files, we’d simply refer to it by their file name. The browser will assume it is in the same folder as the index.html.

// File in the same directory
const trial = {
  type: jsPsychKeyboardResponsePlugin,
  stimulus: 'stroop_red_word.png'
}

12.7.2 Files in Subfolders

The most common approach is to organize your files into subdirectories (sometimes with complex nested structures). In these cases, we need to provide the folder structure relative to the index.html file. In this example, I have some images in a subdirectory called images, but inside that directory, I’ve organized my face images into angry, sad, and happy directories.

📂 my_experiment  
├── 📄 index.html  
├── 📄 exp.js     
├── 📄 style.css
├── 📂 jspsych
│   └── 📄 jspsych.js
└── 📂 images
    ├── 🖼️ example_1.jpg
    ├── 🖼️ example_1.jpg
    └── 📂 faces 
        ├── 📂 angry
        │   ├── 🖼️angry_face_1.jpg
        │   └── 🖼️angry_face_2.jpg
        ├── 📂 sad
        │   ├── 🖼️sad_face_1.jpg
        │   └── 🖼️sad_face_2.jpg
        └── 📂 happy
            ├── 🖼️happy_face_1.jpg
            └── 🖼️happy_face_2.jpg

To load these files in my jsPsych experiment, I need to provide the subdirectory names along with the file name to show the full file path relative to my index.html file:

// in the images subdirectory
stimulus: "images/example_1.jpg"

// in the faces subdirectories
stimulus: "images/faces/angry/angry_face_1.jpg"
stimulus: "images/faces/sad/sad_face_1.jpg"
stimulus: "images/faces/happy/happy_face_1.jpg"

12.7.3 Files Outside the Main Directory

Sometimes files might be located outside your experiment folder. This is uncommon, but it is possible to load files that are stored outside the directory that contains the index.html file. For example, maybe you have multiple experiments that share the same files:

📂 research_project
├── 🖼️ shared_logo.png
├── 📂 shared_stimuli
│   └── 🖼️ standard_face.jpg
└── 📂 experiment_1  ← Your index.html is here
    ├── 📄 index.html
    ├── 📄 exp.js
    └── 📂 images
        └── 🖼️ local_image.jpg

In this case, we need to use the ../ notation to indicate that something is one level up from our main directory. Again, remembering that our main directory will be where the index.html file is:

// Go up one level (..) to access files outside your main directory
stimulus: '../shared_logo.png'
stimulus: '../shared_stimuli/standard_face.jpg'

Key principles:

  • All paths are relative to where your index.html file is located
  • No slash at the start = relative to current location
  • ../ = go up one directory level
  • Always use forward slashes (/) even on Windows
  • Keep paths simple and consistent throughout your experiment

Best practice: Keep all experimental files within your main experiment directory or its subfolders. This makes your experiment portable and avoids confusing ../ paths that can break when you move your experiment to different locations or servers.

12.8 Summary

Multimedia integration is both powerful and challenging for web-based experiments. The ability to present visual, auditory, and video content opens up new possibilities for creating engaging and ecologically valid experiments. However, success requires careful attention to several key principles.

Preloading is absolutely essential for any experiment using multimedia content. The jsPsychPreload plugin with auto_preload: true handles some cases automatically, but you’ll more often need manual preloading for files embedded in timeline variables or HTML. Always include a progress bar and reasonable timeout limits to manage the user experience.

File format selection significantly impacts both compatibility and performance. Prioritize widely supported formats: JPEG for images, MP3 for audio, and MP4 for video. Provide fallback formats when concerned about compatibility and provide informative error messages to participants. Optimize file sizes to ensure reasonable loading times across different network conditions. To reduce file sizes, resize images/videos to be the size they are being presented. For example, if your image is only being shown at 200x200 pixels, don’t load a 1000x1000px image. Resize it using a photo editor.

Embedding multimedia in HTML instructions can help with participant comprehension and engagement, but remember that these files also need to be preloaded. Combine visual examples, audio narration, and video demonstrations thoughtfully, always providing alternative text descriptions for accessibility.

Troubleshooting multimedia issues: File path problems are the most common issue—use consistent relative paths and check the browser’s developer tools for 404 errors. Format compatibility issues can be avoided by sticking to standard web formats and providing fallbacks when necessary. Performance problems often stem from oversized files or inadequate network conditions, so optimize your content and test across different connection speeds.