My journey with VR in education began, as is so often the way, by accident. I was invited to a demonstration session and, while intrigued, I was far from convinced of the value of VR in a school setting; but I went along anyway to find out what the fuss was all about.
The intro session guided us through some immersive videos and at this point, while surrounded by a virtual forest, I was immediately converted. I could see the potential for music teaching, and my own music technology teaching in particular.
So what are VR and AR?
I’m sure you’re aware of the term virtual reality – but it was interesting to explore how it actually works and how users interact with it. Content is projected onto two lenses that trick the brain into thinking that it’s seeing things in three dimensions. If it’s a photo or a video, this is created by stitching multiple photos together to create a 3D image during playback. Other software is animated in a similar way that combines generated images to create a 3D world. It’s amazing how realistic it can feel.
Augmented reality (AR) is something slightly different. Whereas VR completely immerses the viewer, AR overlays artificially created content over the top of a camera feed of the real world. The simplest version of this is having a web browser projected in front of the user while they sit on the sofa or at a desk. The only difference between this and using a traditional device is that you have a giant helmet on your head and the screen isn’t really there.
While VR and AR have similarities in how they work – they both project artificially created images and software into a 3D environment – they have different potential uses in music education.
Immersive videos
The first activity to try is the use of video and photos. My first experiments were to create some 3D videos to allow students to experience an environment that they would struggle to access otherwise. The first of these were some tutorial videos that I recorded in 3D of one of the college studios. These spaces are open to the students but are practically small to teach in. Previously, the only method for teaching the students how to use the spaces was to divide the groups up into smaller numbers and repeat the sessions until all students had been taught. But with a 3D camera and a suite of VR headsets,it potentially allows the session to occur simultaneously: students can ask questions and hear all the responses, which means that they have less time without me in the classroom while I teach in another space.
The other immersive experience I have trialled is the creation of 3D images of a field trip to Rockfield Studios in Monmouth. Some of the students went, but others weren’t able to go, so to avoid their completely missing out on the experience, I captured some 3D footage to give them an immersive view of the studio.
A virtual tour of Rockfield Studios, Monmouth
This has enormous potential for virtual concerts, to allow students to attend events remotely but still feel like they are there. I have recorded some of the college concerts in 3D and shared these at open days for prospective students to experience the ensemble music-making. In the future, if the live-streamed productionsfrom the Royal Opera House and similar are shot in 3D, students will be able to watch shows that they might otherwise never see.
All of these uses so far have been entirely passive. VR headsets allow users to interact with the virtual world as well. What music software is available for VR headsets?
Music-specific applications
The number of specific music applications available for VR headset is somewhat difficult to identify: there are so many apps that claim to be music-related but are actually games that happen to involve music in some way (smashing virtual rocks in time to music, anyone?). But there are some definitely innovative musical uses of VR available.
It would be appropriate at this stage to highlight the excellent article from Ben Sellers in the August 2024 issue of MT. Ben’s article highlighted the use of VR to assist in the teaching of piano using apps such as PianoVision. For those of you that didn’t read this, the app works in a similar way to Guitar Hero, with melodies presented as blocks floating over the correct key at the right time. However, because the app is using AR and VR, it presents these notes over a real keyboard so the student can play their melodies on a real instrument.
There are similar applications for other instruments, including Guitar Strummer for guitar and Paradiddle for drums. These currently only use VR rather than AR, but I suspect it’s only a matter of time before augmented reality tuition software is created on a wide range of instruments.
The app that has excited me most is from Korg and is called Gadget. This is a VR emulation of a number of the synthesisers, drum machines and sequencers. It allows the user to programme sounds and create electronic music using virtual instruments in 360 degrees. It’s quite unusual to have to turn around physically to interact with software (and I genuinely found myself needing a spinning chair), but it was an engaging way to make music.
I’m not sure how long this would keep someone engaged, but the potential it shows for what the future might hold is more exciting.
Working with VR headsets
VR headsets initially seem quite straightforward: the user puts the headset on and points a controller or looks at the item they want to interact with. However, this is actually much trickier than it sounds. And once this is multiplied by a class-set of VR headsets, it becomes even more complex. Trying to help a class of students access exactly the same video from the files folder and activate it at the same time is complicated – and this problem is exacerbated by the fact that, because it is immersive, the teacher cannot see what the student can see. This is then compounded further if VR is combined with other technology, such as a class set of keyboards!
However, products such as VR Sync and Showtime VR allow a group of headsets to be controlled centrally. The content is uploaded to all the devices simultaneously and triggered centrally by the teacher. This way there is no need for complicated explanations of how to access files: the teacher can trigger everything to all devices simultaneously.
A selection of virtual synths creating a virtual studio
Obviously, issues such as ‘don’t walk into stuff while wearing a headset’ need to be considered; but in addition, working with a VR headset can cause feelings of nausea in some people – so have a sick bucket to hand. Or, more likely, a 2D equivalent of what you’re doing to use without headsets.
The elephant in the room
VR headsets are expensive – there is no way around this. The set at my college was funded by an external source and is also intended to support biology and earth sciences teaching. This is great because it means that when those subjects aren’t using them, other subjects can borrow them. But in exchange for the loan of the expensive equipment, I’m sharing good ideas of how to use the headsets to offer the other subjects something in return.
Final thoughts
These last few paragraphs make it sound like the negatives of using VR headsets far outweigh the benefits – and there are indeed still problems I’m trying to overcome effectively. But I do fully believe in the benefits that VR, and more likely AR, can offer music students.
The opportunity to overlay teaching instructions onto an instrument could really help learning and provide scaffolding as students learn. Further, as the technology develops it could allow students to have musical experiences that otherwise couldn’t be achieved. I have a daydream vision of a student viewing the inside of a famous venue or recording studio while performing, and getting the feeling of what it would be like to perform in that environment without leaving the comfort of the music classroom. I think this is still some way off – but not as far as it has been, thanks to developments in virtual reality.
