Pedagogical strategies for introducing 360° VR in class

Year 7 students at Trinity College are set the task of creating a vision of the school of the future using 360° VR as the medium of communication; these VR visions used to inform conversations in the school community from a student perspective. Before they begin the project, the class does an initial lesson to become familiar with the hardware and is guided through two brainstorming activities by teachers Jessica Simons and Steve Grant. These brainstorming activities are intended to get students thinking about safety in VR and have them identify engaging design feature of a 360° VR experience.

Students are asked to work in their small groups to use their desks which as whiteboards to write down a set of safety pointers for using VR. The teachers then guide a whole class discussion to come up with a set of safety guidelines that are synthesised by the teacher on a whiteboard. Here are some of the student’s safety ideas:

TRinity safety 2

Trinity safety 1

Trinity safety 4

This activity was followed by students experiencing a 360° tour using the headset and then having a whole class discussion about what made the tour engaging and what might have been improved. Here is a video except is from the whole group discussion on 360° design features guided by their teachers: 

These pedagogical strategies for first lessons with VR promote student agency and responsibility for safety and prime their imaginations and critical thinking skills through evaluation a user experience of the technology.

P.S. In case you want to put a face to the teachers in the video, Steve and Jessica are pictured below after the delivery of this lesson. To find out more about them go to the Team page of the VR School website.

Southern Montessori School joins the VR School Study

Southern Montessori Middle School is excited to launch our VR project. Southern Montessori’s VR project is part of an integrated Humanities and Science unit based on the inquiry question: ‘How can we secure food for our future?’ Students will be using VR to create their own biome, identify problems arising from human impact, and find solutions to these problems. Students will be challenged to demonstrate their learning in a creative and engaging manner.

Southern Montessori Middle School is a mixed age year 7-9 community located in the southern suburbs of Adelaide with a strong focus on academics. We combine our thirty six Year 7, 8 and 9 students together and work in small, ability-based groups following the Australian Curriculum but presented with Montessori principles. We are committed to innovative approaches to learning that are not only relevant and engaging, but also prepare our students for their future.

Teachers Siobhan Curran and Toni Maddock have developed this unit of work and series of activities designed for students to not only think critically and creatively about the content, but also think creatively about how VR technology can be used as a tool to assist their learning. Having not used VR in the past, students and teachers alike are excited to take part in this research and to see what the students can achieve.

This post bought to you by teacher and co-researcher Toni Maddock

Against reductionism: VR for education

I recently received an intriguing inquiry asking if there was a standard for measuring the effective use of VR in education? What a thought-provoking question (and I thank my colleague for this because it really got me thinking). It got me thinking that now is the time to disrupt some common assumptions about VR (and XR – eXtended Reality) technology for learning so that we can genuinely work out how to best to use the tech in schools and other formal educational settings.

Reductivist assumptions – reducing the complexity of learning and of learning with VR – are sometimes evident in the field of VR for education. These assumptions will prevent us from understanding the many and varied issues related to designing educational VR applications and implementing these at scale in classrooms, virtual and real. Reductionist assumptions restrict our critical engagement and our ability to imagine possibilities for VR in classrooms. Reductionism is a hasty and lazy intellectual and practical position that seeks to simplify the multi-dimensionality of phenomena (things in the world such as this thing we call ‘learning’). While reductionist accounts of using VR for education can offer comforting and easily digestible ‘answers’ to difficult or intransigent issues, this approach will, overall, act as a roadblock for educators navigating towards use of the technology to realise its creative, cognitive, moral and social potential for humans.

Here are a five reductivist assumptions that need challenging:

Reductivist assumption 1: Learning is recalling facts and figures and VR should facilitate this.

Let’s not reduce the difficult and joyous processes of learning to just recalling facts and figures for a quiz. Sure, declarative knowledge acquisition (recalling facts, figures, data, information – the core stuff of content knowledge) is important. This is why remembering (or recall as educators say) is a foundational cognitive process of Blooms Revised Taxonomy (Figure 1) [1, 2].

Figure 1: Blooms Revised Taxonomy [1]

Blooms

Researchers often focus on the question of whether exposure to a VR experience can increase recall of declarative knowledge (facts and figures) especially compared to having the same content delivered via a different type of media or through a traditional instructional approach. This type of research is important as it provides a measure of content knowledge acquisition (usually in the short term, unless the researcher re-tests participants to see whether the knowledge has been retained). From a research perspective it is reasonably easy to give a pre and post quiz on facts and figures and compare the results (and perhaps even give learners other surveys that measure factors that might mediate declarative knowledge acquisition such as an individual’s self-efficacy, spatial awareness, motivation etc.).

However, we would be doing ourselves a disservice as educators and researchers if the only type of learning we cared about was recall of declarative knowledge. As Bloom’s Revised Taxonomy points out, we want to know if student understand the implications of what they can remember, can apply it to similar or novel situations (transfer), deploy that knowledge as part of critical analysis and evaluation, and use it as part of a process that creates completely novel perspectives and products.

We require more research on designing and using VR, and other XR tech such as augmented reality, to support learning that includes but moves well beyond the bottom layers of Bloom’s taxonomy. In practice this means examining VR products for their ‘baked in’ or implicit assumptions about what learning is – if applications only promote recall of declarative knowledge with some limited understanding, then that is fine, as long as we recognise this as only one (vital but limited) facet of learning.

We might also ask ourselves why we should make an investment in VR hardware and software if declarative knowledge recall is the only learning outcome from an app especially if this can be achieved through other more ubiquitous, cheaper technology and/or traditional classroom pedagogical practice?

Reductivist assumption 2: We just need a killer VR educational app and the pedagogical use case will follow.

Some technologists like to talk about killer apps (the one app to rule them all) and how it will create the ultimate “use case” (meaning the best way to pedagogically use VR even though they don’t use the word pedagogy). There are also educators who like to flip this and say, ‘pedagogy before technology’. Both positions are naive simplifications.

I’ve said it before, and I will continue saying it – Pedagogically, VR is not one thing.

As represented in Figure 3, we can think of VR as a new form of media that can empower learners through consumption of immersive experiences and some apps allow learners to create their own virtual objects and worlds to demonstrate learning. There are also VR apps that simulate total learning environments such as laboratories or clinical settings.

Figure 3. Conceptions of immersive VR for learning [3]

Conceptions of VR diagram Feature Image

VR applications can offer diverse types of learning experiences Consider the varying degree of active learning that students can have in different virtual environments (Table 1).

Table 1. Typology of VR environments by student learning interaction and autonomy [3].

Typology

We have a long way to go to theorise and explore the many different pedagogical uses for VR and which of these are most suitable for classrooms across age levels, subject areas, and for different types of learning objectives. I hope that there will be a smarm of killer apps that can create a buzz in the classroom and that these reflect beautiful, pedagogical diversity.

Equally, we need to be much more critical in interrogating the pedagogical assumptions that underpin conceptions of instruction and learning in VR apps. It’s no use saying ‘pedagogy before technology’ when VR applications (and other forms of Edtech) already have pedagogical assumptions baked in.

Reductivist assumption 3: VR is the curriculum

VR apps will never be the curriculum – they can never replace the complex and diverse ways that teachers interpret, enact and truly differentiate curriculum in their classrooms. Thinking that a killer VR app will arrive that will replace a teacher’s skillful mediation of curriculum to student diversity is a furphy. What teachers need are VR apps, with real classroom case studies attached to them, that can help them imagine possibilities and enhancements as they plan and implement their interpretation of curriculum for their students. We need to explore how teachers design curriculum that weaves VR apps through it to enhance specific types of learning.

The metaphor needs to be weaving into curriculum not replacing it.

Reductivist assumption 4: We need a standard way to assess learning with VR

Assessing learning with VR will be as varied as its pedagogical uses and the learning objectives that flow from these. Learning is not one thing. Blooms Revised Taxonomy provides a window into the multidimensional cognitive aspects of learning and being clear about the learning objectives when selecting applications is vital. As teachers ask yourself these questions:

Are we using a VR application to assist with declarative knowledge acquisition? Or, to allow learners to develop procedural knowledge and skills they can practice in a VR simulation? Do we want applications that provide opportunities for transfer of existing knowledge? Or select VR environments that can, in-situ, foster ‘soft skills’ such as communication, collaboration, and time-management? Does a VR app assist with developing affective or moral learning related to empathy or examining belief systems, for example? Are we looking to provide opportunities for learning that involve verbal and non-verbal communication with others for (inter)cultural understanding and exchange? Or, to provide a virtual forum that gives students an opportunity to meet experts who can share their wisdom in dialogue and action?  Do we want to use VR applications that can fire up the imagination to promote creativity and the exchange of creative processes and products? Or select VR environments that give students access to unique artistic, intellectual, cultural or sporting events?

Just as VR is pedagogically not one thing, its potential nexus with the varied types of learning and learning objectives creates a rich educational tapestry. For each of the types of learning listed above, the teacher would identify or develop assessment criteria with metrics and non-quantifiable means of determining if learning objective/s had been met, and the role of VR in this.

While commercial VR is a young technology in formal educational contexts such as schools, we have reached a point where we need to complicate our conception about learning with the tech including our approach to assessment, not simply it.

Reductivist assumption 5: Hardware choices are technical choices

Hardware choices are difficult. In schools we are talking about investment of precious resources with an evolving yet not established evidence base on pedagogical models and efficacy for learning with VR. Hardware choices are not however only technical choices. The hardware, platform and software that teachers choose will have ethical implications for their schools and classrooms.

This is a space filled with tensions and unknowns when legally and ethically it should be clear to educators, students and their families exactly what data is being collected, harvested in real-time and shared/sold-on by tech companies whose VR hardware, software and integrated platforms are being used in classrooms. Artificial intelligence can automatically harvest vast amounts of highly identifiable biometric data (information about individual bodies such as gaze patterns and pupil dilation, movement, proximity to virtual objects, voice etc). Is this data being collected, for what purposes and with what consent? Camera built into VR headsets can capture the real environment that students are in – what implications does this have for privacy?

Manufacturers of hardware usually put an age limit in their online safety advice, and it would be wise for teachers to check this too before procurement. Educators should also be aware that social VR, while opening the world up to learners also has child protection issues.

Many countries have weak regulation regarding data harvesting and the selling-on of such sensitive data including biometrics, which is usually gathered without us knowing. It is up to teachers to think ahead on these types of ethical issues and make fully informed, justifiable procurement decisions. I know this is a difficult job and puts educators in a quandary, but technical choices in this field are also ethical choices.

FYI – The Voices of VR podcast frequently covers privacy in XR – https://voicesofvr.com/

This post is bought to you by A/Prof Erica Southgate.

References

[1] Vanderbilt University (n.d). Blooms Taxonomy Diagram. Retrieved https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/

[2] Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into practice41(4), 212-218.

[3] Southgate, E. (2020). Virtual reality in curriculum and pedagogy: Evidence from secondary classrooms. Routledge.

Cover photo by Rodion Kutsaiev: https://www.pexels.com/photo/white-and-brown-round-frame-7911758/

The metaverse

Ever since Facebook announced its vision for their metaverse on 28 October 2021, including the company’s name change to Meta, there has been a buzz about what it might mean for the future of the internet and our digital (and real) lives.  

Of course, this announcement was set against the recent warnings from a reputable whistle-blower about the harm the social media company is doing including to children and young people through its algorithms that shape user beliefs and behaviour, and inadequate moderation of harmful content.  

This blog post unpacks the idea of the metaverse, taking into account Facebook’s vision but also extending beyond it, to understand its history and highlight some implications for teachers.

Where does the term metaverse come from?

English teachers – You Are Up!

The term metaverse was coined by Neal Stephenson in his 1992 cyber punk novel Snow Crash. It referred to a computer generated universe.

40651883._SY475_

Snow Crash is a rollicking sci fi read that has fired-up the imagination of those interested in possible technology futures with its fascinating portrayal of the persistent immersive 3D digital world of the metaverse that can be jacked into through a personal headset or public booths that produce a lower grade, glitchy avatar. In fact, the novel popularised the word avatar. It also highlighted the dangers of corporate and government control of knowledge and its infrastructures, dreamt up a devastating hybrid DNA and digital virus, and featured deadly semi-autonomous weapons called ‘rat things’.

An aside: For an earlier version of the metaverse, but this one was called the ‘matrix’, see William Gibson’s (1984) Neuromancer, a dazzling tale about a VR universe inhabited by mastermind AIs that influenced the Matrix film trilogy (soon to be quadrilogy).

What will the metaverse be?

The idea of the metaverse extends beyond Facebook’s (proprietary?) influence and has been described as a spatialised interoperable version of the internet. At the moment no one really knows what the metaverse might be like although there are current smart glasses, persistent VR spaces and gaming sites that provide a window into social, commercial, communication and creative aspects of it. Users will probably connect with the persistent interfaces, spaces and layers of the metaverse using a VR headset or smart glasses or on a screen (or with some type of yet-to-be-invented hardware that can integrate aspects of these). There is also a future vision, and investment into research, for direct human brain-computer interface. The metaverse will be populated with people in avatar form and by AI-powered virtual characters in human and other forms.

Here is a description of what the metaverse might be:

“The metaverse is the idea of a shared digital universe in the cloud created by merging virtual spaces that are physically persistent together with augmented reality (AR) layered over the real world. The metaverse is singular because the concept includes the sum of all virtual and online worlds along with all AR layers enhancing the physical world… Besides games and hangouts, it will include social media platforms, workplace tools, investing resources, online shops and much more. You’ll be able to immerse yourself completely in this spatial internet using virtual reality (VR) technology or just interact with bits of it that are layered over your physical space via AR. Instead of a profile picture, you’ll be represented by a complete digital avatar or persona. You’ll be able to meet up with your friends’ digital personas and wander around visiting virtual places and attending virtual events.” https://history-computer.com/metaverse-the-complete-guide/

For those interested in how Facebook’s metaverse might be designed in stages see this excellent article from Avi Bar-Zeev, veteran developer of and commentator on all things eXtended Reality (XR).

What does the metaverse mean for teachers and students?

1. Be curious but don’t believe the hype: There is a fair bit of publicity around the metaverse, and this will infiltrate the EdTech space – just remember that the metaverse isn’t here yet (at least in a scaled-up interoperable way), and some suggest it may never arrive. So, it’s good to be intrigued without buying into the hype.

2. Keep up with current research on immersive learning: We are still in the early days of building the evidence base for the effectiveness of immersive technologies for learning using headset-mediated VR and augmented reality experienced through glasses or via screen, especially in schools.  Results are promising but ongoing rigorous research is needed so that we can confidently embed immersive learning into school classrooms in ways that make pedagogical sense and align with curriculum across subject areas. Asking questions about the evidence base and keeping up with the research on immersive learning is vital as knowledge about this will allow us to ask the right educational questions as the metaverse evolves.  

3. Get interested in the (dry) but important areas of privacy law, digital legislation and regulation, and AI ethics: The idea of the metaverse only amplifies existing concerns regarding the automated harvesting, sharing and use of data without user consent including biometric data which is about and of the user body (facial recognition, pupil dilation, gaze and movement tracking etc.) and which can be highly identifying. There are many different forms of biometric data and plenty of biometric harvesting tools available and so we need to watch this space carefully. Automated nudging of behaviour and the affective moods of users will be diffused through the metaverse as current visions see this as a place to advertise and sell products to us as well as collect our personal data in ways which will be highly embodied and emotional. The inclusion of cameras in smart glasses and VR headsets adds another layer of complexity to maintenance of privacy. The Internet of Things will seamlessly fuse with the Internet of Bodies creating legal, ethical and social dilemmas for all of us, personally and professionally. Children and young people will be differently impacted at each stage of their physical, cognitive, moral, and social development. The teaching profession needs to ask who will regulate the metaverse, define its standards, and build and control its infrastructure and content, as this should inform decision making on procurement of technology for schools. No teacher wants to bring unethical technology into the classroom and so we need to start understanding and applying ethical frameworks now and into the future as the metaverse merges with aspects of our everyday lives in work, leisure and learning.

4. Empower children and young people to have a say in what the metaverse should be: Look for places in the curriculum where students can investigate and use the technologies related to the metaverse as well as explore public and industry discourse about its ethical and social implications. Such opportunities should expand the boundaries of digital literacy education to take in civics and citizenship, the environmental impacts of technology, ideas about human-machine relationships, and re-formed conceptions of learning, creativity and identity in the new machine age. Some industry doyens, such as the CEO of the child-targeted Roblox gaming platform which has 42 million daily users logins, suggest that children are already in a proto-metaverse and that one day such platforms will be pivotal to a metaverse providing everything from learning, shopping and business communication tools. Schooling systems rarely recognise the digital leisure life of children and youth, and yet industry is watching and factoring this into their plans for the metaverse. It is important that we as educators facilitate children’s critical engagement and agency in this space so that they are not viewed just as consumers or as data points. The voices and visions of children and young people should be integral to shaping a metaverse which upholds human rights including the rights of child.

The post bought to you by A/Prof Erica Southgate who is looking forward to having a snazzy Star Trek Borg avatar in the metaverse.

P.S. For those interested, here is the full Facebook Meta announcement.

Snow Crash novel cover featured in this post is from https://www.amazon.com/Snow-Crash-Neal-Stephenson/dp/0553380958

Happy 5th birthday to the VR School Study

In late August 2021, the VR School study celebrated 5 years of ground-breaking research. Associate Professor Erica Southgate highlights three key findings from research so far:

Research papers and teaching resources are available on the VR School website under the Resources tab and there is a book for those who are interested:

Teachers reflect on 360° VR for language learning

This post reports on Athelstone School teachers’ views on using VRTY, a 360° content creation platform, for learning Italian with primary (elementary) school students. To catch-up on the research go here and here.

Language teachers Jo Romeo and Angelica Cardone provided extensive reflections in video and written form throughout the study. They noted that most students were engaged in the learning task of creating their virtual tour of Italy and incorporate the mandated Italian directional language and greetings. Teachers were particularly pleased to see less technologically confident students gain skills by collaborating with their peers either in pairs to create one virtual world or through peer-to-peer interaction more generally.

Teacher written reflections suggested that throughout the unit of work students were developing the Deeper Learning capacities of effective communication and problem solving through self-directed learning and an academic mindset featuring persistence when confronted with a range of difficulties:

“(The project) has enabled aspects of learning as they (students) have designed and created their own (virtual) worlds without too much teacher input. They have explored the platform on their own and used it to showcase their language and IT skills. Students did their own research on well-known landmarks as well as using their prior knowledge to include in their VR worlds. This has enabled them to learn factual historical information about different Italian landmarks and has also improved their vocabulary on directional language.… Students enjoyed recording their voices for the sound markers (that were embedded in the 360° scenes) and some students also researched how to pronounce particular words. They became independent workers as most of the time they problem solved on their own trying different strategies to see if they worked or didn’t. This displayed determination and commitment to successfully complete their (virtual worlds).”

Throughout the research, teachers learnt about the potential of immersive storytelling for language learning and students learnt about this too, guided by a mix of instructional strategies and creative processes. Instructional strategies included explicit teaching, scaffolding of student independent research and student production of different types of interactive media in Italian and English to be embedded in the scenes of their virtual Italian tour. After students had created several interconnected 360° scenes, teachers encouraged them to make audio files of themselves (sometimes with peers) orally using the directional language central to the curriculum. These voice recordings were then embedded in appropriate places in 360° scenes along with other media students had sourced or created such as photo and text information pop-ups providing historical or cultural facts related to the scene.

Students exhibited joy when experiencing their 360° creations through a VR headset, as the teachers explain:

“Most students reacted (to the immersive experience) with expressions such as ‘This is amazing’, ‘This is so cool!’. They were actually able to experience firsthand by being engaged in their virtual world. … (T)hey were able to interact more with the world they created using the headsets because for them it felt like they were in Italy and experiencing the tour around Italy rather than just seeing it on the screen.”

“The students were excited and eager to view their worlds in VR using the headsets. It was fantastic to see their enthusiasm and wonder at being able to view what they had created on a screen using the VRTY platform into what felt like ‘real life’.”

Longitudinal, deep teacher reflection is a key source of data for the VR School Study. Teacher reflections over time provide important insights in to growth in teacher professional learning, student learning and the success of different pedagogical strategies and curriculum planning approaches when using VR real classrooms.

Cover picture: Our last real-life team selfie before the Covid pandemic hit – Front: A/Prof Erica Southgate; Rear (Left to Right): Athelstone School language teachers Angelica Cardone and Jo Romeo, and Principal (and language teacher) Gyllian Godfrey. The study was funded through the South Australian Department for Education Innovative Language Program Grant.

State of eXtended Reality (XR) and Learning Report

The inaugural Immersive Learning Research Network ‘State of XR & Immersive Learning Outlook Report’ has recently been released. eXtended Reality (XR) is an umbrella term for virtual, augmented and mixed reality technologies and immersive learning is a concept used to cover education via these technologies. Associate Professor Erica Southgate, Lead Researcher on the VR School Study, was one of a hundred international experts consulted as part of the report. She is quoted several times on the pedagogical and ethical implications of using VR in schools. This free report is a must read for educators everywhere and can be downloaded here – https://immersivelrn.org/stateofxr_2021/

An interview about the VR School Study

The VR School Study has featured in an interview published by the Independent Schools Association of NSW (AISNSW). The interview covers areas such as leveraging the learning affordances of VR to develop deeper understanding, problem-solving and creativity with students. You can read the interview here.

Researching VR for education

This post provides a snapshot of some of the ways the VR School Study researches the use of VR in schools, with the framework also applicable to other formal educational contexts. VR School is an ongoing multi-site study that employs a mixed-methodology (qualitative and quantitative) approach to research. The study is premised on a multi-perspectival conceptual of education with and in VR. The diagram below outlines some of the key areas that are explored in the research.

Each of these areas prompts a range of questions about virtual reality for education. The table below highlights some of these questions with associated methods for collecting data that might shed light on them.

AREARESEARCHQUESTIONSMETHOD
PedagogyHow can teachers leverage the signature pedagogies of their subject areas/disciplines to ensure deeper learning through VR for their students?
How can teachers leverage the learning affordances of VR for deeper learning?
What are the pedagogical principles or assumptions the are evident in VR applications?
Classroom observation
Teacher reflection
Surveys
CurriculumHow can VR be woven into a unit of work which includes the normal range of conventional learning activities in a curriculum-aligned way?
Can curriculum objectives be adapted to take advantage of the learning affordances of VR?
Classroom observation
Teacher written and verbal reflection
Document (curriculum) analysis
AssessmentHow can VR be used to develop novel, engaging and authentic types of formative and summative assessment?
How can student peer and self-assessment be built into VR projects?
How can VR be used to develop novel, engaging and authentic types of formative and summative assessment?
What are strengths and limitations of conventional assessment types in understanding learning?
Teacher and student written and verbal reflection
Document (curriculum) analysis
Achievement analysis
Student work sample analysis
Student learningHow can students use VR to demonstrate content mastery, collaboration and communication skills, new conceptual understandings, problem-solving skills, metacognition and an academic mindset?
What is the student experience of learning through and in VR?
How can students move beyond the novel effect of new technology to develop deeper learning?
Surveys
Student work sample analysis
Student and teacher written and verbal reflection
Achievement analysis
Student talk and behavioural analysis
Observation
Teacher professional learningWhat is the teacher experience of learning to use an emerging technology in the classroom?
What types of formal professional learning, expert and peer support do teachers require?
How do teachers learn from each other and students during VR projects?
Teacher written and verbal reflection
Observation
Survey
Ethics and safetyWhat are the ethical, legal, safety and child development issues related to using VR in classrooms?Document analysis
Observation and testing
Surveys and experiments (cross-sectional and longitudinal)
Organisational arrangements and cultureWhat are the technical, practical and organisational enablers and barriers to embedding VR in classrooms in a curriculum-aligned way?
What conditions are required for pedagogical risk taking using an emerging technology?
How does the culture of the school support or impede innovation?
Teacher and student written and verbal reflection
Observation
Survey
Document analysis

While these are only some of the questions and approaches to data collection that the VR School study is exploring across primary and secondary schools and in different subject areas, it is worth noting that there is a commitment to participatory research: That is research with teachers and students, not on them. Elevating the knowledges of teachers and students will be key to understanding where VR fits best in education and in scaling up immersive learning in schools.

Cover image from Pexel.

Immerse + Imagine with Alex Bowles

Could you tell us about your professional background Alex?

I came to VR from a background in film and television production, mostly advertising. I got my start at Industrial Light + Magic (ILM) in their commercial production division. This is also where I got my introduction to a lot of the tools that have become central to VR development, like virtual cameras, 3D modelling, CG animation, and rendering pipelines.

At the time, I was less interested in the on-screen possibilities of digital effects and more interested in the ways that the tools and techniques used for effects work were changing the production process itself. Pre-visualization was especially attractive.

I loved the idea of creating virtual sets that allowed creative and production teams to explore their ideas in this relatively cheap, very elastic, and super forgiving design space. Everything about it was the exact opposite of physical production. To me, it was clear that decoupling design from execution was hugely liberating in creative terms, but also a great way to reduce a project’s risks. That’s a rare combination. The key was having a virtual production that offered enough detail, precision, and accuracy to reliably map decisions made there to the real world.

When did you first get interested in VR and why?

A few years ago I’d been researching a program for the full-dome theaters used in science centers, natural history museums, and planetariums. The story focused on the overlapping histories of astronomy and cartography. It revealed the ways that increasingly accurate models of the cosmos led to increasingly accurate maps of the world, and by extension, what people were capable of doing within the world. The goal was to make mathematics, science, and technology more accessible to middle-school and high-school students by providing them with a clear view of how they all worked together. This type of interdisciplinary understanding can make learning abstract concepts a lot easier. It can also be very hard to come by in educational systems based on strict compartmentalization of subjects. That’s where third-parties like public science institutions come in.

The problem I found was that the places with these immersive displays couldn’t offer the economic support that this particular program required. I also discovered that these institutions shared a somewhat Balkanized culture. This made co-productions especially difficult, at least for an outside producer like myself. When Oculus announced that it was going to be making full immersion available at consumer-electronics prices, I thought wow, here’s a way to get the program developed and distributed properly.

This proved less straightforward than I imagined. As a friend of mine likes to point out, you should never mistake a clear view for a short distance. It turned out that the technology was both more complex and more nascent than I initially guessed. That’s where I realized my background wasn’t serving me well. When you’re used to life in a very mature ecosystem, you take so much for granted that simply doesn’t exist in an ecosystem that’s just getting started. Even so, I was absolutely hooked. There’s a particular kind of engagement that you get in VR that I haven’t experienced anywhere else. And I think the ability to actually develop an ecosystem around that is a once-in-a-lifetime opportunity.

How do you currently use VR?

First and foremost, the best way to create VR is in VR. This is easier said than done, since so many of the tools, from 3D modeling software to game engines, were all developed with flat screens in mind. This makes it very easy to lose touch with the sense of presence in real space that’s key to the best experiences. The more of the process you can bring into an immersive display, the better.

I also use it for some things I never expected. One of them is fitness. I checked out games like Beat Saber and Pistol Whip just because they were getting a lot of buzz, but then I heard about people who were making them a part of regular workout routines, and getting really great results within 3-4 months and wow, yeah, they were right.

One of my favourite uses is bringing global and spatial context to news stories. Google Earth VR is fantastic for this. Simply being able to go to a place, even virtually, has this really remarkable effect on the way you understand stories, and connect them to other stories. This is due to the very different ways our brains encode spatial memories, as opposed to strictly visual memories. Once you’ve got a place-based component to your understanding, you have a much easier time absorbing retaining, and remembering everything else you learn about a topic. It’s amazing.

That phenomena is actually the basis for what I’m working on professionally, which is outreach programs for historical and culturally important sites. By giving people who haven’t visited these places in person the kinds of spatial memories shared by those who have, it becomes possible to expand the community of those who value a given place, who understand why it’s so important, and who are willing to put effort and resources into preservation.

What are your thoughts on VR and the creative process?

Let me start by narrowing that to the creative process used by film and television production, since that’s the one I know best. The biggest thing I’ve discovered is that you have to get your target audience involved a lot earlier than you would for a 2D medium. Partly, this is because immersive media doesn’t offer a century-plus of accumulated knowledge embedded in institutional structures, production tools, and so on. You can’t count on tried and true best practices to get you most of the way to a finished piece because at this stage, best practices are little more than well-educated guesses.

The other reason to make user testing an intrinsic part of the creative process is that audiences have so much more agency in immersive programs. Even game developers—who are used to having everything based on player input—have a ton of unfamiliar dynamics to manage when people are fully present in a scene, rather than interacting only via a controller and their thumbs. The more room you give people to surprise you, the more they will end up surprising you. This can be frustrating if you’re focused on a specific vision, so it’s helpful to remember that some of your most promising creative opportunities can come out of these unanticipated interactions. The sooner you can get a read, the better.

All this gets a lot easier if you recognize that the core of the medium isn’t what people see on a screen. The core is how they physically respond to events in three dimensional space. This is why it helps to get the basic physical interactions that are central to the experience prototyped and tested as early as you can, so you can use the feedback you get to determine the project’s final form.

This prototyping stage doesn’t need to be in VR, by the way. Some of the smartest studios I know rely heavily on brown-boxing, which is literally using cardboard boxes to work out where different elements should be placed in the space around users, and to make sure people can move around comfortably enough to enjoy the experience. Once that’s sorted, you can design the virtual experience on top of this real-world foundation.

What advice would you give teachers and students who are thinking about using VR for creative projects?

The first thing, and maybe most important thing to do is to experience as much of what’s already out there as you can. Paying close attention to what you do and don’t like should leave you with a general sense of direction.

One of the first things you’ll discover is that VR isn’t actually one technology so much as a bunch of different technologies that are being combined in new and interesting ways. These combinations are the source of a lot of excitement, but they can also be the source of a lot of frustration, especially since pipelines and interoperability standards are so nascent. As a friend of mine put it, in game development, everything takes three times longer than you think it will, and in VR, it takes three times longer than that. So do make a point of being kind to yourself. It’s very much a learning-by-doing effort, and you’re going to get it wrong, a lot. And don’t worry about feeling you don’t know everything about everything because you never will. It’s just too complex.

Eventually the field will deal with this complexity the same way that film and television production, and game development have. We’ll end up with armies of highly-specialised craftspeople who are deeply focused on specific details, and equipped with budgets and schedules to match. But at the moment, we’re nowhere close to that, and I suspect it’ll be many years before we are. For now, the field favours generalists. If this is attractive to you, the best thing is to start with something that seems super simple and purely exploratory, get others involved as early as you can, pay close attention to all the different ways they respond, then building the final product on whatever emerges. Keeping your mind open, especially at the outset, is going to be the most likely way to find your north star.

Blog at WordPress.com.

Up ↑