Remote Sensing: VR for environmental sciences education

Together with Remote Sensing researcher and lecturer at Leiden University, Joris Timmermans, we explored the potential virtual reality for teaching and learning about remote sensing techniques. This project resulted in the virtual journey Remote Sensing in which students of the Institute of Environmental Sciences at Leiden University now can explore Remote Sensing technologies in a whole new and understandable way. For this project, Joris Timmermans was rewarded with the STANS award in the category ‘Best Outreach’.

Educational Challenge

Students often find it difficult to comprehend abstract concepts of novel remote sensing techniques, especially when they then need to apply remote sensing in real world practise. Organisational aspects limit the possibilities of enabling students to practise conducting measurements using remote sensing techniques in the actual field themselves, as remote sensing materials such as drones and aircrafts, are expensive and not always at hand. Understanding these techniques and knowing their potential is however essential for environmental science students’ future field work. Therefore, the challenge of this project was to find out if we could use 360° VR to provide students with an experience that helps them understand remote sensing technologies and their potential benefits.

A journey through the forest

In the experience we developed using 360° video, the viewer travels through the forest led by a voice over introducing the environment and explaining how and why different remote sensing techniques are applied. When the topic of scanning an environment to create a conceptual 3D model of the area is addressed, users actually get the chance to scan the environment themselves. Users can interactively reveal a point cloud model to see how scanned data can be used to reconstruct a scene.

The final result of this project is a virtual reality application built for Oculus Go using Unity3D. This application can be used to effectively communicate the impact of remote sensing techniques.

Design Reflection

Together with Dr.ir. Joris Timmermans, remote Sensing Researcher at the Institute of Environmental Sciences at Leiden University, and his research assistants we started the design process. As his assistants were all (formerly) part of the intended group of end-users, we consciously chose to co-create the story with them to make sure to include the students’ needs and wants in the best possible way.

To bridge the gap between theory and real world experiences, we decided to base the story of the experience on an existing research site: Het Speulderbos. In this forest, ongoing research captures data using remote sensing techniques. The existence of a measuring tower allowed us to capture different points of view; the forest from the ground, in the canopy level and at the top of the canopy. In this way, we were able to make students aware of the level of detail and type of information observable at each level.

Using specific types of remote sensing techniques, one can create images to monitoring biodiversity by not only showing the ‘real’ world, but adding a multispectral layer of infrared or thermal view so that we can experience of nature and technology, which goes beyond the capabilities of the human eye. To include this aspect in the story, we intended to overlay the 360° video with computer generated graphics. It turned out to be quite a challenge to achieve that in a convincing but still aesthetic way. This is an interesting topic for future experimentation.

During development of the scene where users interactively reveal the forest’s point cloud model, we experienced many difficulties in creating a point cloud from the actual scanning data of Het Speulderbos. While correct data was available and we took some time to experiment with different technical implementations, we were not able to find the correct balance between proper display and high performance rendering on the target device, the Oculus Go. This made us decide to emulate a point cloud model ourselves, and basically ‘draw’ points in the air on the surface of invisible 3D models of trees. From a technical point of view, it would have been interesting to use actual data of the forest, however, we believe that the current implementation is successful when thinking about the intended goal: giving users the experience of scanning a forest and creating point cloud model themselves.

As a final note, we would like to point out that we experimented quite a bit with different voice over styles. We tried out varying content as well as tone of voice in order to make the story compelling and break away from lecture-style presentation. In future projects, we would like to continue with this experimentation to find the right voice over approach for the intended purpose.

Reactions and Implementation

Remote Sensing gives users a unique insight in the forest and remote sensing techniques. While the application is primarily built to be used in environmental sciences education, we believe it is also an interesting experience for anyone with an interest in biology or environmental science related topics. The application has been presented during a presentation during EGU 2019 (the European Geosciences Union conference). Testing of learning effects is piloted in the curricula of environmental studies at the Law Faculty, as a start in the MSc course ‘Human Impacts on Biodiversity’. The first responses of students to the added value of this new learning technology are positive. Read more about the student reactions and effectiveness in the article This is how Virtual Reality brings Education to the next level.