Collaborative Virtual Museum for All Senses (CoViMAS) is an extended virtual museum which engages all senses of visitors. It is a substantial upgrade and expansion of our award-winning Glamhack 2018 project “Walking around the Globe” http://make.opendata.ch/wiki/project:virtual_3d_exhibition which had the DBIS Group from the University of Basel team up with the ETH Library to introduce a prototype of an exhibition in Virtual Reality.

CoViMAS aims to provide a collaborative environment for multiple visitors in the virtual museum. This feature allows them to have a shared experience through different virtual reality devices.

Additionally, CoViMAS enriches the user experience in virtual space by providing physical objects which can be manipulated by the user in virtual space. Thanks to the mix'n'hack organizers and FabLab (https://fablab-sion.ch/), the user will be able to touch postcards, view them closely, and feel their texture.

To add the modern touch to the older pictures in the provided data, we add colorized images alongside the existing ones, to have a more lively look into the past using the pictures in the Virtual Museum.

CoViMAS joins forces of different disciplines to form a group which contains Maker, content provider, developer(s), communicator, designer and user experience expert. Having different backgrounds and expertise made a great opportunity to explore different ideas and opportunities to develop the horizons of the project.

Two vital components of this project is Virtual Reality headset and Datasets which are going to be used. HTC Vive Pro VR headsets are converted to wireless mode after our last experiment which prove the freedom of movement without wires attached to the user, increases the user experience and feasibility of usage.

Our team content provider and designer spent invaluable amount of time to search for the representative postcards and audio which can be integrated in the virtual space and have the potential to improve the virtual reality experience by adding extra senses. This includes selecting postcards which can be touched and seen in virtual and non-virtual reality. Additionally, to improve the experience, and idea of hearing a sound which is related to the picture being viewed popped up. This audio should have a correlation with the picture being viewed and recreate the sensation of the picture environment for the user in virtual world.

To integrate the modern methods of Image manipulation through artificial Intelligence, we tried using Deep Learning method to colorize the gray-scale images of the “otografien aus dem Wallis von Charles Rieder”. The colored images allow the visitor get a more sensible feeling of the pictures he/she is viewing. The initial implementation of the algorithm showed the challenges we face, for example the faded parts of the pictures or scratched images could not very well be colorized.

Although the VR exhibition is taken from our previous participation in Glamhack2018, the exhibition needed to be adjusted to the new content. We have designed the rooms to showcase the dataset “Postkarten aus dem Wallis (1890-1950)”. at this point, the selected postcards to be enriched with additional senses are sent to the Fablab, to create a haptic card and a feather pallet which is needed to be used alongside one postcard which represent a goose.

the fabricated elements of our exhibition get attached to a tracker which can be seen through VR glasses and this allows the user to be aware of location of the object, and sense it.

The Colorization improved through the day, by some alteration in training setup and the parameters used to tune the images. The results at this stage are relatively good.

And the VR exhibition hall is adjusted to be automatically load images from postcards and also the colorized images alongside their original color.

And late night, when finalizing the works for the next day, most of our stickers have changed status from “Implementation” phase to “Done” Phase!

CoViMAS is getting to the final stage in the last day. The Room design is finished and the location of the images on the wall are determined. The tracker location is updated in the VR environment to represent the real location of the object. With this improvement the postcard can be touched while being viewed simultaneously.

• Fotografien aus dem Wallis von Charles Rieder https://opendata.swiss/dataset/photographs-of-valais-by-charles-rieder
• Postkarten aus dem Wallis (1890-1950) https://opendata.swiss/dataset/postcards-from-valais-1890-1950

• Mahnaz Amiri Parian, PhD Student @ Databases and Information Systems Group
• Silvan Heller, PhD Student @ Databases and Information Systems Group
• Florian Spiess, MsC @ Computer Science Uni Basel
• Fabian
• Stef
• Florence