Setting up a reality portal for fun and profit


#1

Hey folks! I just wrote up a little guide to setting up a simple reality portal akin to the MULTIVERSE Initiative pioneered by SVVR… my solution’s a little less sophisticated and requires an external service, but it does work and is pretty easy to set up! Feel free to add comments or questions, if you’re interested.


SVVR’s pioneering Reality Portal at work! Picture courtesy SVVR & VRScout

Reality Portals for Fun & Profit
How to set up a Reality Portal in VR

A reality portal is a two way window between virtual realities and the real world. It allows users in virtual reality to converse and interact with people in the real world, and vice versa. Done correctly, a sense of real human contact can be accomplished. This document describes how reality portals work, and how you can set one up for your own classroom, office, meetup or hangout, or in your home. Reality Portals were pioneered by Karl Krantz and others at SVVR. You can learn more about their more sophisticated MULTIVERSE Initiative program, and their technology on VRScout..

How’s it work?

In a nutshell, you need to stream video and audio between the real world and the metaverse. To do this, you need the equivalent of a screen, a camera, a mic and speakers. On the real-world side, this is real hardware in the form of a laptop or desktop PC with a webcam and a video screen. On the VR side, you set up an avatar to act as the ears and mouth of your real world users , and then set up a web entity as a video screen for folks in VR to see the camera view.

Here’s a super easy, step-by-step process that uses Screenleap to use your mobile phone as the video source for your viewers in VR. Streaming the video from your phone camera makes it easy to move the camera around as needed! Screenleap is a service with free and paid options, and we recommend it because it’s very reliable and easy to set up.

Set up your VR environment for your VR users

  1. Set up or go to a domain where you have edit rights
  2. Add a Web Entity using the create tool
  3. Set the URL in the web entity to the Screenleap URL you choose below.

Set up the computer for real world users

  1. With a laptop or desktop PC, connect your video out to a projector or screen
  2. Hook up a mic and speaker. Most laptops will do a good job. But if you want really good feedback cancellation, SVVR recommends Jabra 510MS.
  3. Choose the mic & speaker in your audio settings
  4. Connect to the domain in step 1 with the PC. You should now be able to talk to and listen to users in the domain

Set up your streaming camera

  1. Easiest solution out of the box is to install the Screenleap app on your phone and use that for your camera. This will provide an easy to use, portable camera solution which will transmit only video to the Web Entity screen in step 2. Get the Screenleap app on your phone’s app stores, here:

https://play.google.com/store/apps/details?id=com.screenleap.android.Screenleap&hl=en_US

  1. Set up a screenleap account if you havent yet. If you are going to have a lot of users in world (like, more than 8) you will want to just buy a basic account from them, otherwise if it is small and you are OK with the limit, use the free version.
  2. Start up the Screenleap app and choose Broadcast Camera. Leave Assign Access Code blank and press Start Broadcast.
  3. Your phone camera video is now being broadcast to web browsers! Just point your browser to the address at the bottom of your screen. It will be something like:

https://screenleap.com/myScreenleapUserName

  1. Put that URL in the Web Entity in Step 2.

Now people in-world can see your mobile phone’s camera! There’s also a screen mode, where you can show the contents of your phone’s screen rather than camera.

Congratulations! You have become a trans-dimensional cosmic entity!

If you’ve got any ideas, questions or comments feel free to add them below.


#2

Thank you so much for this!
I hope I can realize this at the Munich VR MeetUps :purple_heart::sunglasses:


#3

Thank you, Caitlyn! I’m going to try this as soon possible!!


#4

Thanks Caitlyn. What I’d really like is to broadcast my phone camera like this while also viewing a stream of my PC screen on the phone. The PC would be running Hi-Fi (looking out from the web entity) so that way I could take the portal anywhere with just the phone.
But Screenleap pauses the broadcast whenever the app isn’t displaying :confused:


#5

The magic behind the curtain. Thanks for sharing.


#6

BTW if you do use screenleap and a phone, I suggest something like this over your reality portal’s screen, to hold the phone. You needn’t use a current phone with phone service enabled, an older unused device like an old iPhone or Android phone which connects to your venue’s wifi and has a good camera should be able to run the Screenleap app good enough. While I can’t vouch for the quality of this particular gooseneck below, you should be able to get the idea from it:

@Phil in your case you might want to use the Screenleap desktop app. It can stream your whole or partial desktop the exact same way the phone app works. You can then have a video screen on your desktop that streams your camera’s view (using a webcam, or similar) assuming you can watch the video stream on your desktop, and then when you want to share a desktop you can just minimize the application with the video screen and switch around as you see fit. The SVVR Multiverse Initiative rig uses a fixed GoPro camera and does something kind of similar except with their own video streaming configuration rather than Screenleap.


#7

Yeah, Screenleap looks worse for me than SVVR.
I read that they will open source their toolset but couldn’t find it yet. . maybe it’s future plans?


#8

Hope so! Their solution uses WebRTC. I do hope they open source it! It does indeed look great.

If anyone has encountered a similar service which uses WebRTC and looks better, let us know here. I believe some of our users have had success streaming out from VLC and using a restreamer server on a VPS and had better quality results.

Screenleap works in a pinch, though, and most importantly is super easy to set up. It can look better if you stream it from a PC and force it to 420p, making sure your web-entity in world matches the aspect ratio, and if you use a web-entity-overlay rather than a standard web entity you’ll get a higher FPS.


#9

Ping @GeorgeDeac
He did some great WebRTC work with TransmissionGate

Re: web entity overlay… What’s that?


#10

Web Entity Overlays – it’s another way to do a web entity except it’s only drawn locally, on the client. This is how it works with your tablet, btw. So other people aren’t seeing your tablet in world.

Web Entity Overlays render much more quickly in terms of video FPS on playback… regular web entities are currently capped, which makes them choppy. Using an Web Entity Overlay instead can produce a buttery smooth frame rate.


#11

How can I create one?


#12

Regarding WebRTC, on Monday Maker someone shared there screen using an open source WebRTC project I’d not seen before:

https://janus.conf.meetecho.com/index.html

It has examples for screen sharing, streaming, video conferencing etc.

At some point I’m going to be looking into setting up a reality portal using this for work meetings. The idea will be in HiFi I’ll have a meeting room with portal setup at end of meeting table styled like a standard video conferencing system. Then our real world meeting room will be connected to this from it’s physical displays. This will allow better interaction with home and remote staff.


#13

Edit: This version should work… it will rez an web-overlay where the box is, and will cap out at 90 fps :slight_smile:

http://hifi-content.s3.amazonaws.com/caitlyn/dev/movieBox.json