Approximate cost to launch and operate a VR site


#1

Hi,

I intend to start a VR world (site) containing houses and streets, about 5x5 kilometers.
It must support about 100 simultaneous visitors.

I would be grateful to know what are the expected main costs to launch this site and what are the alternatives.
E.g.

  • fees for a VR-location (if such exists)
  • setup and a server with necessary performance (see above).
  • operational costs
  • other

As alternate, is that worth to try do all my self or hire external services e.g. a highfidelity configurate server.
Your advises or proposals are very useful for me.

Thanks!


What is the maximum avatar capacity of a High Fidelity domain?
#2

High Fidelity domains readily support and environment uptown 32km cubed out of box which should work without issues (floating point exceptions not withstanding).

However the requirements for a dedicated 100 visitor domain may require quite a beefy set of servers, and latency.

While they have reached a potential 130+ (stable at 110) users, the technical requirements were quite high: here the full details on it:

https://blog.highfidelity.com/scaling-in-a-single-virtual-reality-space-74e36e2f0ad4 (I see you actually commented on there)

This is suitable when you know an event is comming up and really expect there to be 100+ people. So best option would be to scale up on demand instead to save costs.

As an example, I know that my domain can handle upto 6-8 concurrency with 3 Cores, 1.6ghz, and a 200/20 connection. In that case the up bandwidth is actually the bottleneck.

Currently however there are no -ready- made hosting providers so its quite a bit of -do-it-your-self- until there is the audience and potential business for it. You should ask @Caitlyn if they have can provide such as the folks at highfidelity do have some experience in hosting on their end.


#3

Thank you!

If I understand correct, I could be able to set up my own server or use space on an ordinary web hosting, just pay for enough bandwidth.

As for the visitors, the are intended to come and walk in smaller groups on 2-8 people (often maybe 2 or 3). So from the perspective of each group, there will be 2-5 ohter groups in the field of view of each visitor and the visitor will fokus on people in the own Group.

Do you think that case can simplify dwond the requirements for e.g. bandwidth?

Do you have what the worst/expensive scenario would cost in month expenses, roughly? I just need to make a general assessment on expenses, not sot solve the problem, just ensure that I affect enough budget so I dont run out it.


#4

Hello, Help us stress test High Fidelity again!
it s about a Stress Test in February, with near 100 users.
I mean they have rent a Amazon Sever for this event in time.

I m unsure about the pricing for a Event like this, I bet it s roundabout 30 US Dollar for some hours with the traffic and power-) CPU and RAM.


#5

No. You will require a cloud droplet or some form of root access to the machine (virtual or otherwise) in which you will install the Stack Manager. ((renamed Sandbox))


#6

Thanks!

If am not able to do that myself at home, roughly in your opinion, how much must I expect to pay to someone willing to act as provider?


#7

With all due respect to you; you’re asking a very complicated problem and providing very little details. :smiley: as @Menithal mentioned, there is no current provider of such things.

You did describe a top total of near 100 users, but that those users would be sub-set into groups of 5. Are your groups of 5 going to be distanced enough to prevent the system from sending IK updates from Group B to Group A, etc…? How about physics engine interactions between groups? It becomes a very complex question as you can imagine…

Aside from this, it is going to depend very heavily on your bandwidth needs. If you intend to run this as a commerce space, I would highly ((seriously)) consider looking into how to run Sandbox; and more specifically how to run Sandbox server instances of different types on distinct machines. Once you come to terms with that, finding a hosting provider should be more clear.

If you know that will not be something you are comfortable with; I will repeat as @Menithal says, just go ask High Fi if they’ll set you up with a package. If enough of us do this, maybe it’ll change their business model a bit. Who knows?


#8

Very useful, thakns.

I just asked HiFi about that.

By the way, as you understand, I am just a poor “house owner” having a problem, with no ambitions to be a developer. I ask “the plumber” (Hifi) to give me a solution. I don’t care about how is implemented. Just do my work. If parts of my work are to heavy, then I try to addapt me to the technical performance. This proces may be itterative, like a negociation.

By the way, I am very surprised that there are no providers of VR. Bad sign? Means that the VR is not launched?
Other concurent systemm, though less modern OWL Open Wonder Land), is providing hosting: https://www.cyramix.com/buy


#9

Here is an idea of what Amazon is charging for the on-demand service…

Here’s a walkthrough how to create a windows Hifi host, please PM me if you would like some help setting this up. https://wiki.highfidelity.com/wiki/Amazon_Windows_Stack

Why not Linux? since High fidelity is made for Windows primarily it will be easiest operationally to run the frequent updates and server configurations.

My recommendation… get a server you can afford and then see what happens… if you have to go bigger then its not that hard to increase the capacity.


#10

Thanks!

Some questions.

In my application I count on an average of 30 avatars per hour. But the server must support tops of 100 avatars.

In the example in wiki you specify the Amazon t2.small package for 10 avatars with audio. Some idea which package would be suitable for 30 (max 100) also with audio?

What do you mean with “target”? Is it “goal” “own application requirement” or is it a special HiFi term?
S3?


#11

Amazon services are pretty scalable, several of the packages would probably work. In terms of overall maximum hosting cost, what you could do is take the cost of equivalent resources on SL, 300/mo per sim?

If you have 4 sims in SL is could be compared to Hifi in terms of land mass, at $1200 / month would rent a hell of a server on Amazon.

s3 is the storage service from amazon, much much cheaper than servers instances to store you content,


#12

Uhmm, 4 sims is only 512x512 meters. To compare a 5x5km in high fidelity you need 20x20 sims in secondlife. that’s a scary amount of 400 sims. not going to calculate the monthly cost of that. :grin:


#13

Oh yeah for sure, I was sort of thinking minimum land size ( 4 sims reasonable size) but importantly the resources each sim has for concurrent avatars, I don’t think I was ever on as sim in SL with more than 25 avatars and still had a nice smooth experience.

The total land size is not really important when it gets to High fidelity… but in my mind we should be focusing on smaller land sizes so we can keep the density high. (chance of running into someone or finding some interesting content is better)


#14

All right! So 300 usd/mo would be a realistic target for one sim.
Do I have strong reasons to choose Hifi as a platform or it would be more resonable for me to choose Secondlife?
I wish to spend my time for my vision, not for learning how I program a platform. But I need some flexibility for someone would be able to fix s/w improvements for me.


#15

Yeah 300/mo would get a smokin server, Hifi is different than Second Life.

It’s early here, but it has the most potential I think.

I personally haven’t done anything new for SL in a couple years, although still visit from time to time,

Been focusing on VR / Rift-Vive capabilities in whatever small way my humble skills can in hobbyist way.

That being said, takes years for a small team to build cool experiences in VR, question is where will the platform be at that time?

Hifi being open source means it should be around and hopefully just catching the wave of cheap headsets, others… not so much.


#16

@Petrigeo Hi there,

Using your metaphor of a house owner, Hifi is a manufacturer and supplier, you still need a plumber, we are all doing our own plumbing.

You need to be looking at a dedicated server on a 1Gbps network.
You also will be looking at Windows because Linux is “almost workable” for a linux geek, if you’re not a linux geek you’d be in for a world of hurt trying to build this thing at every release. Windows is reliable at this stage.

I suggest Windows Server 2008 or 2012 preferably with a good size SSD.
My recent experience has shown a minimum requirement of 16Gig ram if you want 6 concurrent avatars with a reasonable amount of content (textures, models, animations), any more avs than that and you might as well go for 32G. You can get 32G for less than $100 a month.

Try to get a flexible plan and start small. Dont expect more than handful of users to be actually logging in at once.
You could get away with 8Gig and that would see 3 - 4 avatars pretty smooth, audio and voice is where you will see the signs of strain.
Heres a couple of outfits I have used with good results.


https://www.chicagovps.net/services/dedicated-servers

I haven’t had experience with Amazon, they have some kind of variable deals and hourly plans I believe.

You’re also going to need a regular webserver to store and deliver the assets, this can be put on the same server but for load balancing its best to be a separate host, which is not using your server bandwidth.

Apart from the monthly hosting/s the only costs are the Hifi “Place Names” which are $20 a year.

Its going to take you hours and days to work all this out.
Ping me if you want some serious help with the Hifi plumbing, I have done this many times before and I am very reasonable, I guess I am the closest thing to a hosting provider that Hifi has got for now.

Cheers @Adrian


#17

I already set my mind to a good 4x4km , that’s a good size to build with, and not to small you run quick out of space.

But 1x1km, it need for some things some thinking but could be fine to


#18

Just as an example - while testing for various tasks on my QA testing list I setup a domain capable of hosting 150 concurrent agents then using scripted agents placed 150 simulated agents in.

I used cloud based servers and if I duplicated that configuration again with intent to run full time it would cost $400 USD/month just for server time and that’s on a cloud provider known to be fairly economical.

If I were serious about doing such a thing I’d actually bump hardware config up to something costing more like $720 USD/month to give some head room for max load situations. Thankfully, I’m not serious about such a thing. Yet. :slight_smile:

It’s a complex thing provisioning a server for high agent count – it’s not only about CPU and RAM capacity – it’s also about network capacity as HiFi consumes tremendous amounts of network capacity at high agent counts.

Here’s what bandwidth looks like for just avatar mixer at 100 agents…

Then CPU load on a 8 CPU droplet, again, 100 agents.

Another high CPU count machine ran audio mixer with similar stats. And yet another droplet with 4 CPUs ran remaining services. While a 16 CPU machine hosted scripts to simulate agents and barely was able to do job… but simulating agents isn’t a concern in this discussion.

It would be nice if we ever get to the on demand processing via open market sell CPU cycles to HiFi cloud we thought would be such a huge part of this project, way back when… Then you could scale up/down in real time, pay only for capacity in use dynamically adjusting for load. Someday… hopefully this will get there someday.


#19

Nice info and charts. A couple of questions: What are the avatar counts at various times? I assume at 2:24 PM is when the full 100 avatars were present. What about between 2:02 PM and 2:24? Were all the simulated avatars started at once, or did they gradually build up minute by minute, and how many simulated avatars were present at about 1:55 PM where the small hump in the charts is?

Also, were the domain servers running on the same service (and possibly the same data center) as the scripted avatar simulator? and do you think there might be a difference in the stats if the domain was at one location and the avatar simulator was 1000 or more miles away? And just wondering, what exactly is being simulated? Full audio, full head and hand controller positions etc.?

And one last question. At the peak it looks like the bandwidth used is about 400 Mbps. So for 100 avatars that is about 4 Mbps per avatar. If I remember correctly that is more than twice as much bandwidth as I’ve seen used per avatar when 5 or 10 avatars are present. Is this correct?

Thanks again for the info and charts. It’s nice to see real data about all this.


#20

Basic setup was all droplets in same data center 1 droplet ran DS, Entity Server, Entity Script Server and Messages – since domain had no content to speak of but for a prim to land test agents on… it showed little load CPU or Network. Another droplet ran Avatar Mixer, another with Audio Mixer and last another running script services.

The lead up to just after 2PM was all provisioning droplet - downloading updates and my linux AC/DS packages… 1:55PM was when I pushed system images to other servers (Audio and Script) and brought them up.

Correct - just after 2PM I told script server to load 100 instances of agent simulator and march to top began.

Simulation included minimal AV movement (enough to insure agent wasn’t seen as not moving at all) - they spoke random phrases. There’s a test script to make simulated agents in repo – its precise name escapes me at moment, but, it pretty much – for CPU/Network use looks like a normal human’s impact.

I’ve always seen server -> client bandwidth as 4.5 - 5.5 Mb/s per client so that was one number that didn’t surprise me.

What I’ve learned to date is… up to about 60-75 agents it’s reasonably straightforward stuff - 100+ gets more interesting. 150 can be made to work, but, things become increasingly critical of dividing services across machines and giving enough resources to get job done. Past 150 there be dragons. Lots and lots of dragons.