Max number of avatars in Sandbox?


#1

Hey All,

I’m still trying to wrap my head around just how High Fidelity works. I learned that I need a service like Digital Ocean if I want to protect my content and have more than a handful of people on my “property”. If I don’t care about content protection, at least in the short run, how many people can be at my Sandbox? I assume the Sandbox app is running on a server somewhere in the world.

Thanks!


When is HiFi expected to be user friendly to the masses?
What is most important when setting up a dedicated sandbox computer?
So, are we able to do self-hosted content yet?
#2

Unless you mean the domain NAMED Sandbox, the sandbox you get is running on your local machine.


#3

If you’re hosting on a Digital Ocean droplet, something you have to obtain/setup/manage then two factors determine how many concurrent visitors you can have. 1) Bandwidth available and 2) CPU available. Informal testing shows a DO droplet capable of serving 40 avatars concurrently before saturating network (though that varies considerably by region you host DO droplet in and its network load for other instances shared on same hypervisor). CPU is a lot more tricky… but you start running into CPU saturation far before 40 avatars on the inexpensive DO single CPU droplets and dual CPU variant.

The glorious promised future of distributed on demand processing power will go a long way to allowing scaling up to much much greater concurrent visitors, when/if it ever comes… it once was a central point of the system architecture, but, hasn’t had much said about it in a long time.

In most simple terms - don’t expect to have much more than 10 concurrent visitors without problems unless you spring for hardware capable of it… you’ll start running into audio issues, movement issues etc etc etc if you saturate CPU.

Someday this will all be easy, but, that remains in The Glorious Future, for now.


#4

Yes, I meant the “Sandbox” this is one of two apps that you get when you download what High Fidelity give you. There other app is “Interface”. No I know for sure to give Digital Ocean a really hard look. Thanks!!!


#5

This question gets asked a lot. It ought to be in the Wiki. The short answer is you need about 2Mb/sec of bandwidth per avatar, but because people visiting your domain (aka sandbox) need to fetch a ton of data to see the scene, you need a big pipe, 50+Mb/s. How big depends on the complexity of the scene and how many people visit your domain anew. So far, I’ve seen many domains required over 100MB of asset transfers when I visit them. Now it is clear that almost none of the content in domains today is optimized, so there will be room for improvement to reduce the asset bandwidth costs, but that’s for the future.

For inexpensive droplets in Digital ocean, figure around 15-20 avatars. For a domain hosted at home on a Comcast network connection, drop that to 7. Note this is based on a lot of simplifying assumptions on typical content in a domain, typical avatar movement - your mileage may vary.


#6

Well, that explains why DO offers plans with more than one CPU. Thanks!!!


#7

Linode has a $20 plan that offers 2 CPS and 250Mbs out for just $20.00 per month. Is that sufficient for 30 or so avatars? I’m just pulling the number 30 out of the air. For $40.00 per month, that doubles the CPUs to 4 and the outgunning to 500 Mbs.

Thanks!!!


#8

From two years of running this stuff - probably not unless you have a very basic content set. Linode specifies its allowed peak bandwidth – so - first look that up and divide by 2. If it’s less than peak network port metered speed you’re ok there. Next comes CPU use… you might get away with a 2 CPU machine if your content has little to stress physics computations on server side - 30 avatars will tax CPU for entity server, audio and avatar mixer if you actually managed to get 30 avatars concurrently…

My gut feeling is with decent content and acceptable performance you’d be looking at more like 15. Again - there’s long term infrastructure plans that will deal with this - supposedly. But - here, now – today… it’s going to get interesting trying to support 30+ avatars on a single machine… to scale now you’d need to have multiple machines each running the heavy CPU tasks like entity server, avatar mixer and, perhaps audio… those would provide service to another machine running domain-server and the lighter-weight tasks like message-mixer, ATP server and, if used at all, server side script engine. If lots of server side scripts come into play then expect to need another machine for scripts. There’s merit to having ATP on another machine instance since it would keep it from saturating your bandwidth as new users do initial cold cache load.


#9

Thanks for your answer! My head just exploded, but thanks just the same. :slight_smile: So, for the moment until infrastructure is up to speed, I think you essentially said, “more is better.” I’ll start with wth the 2 CPU and 250Mbs plan and scale from there if the need arises (and I hope it does).


#10

BTW, my sincere thanks to the three of you for hand-holding the new guy. It is much appreciated

JR


#11

If I were you, I would try out running the domain server (“sandbox”) from my local machine for awhile before I spent the time and $ on a cloud server.

Hosting your assets ( models, textures, scripts, sounds etc.) on a web server somewhere else is easy to do and will help bandwidth problems from being an issue even if you have your domain on a cloud server.

I have been hosting a HiFi domain from my house here for a long time on an old computer that just runs without monitor, keyboard or mouse with the models, textures etc. hosted on amazon. This month’s bill from amazon is $0.72 and it has never been over a dollar. There are other ways to host your assets for free.

I’ve never had more than 8 or 10 people visiting at the same time that I know of, but I never really had any problems either. There are a few of us who just host from home here. Up bandwidth and ping from far away places seems to be more of an issue than if hosting at home. Also, I’d go with 2 or 4 cores with enough gb of ram.

Anyway, just saying you might want to host it yourself until you really know you have a need for something that costs.


#12

Hi Twa,

So what I hear you saying is that there is a way to “link” content (models, sounds, etc) sitting on server somewhere not my home and run the HIFI “sandbox” app on my machine at home? Is that correct? If so, is there a tutorial for that? This stuff is easy for a lot of you folks, but it all pretty much makes my head hurt. :wink:

Since you are talking just about hosting, would an older Mac Pro be OK or should it be a Widows rig? I ask because Mac is my comfort zone. If not, I’ve seen refurbished older Dell Xeon workstations for a couple of hundred dollars on Newegg.


#13

Yes, running a domain (sandbox) sever and hosting the assets (models, textures, scripts etc.) are two different things. But you can host them both on the same machine with ATP, although every time someone new arrives that does not have your content cached, it will cause your bandwidth to spike.

I know little about current Macs. You should ask DrFran about that as she is an Apple person and hosts her domain at home.


#14

General agreement except for physics because physics is performed client-side. Someday there will be a super-observer physics assignment client (seriously needed), but that’s only needed when no one is around. I also want to add that along with bandwidth limits anyone running a domain needs to determine the monthly data transfer cap for your hosting VPS because that too determines total costs especially when over the cap.


#15

I guess I need to learn what ATP is. :wink: There was mention of it somewhere in a conversation about a “stack”, but then I read that the stack has been replaced by the sandbox at the beginning of this year. Thus, I’m assuming that whatever ATP is, it is found in the sandbox menu(s) somewhere? But the only thing I could find in the Docs was this: “Additional settings for the Asset Server (ATP), Audio Buffers, and Avatar Mixer are available by selecting Show Advanced from the sidebar menu. Advanced setting parameters are subject to change. Documentation to come once stabilized.”

Oh! Thanks for the DrFran referral.

JR


#16

Yes, there have been marketing based name changes. Here are the ‘cliff’ notes:

domain -> sandbox
stack -> sandbox (OK yes, mixed metaphors)

stack is a collection of:

  • sandbox (previously known as a domain server)
  • various assignment clients
  • audio mixer
  • message mixer
  • avatar mixer
  • ATP server
  • Domain scripts
  • Other stuff

All of the above is installed together nowadays for the Windows and mac builds. Long ago they were separate things (it’s better done as a unit). As for Linux, well, lots of sharp edges - work in progress.

ATP is Asset Transfer Protocol. There is a server in each domain that lets you host your domain’s contents locally. The only place I could find anything on it documented is here: Asset Transfer Protocol Mappings. But, like I mentioned earlier, if you have a domain with a filled out scene (lots of stuff), you can easily swamp your home network’s bandwidth. This is why you do better hosting your assets (stuff) in a commercial hosting facility like AWS. It’s still a bit clumsy. You have to do your own system of placing stuff manually there. There is no integrated inventory or asset management in HF. We’re all expected to do that on our own (it builds courage).

Also, I agree with @Twa_Hinkle that for anyone other than crazed geeks with deep Geek-Fu knowledge, the easiest thing to do nowadays is just to install the entire package onto your local computer. If you have a spare computer, it makes for a fine ‘server’. Here is where I keep my ‘conference’ domain. Yes, it is an older Dell laptop with a network attached disk. I usually run it with the lid closed, works great. I’ve had 7 avatars in that domain with no problems. Do not expect to see many people visiting because the user base is quite small presently. The biggest crowd to date is around 18 people and that is at meetups.


How big is a world in High Fidelity
Digital Ocean, Linode, or another for the complete noob
#17

THNAK YOU!!!

“Geek-fu knowledge” That made my evening. :slight_smile:

JR


#18

There seems to be some server side CPU work, not necessarily computing physics to level of client, but – there seems to be some work on server side process to do, at least, a basic server side computation set to keep client <–> client physical items in sync… that seems to be happening in entity server. If you watch its CPU load during manipulation of physical objects and simply moving and colliding with entity items you can see it working much harder. I’m not sure if it’s actually physics calcs or “whatever”, but going back lots of months there was code added that I vaguely remember dealing with such things on server side and a subsequent increase in entity server load.

And – yes, I hope someday to see dedicated physics processors for super-observer physics specific task. This stuff is never going to scale to potential of a 35,184,372,088,832 cubic meter space potential until we can partition it into computational areas… etc etc etc. The Glorious Future… one can hope.


#19

I think we managed 24 ~ at one of the Friday meetups a few weeks back. They were troubleshooting audio at that time and mentioned a break-down threashold. Meh.

For original poster: seek Intel Core with large HDD (if you plan on ATP on same machine) imho, if you’re going to spend $40 / month on a remote computer and you’re uncomfortable with Linux, you’re better off throwing the money at a not-so-bad desktop. Figure, $20/mo at 12 months and you can get a $240 ~ $300 computer. Internet speeds will catch up eventually, and honestly, once everyone has your content cached… that bandwidth spike should stop until the next “new user” shows up. Also, there is a bandwidth threashold you can set “per avatar / per agent” inside the server settings web config page. “Sandbox Settings” :slight_smile: sandbox… ugh… still can’t swallow that pill…

I just replaced my primary domain with one of these. Let’s hope it works.

Also, pro-tip, if you expect to use Windows Remote Desktop make sure it has Win10Pro.


#20

Because the costs of vps compared to running it from home. and in the days bandwidth.

I rent a 16vCore vps for 38 euro in big datecenter. and so far not seen problems. The have 8vCore one for 24 euro. enough disk storage on no limit on data. except after 5TB it falls back to low speed you need to manual activate the higher sped for the next 1 TB.

Only using windows btw.