Need help with VMWare


#1

Running the stack manager on a windows 7.1 vps. (build 92)

The domain-server.exe crashes as soon as I turn my avatar.
When the server crashes I see this in the problem signature:

Fault Module Name:    Qt5Network.dll   Fault Module Version:    5.4.1.0

Anybody have any ideas of what I might need, what runtimes I should have?

Here is the last few lines of the domain server log

[DEBUG] [09/21 11:42:09] Refusing connection from node at 127.0.0.1:64817
[DEBUG] [09/21 11:42:10] No match for assignment deployed with “d2209a00-d9ea-48b8-921d-447dd1a89f72”
[DEBUG] [09/21 11:42:10] Refusing connection from node at 127.0.0.1:64817
[DEBUG] [09/21 11:42:11] No match for assignment deployed with “d2209a00-d9ea-48b8-921d-447dd1a89f72”
[DEBUG] [09/21 11:42:11] Refusing connection from node at 127.0.0.1:64817
[DEBUG] [09/21 11:42:12] No match for assignment deployed with “d2209a00-d9ea-48b8-921d-447dd1a89f72”
[DEBUG] [09/21 11:42:12] Refusing connection from node at 127.0.0.1:64817
[DEBUG] [09/21 11:42:14] Unable to fulfill assignment request of type 7 from 127.0.0.1:64817


#2

Sounds a bit like my crash on my VPS at the end of the last meeting and the next moring it where crashed again. i have send log’s to @chris It did happen in the same qt5network.dll

https://alphas.highfidelity.io/t/simsquare-server-crash-after-meeting/7167


#3

I have been getting the same thing in my logs. I have been getting random server crashes on Ubuntu too but the server will stay up for hours and the logs show nothing when the crashes happen. I posted about it here, but I didn’t mention the crashes because I didn’t think the log entries had anything to do with it (the log shows up on boot but doesn’t crash immediately).


#4

Getting the same thing on windows @chris is this a known issue? my domain blinks off as described above its on windows 8


#5

For the devels chasing causes of this - a hint. I’ve never seen this happen IF you launch AC’s one by one with a type flag. If launched with no type and min/max pool of unassigned ACs where they pick up what DS needs then, yes, I’ve seen issues like this.


#6

So running (in Linux) nohup ./assignment-client -t 0 &> assignment.log& should fix it?

I will try this and report on it tomorrow, if I remember to.


#7

But, Am running on windows with the domains erver and seen some crash type too after the meeting.


#8

I run each separate - not for any good reason than old habits. That said - other than ATP issues which is a beast of another color - I’ve never experienced the crashes other do.

I run
assignment-client -t 0 > ac0.log&
assignment-client -t 1 > ac1.log&
assignment-client -t 6 > ac6.log&
domain-server > ds.log&

If you don’t want to overwirte logs replace > with >> (for anyone unaware of that).

0 = Audio server, 1 = avatar mixer, 3 = atp, 6 = entity server. I don’t have any server side scripts so I omitted any ACs for such. I’ve been running a domain since first week of last October, so just short of 1 year, and running as I said here last 9 months +/-.


#9

Up until a week or so ago I had about 4 months of uptime.

Ok so my executable looks like this:
nohup ./assignment-client -t 0 &> ac0.log&
nohup ./assignment-client -t 1 &> ac1.log&
nohup ./assignment-client -t 6 &> ac6.log&

I will let it run all night and see what happens, thanks.


#10

Good luck and, anytime. Be interested to hear if that changes anything. I may just have been somehow lucky.


#11

Ok, here I am 15 hours later and they are still up.


#12

2 days now, still up.


#13

Ok, I take that back. I was scripting in the domain and somebody showed up and did some voice and the domain crashed. The Avatar-Mixer and Entity-Server ACs failed to reconnect on restart but Audio-Mixer reconnected successfully.

The only suspect I see in logs is
[CRITICAL] [09/25 02:33:10] [6796] [entity-server] Cannot open gzipped json file for reading: “resources/models.json.gz”


#14

Just ran a solid week of uptime without updating the server and no crash…


#15

Because of some crashes I now use a cronjob shell script to check for the processes and if they are missing they reload.

Its actually all tied into my auto updating script since I now compile the AC/DS on another machine, package it and upload it to a place where all my other sub servers can check on the newest version and update as needed.

I actually used the methods hifi laid out in https://github.com/highfidelity/hifi-cloud to create my own Ubuntu 14.04 compile box. The binary actually runs fine on 15.04 slaves.


#16

I compile and run everything on 15.04.

Care to share your cron script?


#17

I have a jenkins server compile the binaries externally and then post them to a site I host for all of my (and @Adrian’s) domains to pull from, I can’t share it as it would be having a bunch of people pulling my compiled binaries from my amazon bucket.

In short the script just polls a text file that is updated every new compile to see if the version changed, if it did then download the new version, load the libraries and binaries and then restart.

All domains are updated within 10 minutes of a push to their git repo.