Models STILL disappearing from server


#1

This is an ongoing problem that I have brought up many times in the past and it is still happening.

Models are just going away, first they shrink into a tiny thing, or sometimes they go to the origin either tiny or massive but then the get dropped from the list completely.

PLEASE SOMEBODY HELP

I cant keep going thru life replacing the same things over and over again.

This is still a problem, the model is no longer on svo file and there must be a reason for it.
It was there yesterday when I logged off, I havent restarted the server in that time but when I logged on today it is gone, no trace of it in the svo file.

This is after I spent ages realigning the model for the third time.

This combined with a backup system that doesnt work leads to hair pulling, screaming and me redoing the same work over and over again.


#2

I agree, for me running on Windows:

  • Models and entities disappear from time to time
  • Backups don’t happen

I don’t run my local stack manager very often, but the most recent backup file is dated 28 Jan.

From log file today …
DEBUG] [02/11 19:04:41] [entity-server] Time since last backup [ 30002561 ] for rule [ “Half Hourly Rolling” ] exceeds backup interval [ 0 ] doing backup now…
[DEBUG] [02/11 19:04:41] [entity-server] Rolling old backup versions for rule “Half Hourly Rolling” …
[DEBUG] [02/11 19:04:41] [entity-server] Done rolling old backup versions…
[DEBUG] [02/11 19:04:41] [entity-server] backing up persist file “resources/models.svo” to “resources/models.svo.backup.halfhourly.1” …
[DEBUG] [02/11 19:04:41] [entity-server] ERROR in backing up persist file…

… plus many similar entries.

I’m going to uninstall and completely wipe my Stack Manager directories, then start again.


#3

Hi Guys. I will switch my stack manager over to windows tomorrow and to reproduce the results


#4

one thing to note… we don’t “do backups” if nothing has changed. So to David’s comment if you aren’t added or changing things, then you won’t see new backups.

As for Adrian’s original post. We need some more info to go on. I’ve very curious about this idea of the entities changing shape and moving on you. Do these entities have scripts? Is it possible someone is logging into your domain and changing them on you? Is it possible you have some script running that might be changing things? I’m just guessing here.


#5

It’s not hard to reproduce, I have models disappear all the time too, I did find one shrunk down really small at the origin, and other times, stuff just goes bye bye, especially if interface crashes, I’ll get back on and everything in the domain is gone. Not running any scripts at all in the domain, so not script related. It’s been like this since I’ve been getting on, I chocked alot of it up to us being in alpha, but at this stage (IMHO) we need to hammer down these bugs first moving forward and work towards a more robust recovery system. I know alot of good coders are all working on stuff, so I don’t complain, they don’t need me bugging them that this or that’s not working, but since it’s come up, I’ll mention my issues now. Also, at times, avatars and entities and metavoxels become unchecked and my avatar has an Iron Man helmet attached to the head. Kind of like being “ruthed” for all the SL folks out there.


#6

I have also noticed models disappearing,

It is hard to say how that is happening, isn’t there a log where is logged when and through which action models get deleted ?

My best guess is that something goes wrong when we are fiddling around with the models, potential causing a interface crash and resulting in the unwanted removal of the models.


#7

Thanks everybody for the replies.

@Zappoman with regard to the backups, I get that the daily backup wont fire with every change, but it should fire at the end of the day, and I am watching the half hourly while I work and have made many additions and deletions over a period of hours yet the halfhourly never updates since 3 weeks ago. I have checked the rules and they are ok. I have resorted to manual backups for now.

The objects that disappear are quite often large ones, 150 x 150 ground surface simple flattened cubes with texture, and locked. Sometimes I find it at the origin and it is tiny, it is about the size of the little red ball at origin, several times objects have mysteriously gone to the origin and shrunk, in these cases I can restore its size and location manually, once my entire venue of 7M and 80mtrs across moved itself to the origin but was huge like 2000mtrs across, again I could restore it.

Months ago I had some trees that would disappear, and house that would disappear regularly, I had a bad script in the domain and blamed that. I cleaned the stack and started fresh, it was good for a couple of months then when voxels were removed I added these 4 big grass floors and locked them, and one of them, usually the same one, would disappear, and the list would only show 3 (I then go to the origin to see if its there, its not) but this most recent time there was 4 on the list and I thought it was finding the same one twice but turned out that there were 2 in the same place, it had moved back to the place I duplicated it from, like undo.
They were all locked so I had to unlock one to move it back.
It wasnt missing it was moved

On another server the entire city of venice turned into a cricket ball at the origin, it was restorable, but the next day (yesterday) it was gone completely, not even in the svo file, in fact the svo file had reduced in file size and the text was visibly smaller. I restored the city from the models. There have been no scripts on this server.
They have been locked but I have just recently enabled trusted editors and will watch closely. Someone would have had to unlock it to delete it.


#8

Something to note for backups. A backup will not run if there is no changes to your domain (maybe we need to address this differently to avoid confusion).

Can those that are having an issue with backups that should be working OR models that are disappearing do the following:

  1. Send a copy of the directory listing of the resources directory. A screen shot and a zip of the file.
  2. The complete assignment-client log file.

You can either reply to the forum post or email it to me.


#9

I uninstalled Stack Manager (Windows) and completely wiped the data directories it was using. Then reinstalled to start afresh.

Backups seem to be working, though it seems that there was a glitch in which a half hourly backup was missed.

Of 3 FBXes and 12 entities I’ve added so far:

  • 1 FBX has disappeared of its own accord (is no longer listed in the list of entities).
  • 1 entity temporarily disappeared (I found it at the origin, then it snapped back to its correct place).
  • 1 FBX temporarily disappeared (it was reported as being at the origin though I couldn’t see it there).

I’ve already provided a Zip of my whole data directory to ZappoMan.

Note: Assignment client crashes always occur when you stop Stack Manager (Windows).


#10

@Adrian @ctrlaltdavid @John_Laury @MarcelEdward

We have added some logging to track this down.

It will go out when this Pull Request gets merged (within the hour) https://github.com/highfidelity/hifi/pull/4285

You will need to restart your Stack Manager to pick up the change. This will then add a check box in the settings to track all content changes. It will log the UUID of the editing node, the ID of the entity, and the properties that changed.

for example:

[DEBUG] [02/13 15:52:21] [entity-server] User [ QUuid("{a84e33dc-dd61-490d-85de-4fe5f1926e32}") ] editing entity. ID: [ id: QUuid("{f1466944-c68f-424b-95c6-d529a37a3523}") , creatorTokenID: 4294967295 , isKnownID: true ]
[DEBUG] [02/13 15:52:21] [entity-server] properties: EntityItemProperties[
rotation : {type=‘glm::quat’, x=0.124407, y=0.231189, z=-0.0594644, w=0.963073}
last edited:1423871541917970
edited ago:0.003049
]

It may make things very noisy, so if you are not experiencing entity loss I would turn this off.

When an entity goes missing, send us the log.


#11

@chris I’ve reproduced a model and an entity disappearing …

Uninstalled Stack Manager and deleted C:\Users<username>\AppData\Local\High Fidelity\Stack Manager directory.
Reinstalled Stack Manager: Windows 53
Interface: Windows 1945

Start Interface
Start Stack Manager for ctrlaltdavid domain
TP to ctrlaltdavid domain
Added a bush
Added a rustic shed
Added two more bushes the same
Deleted rustic shed
Added 3 x red cube entities
Confirmed all models and entities were listed in the list of entities
TP away to sandbox and back … all OK
TP away with model selected, deselect, and TP back … all OK
TP away
Stop Server … 1 x assignment client crash
Start Server
TP back
Only 2 models displayed and 2 entities
Entities list shows only 2 models and 2 entities
TP away
Stop Server … 1 x assignment client crash
Close Stack Manager
Exit Interface … crash on exit
Start Interface
Start Stack Manager
Start Server
TP to ctrlaltdavid domain
Still just 2 x models and 2 x entities displayed and listed in entities list
TP away
Stop Server
Start Server
TP to ctrlaltdavid domain
… It looks like the 3rd model and 3rd entity added were the ones that disappeared
TP away
Stop Server

Zipped up Stack Manager directory. Available as: http://ctrlaltstudio.com/downloads/hifi/Stack%20Manager.ctrlaltdavid.02.zip

Enjoy!


#12

@ctrlaltdavid great repro. Thankyou.


#13

@ctrlaltdavid do you know how long between when you created the last entities and when the server crashed? Do you have a stack trace for the crash?


#14

@chris It wouldn’t have been very long. However, I almost always get an assignment-client.exe crash upon stopping the server, whether or not someone visits the domain while the server is running.

No, I don’t have a stack trace. I’ll try to get one.


#15

Exception and some call stacks attached. It seems to alternate which of the first two call stacks happen first. The third call stack may be a cause or a side-effect. I can dig further if wanted.

P.S. I always get a crash if someone visits the domain. Sometime I get a crash if someone doesn’t, but not always.


#16

@ctrlaltdavid, thank you for this.