CentOS 7 - Domain Server Compile and Update Script


#1

NOTICE: THIS IS MEANT TO BE RAN FROM BRAND NEW INSTALLS ONLY

This script is subject to change and using it is at your own risk which is why I say new installs only

Command To Start The Process From a New CentOS 7 Install

What This Script Does

  • Command pulls the most current copy of the compile script executes it
  • Updates the componants as needed
  • Pulls the newest HighFidelity code and if needed, compiles it
  • Creates a user named hifi to run HighFidelity DS/AC under (for security reasons)
  • Writes to the /etc/profile.d/coal.sh file the aliases for these local commands
  • Runs the DS/AC stack for the first time
    • Just a notice, you can access this via a web browser at address http://IPADDRESS:40100

The Initial Command to Run

bash <(curl -Ls https://raw.githubusercontent.com/nbq/hifi-compile-scripts/master/centos7-compile-hifi.sh)

Future updates and utility commands

After running the above command once, you can call the following commands locally

  • compilehifi
  • runhifi

compilehifi

  • Reference the section “What This Script Does” above.

runhifi

  • Command pulls the most current copy of the run script which does the following
  • Kills any running HighFidelity DS/AC processes
  • Runs the commands, as premade user hifi, to start the domain-server and assignment-client
  • Keeps you from ever having to handle any manual commands to change to the hifi user to run the commands

Link to the GitHub for this script is at: https://github.com/nbq/hifi-compile-scripts.

Any questions or comments feel free to ask by posting here or sending me a message!


#2

I can confirm this works well, I am now running CentOS on remote thanks to Coal.


#3

I have updated the script to handle the newest additions that @b added in. Much less work required on the part of the script now since HiFi handles TBB, Soxr, BulletSim etc.

Tested and it worked completely through on a fresh CentOS 7 install.


#4

TESTING FEEDBACK

Tester: VR_Architect
Date: March 1st, 2015
Instructions Followed: The instructions at the top of this forum post.
Total Time: Stopped after 40 minutes.

Environment: Comcast Internet 105MB down/26MB up, XenServer 6.5 RC1 based vm, 1 vCPU, 2GB vRAM, 10GB NFS on FreeNAS 7200rpm over 1GB NIC, 1 GB NIC, Centos 7 with all upgrades/updates as of Feb 28th, 2015. (XenServer physical is dual X5650 CPU’s @ 2.67GHz, 96GB RAM, dual 1GB NIC, dedicated resources to vm’s)

  1. Overall experience: Copy and paste. Could it get any easier than that? This is very newbie friendly!! Test failed due to tester accidentally stopping the script.

  2. Feature Request: With delays between steps, a newbie might think the script is hung; however, it is just working. Adding some kind of indicator might help, or a note at the top of the script.

  3. Feature Request: Would be a nice-to-have to see what version of HiFi was installed at the end. NOTE: Might be as I didn’t get to the end.

  4. Feature Request: A script to remove the build related code so only the binary code is left behind if desired to convert to a production server. Maybe an additional switch in the initial script also?

  5. Observation: Which permissions do we run this script under? I ran it with a SU admin account.

  6. A New Instruction Post Request: After building one server, how to move the HiFi binaries to another Centos 7 server. For example; what files to copy, where to put them, and what permissions? Maybe another one of your wonderful scripts to package and install :smile:

  7. Question: Does the script do a yum upgrade /update at any time? If not, would it be consider best practice to do after installation? Note that I am a newbie to Linux so I don’t know best practices, just wondering. I did it prior to running this script.

  8. Question: I tested the other forum post for compiling on Centos 7 via command line today also. There were some errors regarding libraries for peripherals. Are these libraries required for the server and if yes, did they make it into this script?

  9. Observation: When/if you modify the script to handle installs or HiFi version upgrades, and if a user has VR content on their server, the user would need to know how to ensure it didn’t get lost during the upgrade or re-running of this script.

  10. Defects: I saw several errors at one point and not knowing if this went to a log file or not, I attempted to copy the errors as they flew by. Unfortunately, I stopped the script by accident and have to start all over again. So I will now restart the script on this vm instance just to see what would happen. And I will start a new vm and start test over. Is there a log file of the script results?
    a. Restarted script results:

    [root@HiFi002 admin]# bash <(curl -Ls https://raw.githubusercontent.com/nbq/hifi-compile-scripts/master/centos7-compile-hifi.sh)
    Installing EPEL Repo.
    Installing compile tools.
    Installing base needed tools.
    Cloning into ‘hifi’…
    remote: Counting objects: 157851, done.
    remote: Total 157851 (delta 0), reused 0 (delta 0), pack-reused 157851
    Receiving objects: 100% (157851/157851), 141.79 MiB | 2.97 MiB/s, done.
    Resolving deltas: 100% (113603/113603), done.
    Already up to date with last commit.
    Source needs compiling.
    Killing Running Processess
    /dev/fd/63: line 181: cmake: command not found
    make: *** No rule to make target `domain-server’. Stop.
    Creating Last-Compile Backup Directory
    Creating Runtime Directory
    cp: cannot stat ‘/usr/local/src/highfidelity/hifi/build/domain-server/domain-server’: No such file or directory
    cp: cannot stat ‘/usr/local/src/highfidelity/hifi/build/domain-server/resources’: No such file or directory
    cp: cannot stat ‘/usr/local/src/highfidelity/hifi/build/assignment-client/assignment-client’: No such file or directory
    Restarting your HiFi Stack as user ‘hifi’
    [root@HiFi002 admin]#


#5

TESTING FEEDBACK Round 2

Tester: VR_Architect
Date: March 1st, 2015
Instructions Followed: The instructions at the top of this forum post.
Total Time: 56 minutes

Environment: Comcast Internet 105MB down/26MB up, XenServer 6.5 RC1 based vm, 1 vCPU, 2GB vRAM, 10GB NFS on FreeNAS 7200rpm over 1GB NIC, 1 GB NIC, Centos 7 with all upgrades/updates as of Feb 28th, 2015. (XenServer physical is dual X5650 CPU’s @ 2.67GHz, 96GB RAM, dual 1GB NIC, dedicated resources to vm’s)

  1. Overall experience: Excellent and completed to end. Hat’s off to the script creator and instructions.
  2. Defects: Not sure if these are defects or not. I extracted any suspect looking messages for someone to see. My apologies for any mis-typing as I actually typed all this from screenshots. Everything seemed to compile. I will test server start/stop after this post.

Looking for connect in socket:dl – not found
Looking for recv in network:dl – not found
Looking for getch in winmn:dl – not found
Looking for idna_to_ascii_lz in idn:dl – not found
Looking for process.h – not found
Looking for 35 include files stdio.h, …, io.h – not found
Looking for 37 include files stdio.h, …, sys/utime.h – not found
Looking for 37 include files stdio.h, …, x509.h – not found
Looking for 42 include files stdio.h, …, idn-free.h – not found
Looking for 42 include files stdio.h, …, idna.h – not found
Looking for 42 include files stdio.h, …, tld.h – not found
Looking for 45 include files stdio.h, …, sys/filio.h – not found
Check size of __int64 - failed
Looking for stricmp– not found
Looking for stricmpi – not found
Looking for inet_ntoa_r – not found
Looking for closesocket – not found
Looking for getpass_r – not found
Looking for _strtoi64 – not found
Looking for idn_free – not found
Looking for idna_sterror – not found
Looking for tld_strerror – not found
Performing Curl Test HAVE_FIONBIO – Failed
Performing Curl Test HAVE_IOCTLSOCKET – Failed
Performing Curl Test HAVE_IOCTRLSOCKET_CASE – Failed
Performing Curl Test HAVE_SO_NOBLOCK – Failed
Performing Curl Test HAVE_O_NONBLOCKHAVE_GETHOSTBYADDR_R_S – Failed
Performing Curl Test HAVE_GETHOSTBYADDR_R_7 – Failed
Performing Curl Test HAVE_GETHOSTBYADDR_R_5_REENTRANT – Failed
Performing Curl Test HAVE_GETHOSTBYADDR_R_7_REENTRANT – Failed
Looking for BZ2_bzCompressInit in cmbzip2 – not found
Looking for 3 include files sys/types.h, …, copyfile.h – not found
Looking for 3 include files sys/types.h, …, direct.h – not found
Looking for 5 include files sys/types.h, …, ext2fs/ext2_fs.h – not found
Performing Test HAVE_WORKING_EXT2_IOC_GETFLAGS – Failed
Looking for 27 include files sys/types.h, …, sys/scl.h – not found
Looking for MD5Init in md – not found
Looking for _CrtSetReportMode – not found
Looking for getvfsbyname – not found
Looking for lchflags – not found
Looking for strncpy_s – not found
Performing Test HAVE_STRUCT_TM___TM_GMTOFF – Failed
Performing Test HAVE_STRUCT_STATES_FNAMEMAX - Failed
Performing Test HAVE_STRUCT_STAT_ST_BIRTHTIME - Failed
Performing Test HAVE_STRUCT_STAT_ST_BIRTHTIMESPEC_TV_NSEC - Failed
Performing Test HAVE_STRUCT_STAT_ST_MTTIMESPEC_TV_NSEC - Failed
Performing Test HAVE_STRUCT_STAT_ST_MTIME_N - Failed
Performing Test HAVE_STRUCT_STAT_ST_UMTIME - Failed
Performing Test HAVE_STRUCT_STAT_ST_MTIME_USEC- Failed
Performing Test HAVE_STRUCT_STAT_ST_FLAGS - Failed
Performing Test HAVE_STRUCT_STATVFS_F_IOSIZE - Failed

Now HiFi source code pull
Looking for pthread_create in pthreads – not found
Could NOT find SDL2 (missing: SDL2_LIBRARY SDL2_INCLUDE_DIR)
Could NOT find RSSDK (missing: RSSDK_INCLUDE_DIRS RSSDK_LIBRARIES)
Could NOT find VHACD, etc….
Building CXX object libraries/model/CMakeFiles/model.dir/src/model/Geometry.cpp.o /usr/local/src/highfidelity/hifi/libraries/model/src/model/Geometry.cpp: In member function ‘const Box model::Mesh::evalPartBounds(int, int, model::Boxes&) counst’: /user/local/src/highfidelity/hifi/libraries/model/serc/model/Geometry.cpp:100:29: warning comparison between signed and unsigned integer expressions [-Wwsign-compare]
If ((*index) != PRIMATIVE_RESTART_INDEX) {


#6

Hey @VR_Architect, I notice the cmake not found error… What happened is when you installed in the other post, you mentioned you did not add the “—prefix=/usr” which is what made it install to /usr/local which makes hifi not find it as well.

Edited to add, the other errors seem interesting, wonder if its missing flags set somewhere for Xen to allow it to do specific things. I never used Xen so I would not know, but never seen those.


#7

Hi @Coal , I did use the --prefix=/usr in my first test on server #1 for that other post. For the test on this post, I used a new server #2 (also the rerun of the script). For the 2nd test on this post, I used another new server #3. So all tests were from fresh new servers.

The first (this post) test errors are from the rerunning of the script and as you suspected, it required a fresh Centos 7 install.


#8

Does this create startup scripts so the server is spawned on boot? or do we need to figure that out, and if so, any hints on what needs to be run so I know what to enter in the startup script?

thanks


#9

On boot, just do ‘starthifi’ or look in the github repo for the start-hifi.sh script and call it on boot directly. I will get something else up soon that works for reboots.

I am going to be compiling my own centos version of the binary soon which people can just pull on updates and it will all be handled that way.


#10

Ok cool, one suggestion though it is minor, you might want to make sure wget is installed from the script, as i ran it on a fresh shiny new Core install of 7 and it failed due to that not being included in the core edition for whatever bizarre reason…


#11

It does actually, no idea why yours did not pick it up @Ronnie -

function doyum {
  echo "Updating Your OS - If you see Kernel listed you should restart when done and after type 'runhifi' to restart HighFidelity."
  yum update -y
  echo "Installing EPEL Repo."
  yum install epel-release -y > /dev/null 2>&1
  echo "Installing compile tools."
  yum groupinstall "development tools" -y > /dev/null 2>&1
  echo "Installing base needed tools."
  yum install openssl-devel git wget sudo libXmu-* libXi-devel qt5-qt* -y > /dev/null 2>&1
}

That is from the script


#12

Weird… then again our shiny new dev server only had a 1.9GB filesystem for some reason… so we’re waiting on our hosting provider to get back to us and fix the issue. So who knows what went wrong! needless to say with such little space the script failed to run fully anyway. so maybe that’s why.


#13

I just tried your script on a new server, it appears to be broken, wget wasnt installed by the script, and after putting it on manually, it seems to flood the console with more errors and fail after just a few seconds

[root@d10252 ~]# bash <(curl -Ls https://raw.githubusercontent.com/nbq/hifi-compile-scripts/master/centos7-compile-hifi.sh)
Updating Your OS - If you see Kernel listed you should restart when done and after type ‘runhifi’ to restart HighFidelity.
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
No packages marked for update
Installing EPEL Repo.
Installing compile tools.
Installing base needed tools.
/dev/fd/63: line 159: git: command not found
/dev/fd/63: line 165: pushd: /usr/local/src/highfidelity/hifi: No such file or directory
/dev/fd/63: line 168: git: command not found
Source needs compiling.
Killing Running Processess
/dev/fd/63: line 184: cmake: command not found
make: *** No rule to make target `domain-server’. Stop.
chown: cannot access ‘/usr/local/src/highfidelity/hifi/domain-server/resources/web’: No such file or directory
/dev/fd/63: line 191: popd: directory stack empty
chown: cannot access ‘/usr/local/src/highfidelity/hifi/domain-server/resources/web’: No such file or directory
cp: cannot stat ‘/usr/local/src/highfidelity/hifi/build/domain-server/domain-server’: No such file or directory
cp: cannot stat ‘/usr/local/src/highfidelity/hifi/build/domain-server/resources’: No such file or directory
cp: cannot stat ‘/usr/local/src/highfidelity/hifi/build/assignment-client/assignment-client’: No such file or directory
Running your HiFi Stack as user hifi
To update your install later, just type ‘compilehifi’ to begin this safe process again - NO DATA IS LOST
[root@d10252 ~]#


#14

@Ronnie, its not broken just be sure you can install these packages manually as its supposed to do it…

yum install wget git

See what it tells you.

The script does work for everyone else so there is some case that is keeping yours from executing yum correctly and if we can isolate it I can try to work with you.


#15

still failing, this time it looks like it isnt downloading and compiling cmake either
root@d10252 ~]# bash <(curl -Ls https://raw.githubusercontent.com/nbq/hifi-compile-scripts/master/centos7-compile-hifi.sh)
Updating Your OS - If you see Kernel listed you should restart when done and after type ‘runhifi’ to restart HighFidelity.
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
No packages marked for update
Installing EPEL Repo.
Installing compile tools.
Installing base needed tools.
Cloning into ‘hifi’…
remote: Counting objects: 165955, done.
remote: Compressing objects: 100% (64/64), done.
remote: Total 165955 (delta 36), reused 0 (delta 0), pack-reused 165890
Receiving objects: 100% (165955/165955), 145.45 MiB | 11.63 MiB/s, done.
Resolving deltas: 100% (119537/119537), done.
Already up to date with last commit.
Source needs compiling.
Killing Running Processess
/dev/fd/63: line 184: cmake: command not found
make: *** No rule to make target `domain-server’. Stop.
cp: cannot stat ‘/usr/local/src/highfidelity/hifi/build/domain-server/domain-server’: No such file or directory
cp: cannot stat ‘/usr/local/src/highfidelity/hifi/build/domain-server/resources’: No such file or directory
cp: cannot stat ‘/usr/local/src/highfidelity/hifi/build/assignment-client/assignment-client’: No such file or directory
Running your HiFi Stack as user hifi
To update your install later, just type ‘compilehifi’ to begin this safe process again - NO DATA IS LOST
[root@d10252 ~]#

EDIT: By manually installing cmake from repot, suddenly we are now a coupel version out of date…
CMake Error at CMakeLists.txt:1 (cmake_minimum_required):
CMake 2.8.12.2 or higher is required. You are running version 2.8.11

the source version the script downloads is not being compiled and installed for some reason, a lot of output is being suppressed by the script in to /dev/null so it makes debugging a little tricky.

EDIT2
when pulling the script apart and running the YUM commands manually, i find that some packages are not found on the mirrors, and this is casing YUM to not install the requires dependancies

openssl-devel-1.0.1e-34.el7_0. FAILED
http://mirror.alfahosting-pro.de/mirror.centos.org/7/os/x86_64/Packages/openssl-devel-1.0.1e-34.el7_0.3.x86_64.rpm: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.

Error downloading packages:
1:openssl-devel-1.0.1e-34.el7_0.3.x86_64: [Errno 256] No more mirrors to try.


#16

@ronnie - there has to be something with your config. I do not know the hardware it is running on, if its virtualized, if so what sort of restrictions are in place. I do not know what sort of dns servers you are using (8.8.8.8 is my usual fall back).

If you wish to contact me on Skype, my name is Holoshed - with that we can work out more but in the end if it boils down to the basic stuff not working such as downloading and compiling cmake, there isn’t much I can really help with right now as I would have to log into it and spend time manually debugging it.

If you want the basic commands to run to test, you can always refer to this manual writeup: https://alphas.highfidelity.io/t/how-to-compile-hifi-in-centos-7-x-domain-server-and-assignment-client/1109


#17

It may be down to patchy mirrors, it ran like a charm on my own virtual server locally yesterday, but this is a physical U1 rack server hosted in Germany somewhere… I’ve been fighting with YUM a little and manually at least im now as far as compiling cmake from the source code the script downloads. as for openssl-devel i still can’t get that to install, after i got around the 404 by running yum clean all, it tells me it cant resolve a dependancy with the following

Error: Package: 1:openssl-devel-1.0.1e-34.el7.x86_64 (base)
           Requires: openssl-libs(x86-64) = 1:1.0.1e-34.el7
           Installed: 1:openssl-libs-1.0.1e-34.el7_0.3.x86_64 (@updates)
               openssl-libs(x86-64) = 1:1.0.1e-34.el7_0.3
           Available: 1:openssl-libs-1.0.1e-34.el7.x86_64 (base)
               openssl-libs(x86-64) = 1:1.0.1e-34.el7
 You could try using --skip-broken to work around the problem
 You could try running: rpm -Va --nofiles --nodigest
[root@d10252 cmake-3.0.2]#

Not sure how to get around the above issue, short of re-formatting the entire server and starting over.

EDIT:
the Qt libs woudl not install either, not sure if there is an error in the yum config, but we’re about the nuke the server and start again.

[root@d10252 ~]# yum install qt5-qt*
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
No package qt5-qt* available.
Error: Nothing to do 

#18

Just nuked the server and started fresh, Same issue again:-

Installing base needed tools. <-- This stage seems to be failing, it exits very quickly

/dev/fd/63: line 133: wget: command not found
tar (child): cmake-3.0.2.tar.gz: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now

Here is the output of the failed yum command above, when run manually.

EDIT: Installing epel-release seems to fail too…
[root@d10252 ~]# yum install epel-release
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
No package epel-release available.
Error: Nothing to do

EDIT 2: Got qt finally installe,d openssl-devel is the only one not installing now, everythign else is compiled and waiting but naturally compiling the stack failed at this point because openssl is not present

EDIT3: Finally after openssl-devel was installed forcefully, and manually, i was able to compile the stack, and everything is up and running now :smiley:


#19

Could your yum repository pointer file be non-standard? The script runs great for my servers still.


#20

the script it’s self is working fine now all the yum stuff is satisfied, however the issue turned out to be a problem with the repo’s themselves and not the script… this is still a mystery but my guess is something went wrong with the mirrors or repos as it is not the first time this problem has arisen for people, yum was unable to install the openssl-devel due to unsatisfied dependancy versions for other libs it needed. after forcing a local install of that package by manually downloading the rpm file and doing what yum wasn’t doing, we were able to progress through to a successful compile.