Techniques to script OpenSim server console commands on Linux

The Problem

How can I script OpenSim server console commands? In other words; how can I script OpenSim server itself from the inside?

I can create Linux shell scripts but these are scripts on the Operating System level. The question is: how can I access OpenSim console commands like “show stats”, or “show queues” per example and put them inside bash script’s loops or conditional statements.

The challenges are:

  •  OpenSim is a separate independent process that runs through the mono framework (1st challenge).
  • It accepts its own console server commands. Secondly, OpenSim server should be always running (2nd challenge).

The Solution

Method 1

Well there is a very good solution which works by using the magic of a beautiful Linux command called “screen”. Now at first it looks that it does not do anything really…

NB: The screen command might not be available and might need installation on some Linux Distros.

On Mac OS X, the screen command can be obtained through the 3 beautiful package managers or package installers I have been long experimenting with: HomeBrew, MacPort and Fink. Actually the BSD based Mac OS X terminal can be transformed into a complete fully-fledged GNU Linux terminal through these 3 packages installers.

It took me some time to discover the “screen command” in the context of OpenSim of course but it was worth the effort. No documentation on how it can be used in OpenSim thus I decided to write this article.

This technique gave me the power over scripting the content of OpenSim server console commands thus allowing me to use them in any shell script (fully-fledge use through the powerful GNU tools like sed, gawk, grep and other scripting capabilities).

Screen is a terminal multiplexer that is used to multiplex a physical console between several processes or interactive shells. It makes it possible to open several instances of shells or instances of the terminal inside one terminal window manager. It has also the capability to send commands from a shell instance to another and even to get the output back. In addition it has the ability of sharing terminal session between users and attaching and detaching them.

There is also another command, maybe better than the screen command but I did not have the time to investigate it further because the methods explained in this article that use screen did the job very well. The command is called tmux which is another better terminal multiplexer.

The following method is actually the results of complete days of trials. I will explain it first step by step.

First we create a screen session, we name the session whatever we want. The –S Option is followed by a screen session name.

screen -S OpenSimSession -d –m

The above command creates a separate running screen session called “OpenSimSession”. The option “‘-S sessionname’ Set the name of the new session to sessionname. This option can be used to specify a meaningful name for the session in place of the default tty.host suffix. This name identifies the session for the screen -list and screen -r commands. This option is equivalent to the sessionname command” [GNU screen manual].

The –d –m options make sure that the session is detached. The option “-m Tell screen to ignore the $STY environment variable. When this option is used, a new session will always be created, regardless of whether screen is being called from within another screen session or not. This flag has a special meaning in connection with the ‘-d’ option; -d –m Start screen in detached mode. This creates a new session but doesn’t attach to it. This is useful for system startup scripts.” [GNU screen manual].

We needed to detach the screen when created that way so that we can continue working with our normal terminal.

Now how to send stuff from the normal terminal which should be executing the main bash shell script that automates things to the screen session we have already created previously “OpenSimSession”. We use something like the following.

screen -r OpenSimSession -X stuff $’mono OpenSim.exen’

Here we are sending to the screen session the command “mono OpenSim.exe” without really entering the session which is detached. We can still access the screen session and attach it if we want).  The option –r is to attach the session. –X stuff are by the way two keywords or options that should exist. The option “‘-X’ Send the specified command to a running screen session. You can use the -d or -r option to tell screen to look only for attached or detached screen sessions. Note that this command doesn’t work if the session is password protected.” [GNU screen manual].

Explaining “stuff”: “X stuff command(s) is a string in the input buffer of a window”.   It “pastes” commands to a running screen session but with less overhead. “You cannot paste large buffers with the stuff command. It is most useful for key bindings”. There is another command for large buffers called “paste” command. [REF of the manual]. Sometimes informally it is said that what really stuff does is to “‘stuff’ strings on the standard input (stdin)”.

The newline n is very important to mimic the effect of an Enter key. The $ sign is necessary also to oblige bash to evaluate whatever commands after the $ sign.

You can wait a little bit after executing the above commands so that the OpenSim server has enough time to load completely. By the way this can be easily done is any shell script through the sleep command like (sleep 60).

The above sending command can be done also through a form like this.

screen -S sessionname -X stuff ‘ mono OpenSim.exe’`echo -ne  ‘\015’`

The command echo -ne ‘\015’ emulates pressing the Enter key. The symbol ` is the back tick.

Now, we can send whatever we want like per example “shows stats” command.

screen -r OpenSimSession -X stuff $’show statsn’

If I want to reattach to the session manually while I am on the command line I can use the following command.

screen -r OpenSim

I can use Ctrl a + d to detach – But anyway the idea of creating an automated shell script is not to be present. I believe the best thing is to keep the session detached and send whatever commands to it in a bash shell loop etc.. At the end I can check the results. The following listing shows the steps used to shutdown OpenSim server completely. Waiting the necessary time is done through the sleep command. At the end, the screen session is terminated through sending an “exit” command by using the –X stuff.

screen -r OpenSimSession -X stuff $’shutdownn’       #sending a shutdown command

sleep 300   # time to wait for OpenSim to shutdown – normally should be higher

screen -r OpenSimSession -X stuff $’exitn’ #exiting the session itself (i.e terminating)

There is a good command I used which is:

screen –ls   #or screen -list

This shows running screens. I also used the following Linux command to check if “mono OpenSim” is really running.

ps –aux | grep OpenSim

Then any script using screen command could be put in a cron job that executes every one second or every 2 second as required … By the way method 1 and method 2 (see later) can be merged together and can be then put in a cron job. The merging of the 2 methods will be explained at the end of the Method 2 section.

Method 2

This method uses a newly added server console command in the latest OpenSim versions tackling this specific issue but still has limitations (explained later).

command-script [scriptfile]

This command runs a script containing only OpenSim console commands.

A script in this context is a file that ends with a .txt extension and contains OpenSim console commands. The style of how these files should be written is similar to the ones you find in .ini files. In other words, the script is NOT a bash shell script meaning does not contain Linux shell commands nor does it contain any type of programming language scripting (like Python) as I first thought. Lines are either commented out by a semicolon or a line can constitute a legitimate OpenSim console server command. It looks something like this.

;Script Name = TestOSCommands.txt

;This is a test script to be issued with “command-script” in the OpenSim console

show stats

show stats

show queues

show region

show stats

;maybe a comment here

show info

;end of script

As it can be seen, it has limitations in the sense there are no looping or other useful programming constructs that might be needed to be done on OpenSim commands. I cannot per example repeat the result of OpenSim command “show stats” after each 2 seconds and then parse the results to retrieve the numbers needed for later statistical analysis.

Combination of Method 1 and Method 2

We can create a script file similar to Method 2 with all commands and then give it to the “screen command” in a bash shell script.

The script file can look like this:

;Script Name = TestOSCommands.txt

;This is a test script to be issued with “command-script” in the OpenSim console

show stats

;ANY OTHER COMMANDS WE WANT

;end of script

Then I can use command-script like as follows in a shell script:

#!/bin/bash

#commands before …

screen -S OpenSimSession -d –m

screen -S sessionname -X stuff ‘ mono OpenSim.exe’`echo -ne  ‘\015’`

sleep 100

screen –d –r OpenSimSession –X stuff “command-script TestOSCommands.txt” && screen –d –r OpenSimSession –X eval “stuff^M”

#commands after …

It is another way of compacting screen commands and saying the same thing that I did say previously in several commands in Method 1. This can be given then to a cronjob set as required (per example every 2 seconds).

It is good when everything is finished to kill “mono OpenSim.exe” because for some reason that I don’t know, even after shutting down OpenSim, “mono OpenSim.exe” is still running. The following command is very helpful and does this automatically.

OpenSimID=`ps –ef | grep OpenSim |grep ‘mono OpenSim.exe’ | gawk ‘{ print $2 }’`

kill -9 $OpenSimID

How to get things back from the screen session?

Not much information on this but the GNU screen manual was of great help here. Let us suppose in this scenario I want to use the OpenSim console command “monitor report”.

The method to receive back the output of what was sent to the GNU screen command is not straightforward but nevertheless achievable. First I send the server console command wanted through the –X stuff options of the screen command

screen -r OpenSimSession -X stuff $’monitor reportn’

Then I wait a while (not too much though) through using “sleep command” then I can issue the following command:

screen –r OpenSimSession –X hardcopy –h monitor/MonitorFile

This creates a file named “MonitorFile” in an already created folder monitor. In other words, this dumps the current screen session terminal content to a file. This file contains all what has been logged/done in the screen session. In this case it contains all the history of OpenSim booting up + the commands that I have issued.

Of course this file needs definitely additional cutting and parsing to get exactly the output of the single command intended. This is why I created several scripts in Python and Linux bash shell to parse them.  These scripts only parses output pertaining to OpenSim console commands that deals in gathering QoS statistics. They are shared on my Github on this link.

The options to log the output of a screen session are as follows (information taken verbatim from the GNU Screen Manual):

Using hardcopy option

— Command: hardcopy [-h] [file]

“Writes out the currently displayed image to the file file, or, if no filename is specified, to hardcopy.n in the default directory, where n is the number of the current window. This either appends or overwrites the file if it exists, as determined by the hardcopy_append command. If the option -h is specified, dump also the contents of the scrollback buffer.” taken verbatim from GNU screen Manual.

There is a way to append to the file created by hardcopy

— Command: hardcopy_append state

“If set to ‘on’, screen will append to the hardcopy.n files created by the command hardcopy; otherwise, these files are overwritten each time.” taken verbatim from GNU screen Manual.

Using the log option

Command: deflog state

(none)

Same as the log command except that the default setting for new windows is changed. Initial setting is `off’.

— Command: log [state]

(C-a H)

Begins/ends logging of the current window to the file screenlog.n in the window’s default directory, where n is the number of the current window. This filename can be changed with the ‘logfile’ command. If no parameter is given, the logging state is toggled. The session log is appended to the previous contents of the file if it already exists. The current contents and the contents of the scrollback history are not included in the session log. Default is ‘off’.

— Command: logfile filename

— Command: logfile flush secs

Defines the name the log files will get. The default is ‘screenlog.%n’. The second form changes the number of seconds screen will wait before flushing the logfile buffer to the file-system. The default value is 10 seconds.

— Command: logtstamp [state]

— Command: logtstamp after secs

— Command: logtstamp string string

This command controls logfile time-stamp mechanism of screen. If time-stamps are turned ‘on’, screen adds a string containing the current time to the logfile after two minutes of inactivity. When output continues and more than another two minutes have passed, a second time-stamp is added to document the restart of the output. You can change this timeout with the second form of the command. The third form is used for customizing the time-stamp string (‘– %n:%t — time-stamp — %M/%d/%y %c:%s –n’ by default).

” taken verbatim from GNU screen Manual.

It is good to clean-up dead/inactive sessions because sometimes it happens that when running many time the same shell script, screen sessions might not be terminated successfully. To remove dead screen session, issue the following Linux terminal command:

screen –wipe

Actually if the message “Remove dead screens with ‘screen -wipe’” is seen during the run of any shell script, it means screen sessions are not closed and still hanging there. This warning message does not influence the correct functioning of the shell script.

You can check and kill dead screen sessions if needed using the following Linux commands.

ps –aux | grep SCREEN

The word “SCREEN” should be in capital letters.

Method 3 – scripting OpenSim console commands through RestConsole module

Remotely via a Restfull HTTP module aka as RestConsole (RestConsole Wiki Article in OS)

The REST console makes administration of OpenSim services possible remotely. What this interface does is it allows me to send commands to the server and retrieve output from these commands through RESTfull HTTP calls.

Why this method is important?

Well! I can script OpenSim this time through writing per example PHP scripts containing OpenSim server console commands or writing scripts using any other web programming language.

How this is done?

The OpenSim Wiki mentioned that sending is straightforward. “Receiving uses reverse HTTP, performing a long poll to a CAPS URI”. The server normally sends a “burst-on-connect” of data to the client which is a tree of all allowed commands in addition to their help information.. This is then used by the client to create the help command.

There is a sample console client called “OpenSim.ConsoleClient.exe” in the bin folder of OpenSim. If you try to run it either through mono or if you are on MS Windows directly, it will give you an error that it cannot find OpenSim.ClientConsole.ini.

There is an “OpenSim.ClientConsole.ini.example” where it should be edited and the “.example” word removed from the end of the file. Content of this file is shown below in addition to what should be changed:

[Startup]

; Set here or use the -user command-line switch

user = Test

; Set here or use the -host command-line switch

host = localhost

; Set here or use the -port command-line switch

port = 8003

; Set here or use the -pass command-line switch

; Please be aware that this is not secure since the password is in the clear

; we recommend the use of -pass wherever possible

pass = 1234

To make things work, the following ini files should be modified.

In OpenSim.ini, precisely in the [Network] category, the first lines normally talk about the Console:

[Network]

;# {ConsoleUser} {} {User name for console account} {}

;; Configure the remote console user here. This will not actually be used

;; unless you use -console=rest at startup.

ConsoleUser = “Test”

;# {ConsolePass} {} {Password for console account} {}

ConsolePass = “secret”

;# {console_port} {} {Port for console connections} {} 0

console_port = 8003

Uncomment [remove the ; near ConsoleUser and near ConsolePass and console_port (as shown above)]. If console_port = 0 it means the default port is used, not a port which is 0 (Obviously this port does not existJ).

In the same file (OpenSim.ini), precisely in the [RemoteAdmin] category,

[RemoteAdmin]

;; This is the remote admin module, which uses XMLRPC requests to

;; manage regions from a web interface.

;# {enabled} {} {Enable the remote admin interface?} {true false} false

enabled = true

;# {port} {enabled:true} {Port to use for the remote admin interface? (0 = default)} {} 0

;; Set this to a nonzero value to have remote admin use a different port

port = 0

;# {access_password} {enabled:true} {Password for the remote admin interface} {}

;; This password is required to make any XMLRPC call (should be set as

;; the “password” parameter)

access_password = “secret”

;# {access_ip_addresses} {enabled:true} {List the IP addresses allowed to call RemoteAdmin?} {}

;; List the IP addresses allowed to call RemoteAdmin

;; If access_ip_addresses isn’t set, then all IP addresses can access RemoteAdmin.

;; access_ip_addresses = 0.0.0.0, 0.0.0.0 …

; access_ip_addresses =

;# {create_region_enable_voice} {enabled:true} {Enable voice for newly created regions?} {true false} false

;; set this variable to true if you want the create_region XmlRpc

;; call to unconditionally enable voice on all parcels for a newly

;; created region

; create_region_enable_voice = false

;# {create_region_public} {enabled:true} {Make newly created regions public?} {true false} false

;; set this variable to false if you want the create_region XmlRpc

;; call to create all regions as private per default (can be

;; overridden in the XmlRpc call)

; create_region_public = false

;# {enabled_methods} {enabled:true} {List of methods to allow, separated by |} {} all

;; enable only those methods you deem to be appropriate using a | delimited

;; whitelist.

;; For example:

;; enabled_methods = admin_broadcast|admin_save_oar|admin_save_xml

;; if this parameter is not specified but enabled = true, all methods

;; will be available

enabled_methods = all

;; specify the default appearance for an avatar created through the remote

;; admin interface

;; This will only take effect is the file specified by the

;; default_appearance setting below exists

; default_male = Default Male

; default_female = Default Female

;; Update appearance copies inventory items and wearables of default

;; avatars. if this value is false, just worn assets are copied to the

;; Clothes folder; if true, all Clothes and Bodyparts subfolders are copied.

;; The receiver will wear the same items the default avatar did wear.

copy_folders = true

;; Path to default appearance XML file that specifies the look of the

;; default avatars

; default_appearance = default_appearance.xml

Enable and change the “enabled” line to true “enabled = true”. Actually follow what has been changed above (Highlighted in Yellow).

Before doing anything, start OpenSim as normal (using just “mono OpenSim.exe”), create a user and login in with a viewer like Firestorm. Then inside OpenSim Console get the HTTP-handlers by issuing the command:

show http-handlers

The result will be http-handlers for different things that can be retrieved via HTTP, HTTP (poll) etc…

Get the address like http://0.0.0.0:9000. You can add at the end /monitorstats/<regionID>

RegionID can be taken from what is shown when issuing show http-handlers command.

To use the console, OpenSim should be started in Linux terminal as follows:

mono OpenSim.exe –console=rest

Method 4 – Through RemoteAdmin plugin

RemoteAdmin module uses XMLRPC requests to manage region from a web interface. XMLRPC is a remote procedure call (RPC) protocol which uses XML to encode its calls over HTTP.  Its configurations are located in OpenSim.ini under the category RemoteAdmin. Operations related to the simulator are always available.

Why this method is important?

Well! I can script OpenSim this time through writing per example PHP scripts containing OpenSim server console commands or writing scripts using any other web programming language.

How it is done?

The main Wiki article is found here.  There are many RPC methods in the Wiki with their elaborate parameters and some RPC methods which are not documented, are deduced directly from the source code. The documented set of methods on the OpenSim Wiki can be found here.

The RemoteAdmin XMLRPC method that is of great use in the case is:

admin_console_command

I can send any OpenSim server console command from a PHP page or any ASP.net page or anything else to OpenSim server. PHP means for me the ability to use fully-fledged programming constructs (loops etc…) in the scripts.

The tool curl can be used also to facilitate things. curl is an open source command line tool and library for transferring data with URL syntax on HTTP. It can be used for other protocols.

Step by step method for running the solution (a PHP small application talking with OpenSim)

1)In the OpenSim.ini the following  should be changed – under the [RemoteAdmin] category

[RemoteAdmin]

;; This is the remote admin module, which uses XMLRPC requests to

;; manage regions from a web interface.

;# {enabled} {} {Enable the remote admin interface?} {true false} false

enabled = true

;# {port} {enabled:true} {Port to use for the remote admin interface? (0 = default)} {} 0

;; Set this to a nonzero value to have remote admin use a different port

port = 0

;# {access_password} {enabled:true} {Password for the remote admin interface} {}

;; This password is required to make any XMLRPC call (should be set as

;; the “password” parameter)

access_password = “secret”

;# {access_ip_addresses} {enabled:true} {List the IP addresses allowed to call RemoteAdmin?} {}

;; List the IP addresses allowed to call RemoteAdmin

;; If access_ip_addresses isn’t set, then all IP addresses can access RemoteAdmin.

;; access_ip_addresses = 0.0.0.0, 0.0.0.0 …

; access_ip_addresses =

;# {create_region_enable_voice} {enabled:true} {Enable voice for newly created regions?} {true false} false

;; set this variable to true if you want the create_region XmlRpc

;; call to unconditionally enable voice on all parcels for a newly

;; created region

; create_region_enable_voice = false

;# {create_region_public} {enabled:true} {Make newly created regions public?} {true false} false

;; set this variable to false if you want the create_region XmlRpc

;; call to create all regions as private per default (can be

;; overridden in the XmlRpc call)

; create_region_public = false

;# {enabled_methods} {enabled:true} {List of methods to allow, separated by |} {} all

;; enable only those methods you deem to be appropriate using a | delimited

;; whitelist.

;; For example:

;; enabled_methods = admin_broadcast|admin_save_oar|admin_save_xml

;; if this parameter is not specified but enabled = true, all methods

;; will be available

enabled_methods = all

;; specify the default appearance for an avatar created through the remote

;; admin interface

;; This will only take effect is the file specified by the

;; default_appearance setting below exists

; default_male = Default Male

; default_female = Default Female

;; Update appearance copies inventory items and wearables of default

;; avatars. if this value is false, just worn assets are copied to the

;; Clothes folder; if true, all Clothes and Bodyparts subfolders are copied.

;; The receiver will wear the same items the default avatar did wear.

copy_folders = true

;; Path to default appearance XML file that specifies the look of the

;; default avatars

; default_appearance = default_appearance.xml

To restrain access to only what we need

enabled_methods = enabled_methods = admin_console_command, admin_authenticate_user ;

Save the file then run OpenSim as usual (mono OpenSim.exe)

Once inside, open Firestorn  viewer or any other viewer and login with a user (the estate owner per example).

2) I successfully created a complete PHP application using some components of the PHP PEAR framework. Ideas were taken from the OpenSim Wiki article documenting “admin_console_command” but I made it easier to work with and made the application modular. This will be shared soon on my GitHub repository.

Now you are wondering how to run PHP without a web server on the Linux Operating system, is that possible? Yes it is. Newer PHP versions come bundled with their own mini-server. The following explain the procedure on Linux[1].

On Linux

Install the php package by issuing on let us say I am on my Fedora 21 machine:

sudo yum install php

3) If you want to run a PHP file directly from the terminal, it suffices to issue in the example of my application:

php index.php

The content of index.php looks something like the follows – this content is edited later into a script –

<?php

require_once ‘RemoteAdminController.php’;

$myRemoteAdmin = new RemoteAdmin(‘http://0.0.0.0: 9000’, ‘secret’);

//send anything you want

print  $myRemoteAdmin ->admin_console_command(‘alert To Everyone’);

print  $myRemoteAdmin ->admin_console_command(‘show stats’);

I am sure there are ways to run PHP files without a webserver on both MS Windows and Mac OS X. I did not have the time to look into this.

I have a GitHub repository where some of the scripts showcasing what was explained in this article are shared .

Hope that was helpful!. Comments are welcomed!

1 thought on “Techniques to script OpenSim server console commands on Linux”

  1. Thank you!
    I was looking for something exactly like this, I want to make a small front end to manage my backups in my running regions (never touched glade before so i guess this is a good start), and this covers the bash scripts the interface will call.

    Amazing!

Leave a Reply

Your email address will not be published. Required fields are marked *