Read-only database server
Installation packages for Manifold 9 include two new executables for the Manifold database server, MANIFOLDSRV: ~\BIN\MANIFOLDSRV.EXE (32-bit) and ~\BIN64\MANIFOLDSRV.EXE (64-bit). Manifold Viewer and SQL for ArcGIS Pro do not include MANIFOLDSRV. The Manifold database server is currently read-only.
MANIFOLDSRV is a console application. The command line syntax is:
MANIFOLDSRV [<options>] <file>
- -addr:xxx -- network interface to listen on, the default is the primary network interface.
- -port:xxx -- port to listen on, the default is 9099, currently unassigned by IANA.
- -user:xxx -- user required to connect, the default is none.
- -password:xxx -- password required to connect, the default is none, ignored if user required to connect is not specified.
- -connections:xxx -- maximum number of connections, the default is 20, use 0 to remove the limit.
- -workers:xxx -- number of worker threads to process requests, the default is 4, the number can vary between 1 and 128.
The file is required and can be of any type supported by 9. If no file is provided, MANIFOLDSRV prints the command line syntax and exits. If the file is provided, MANIFOLDSRV opens it and starts serving connections to it.
If the system does not have a license for 9 installed, MANIFOLDSRV exits without prompting to install the license. Install the license using the main application, MANIFOLD.
To stop MANIFOLDSRV, press Ctrl-C or Ctrl-Break. MANIFOLDSRV automatically stops if the console window attached to it is closed, if the current user starts logging off, or if the system starts shutting down.
MANIFOLDSRV writes log files, similarly to MANIFOLD. The MANIFOLDSRV log files are named xxx-srv.log, extended to xxx-srv-yyy.log for multiple instances of MANIFOLDSRV running at the same time, to distinguish them from the log files for MANIFOLD. MANIFOLDSRV logs each accepted and closed connection as well as messages generated by worker threads that process requests.
To connect to a MANIFOLDSRV database, open MANIFOLD and add a new data source of type: 'Database: manifoldserver'. Specify the server to connect to as a network address plus an optional port number after a colon. Specify the user to connect as, if MANIFOLDSRV has been launched with the -user:xxx option. Specify the password to connect with, if MANIFOLDSRV has been launched with both -user:xxx and -password:xxx options. Specify the path to the data source to connect to, if you want to connect not to the root file used by MANIFOLDSRV, but rather to a nested data source within that file.
The connection string for a MANIFOLDSRV database consists of key=value pairs separated by semicolons. Special characters ('=', ';', '\') in keys and values are escaped. Unprintable characters in keys and values are encoded as '\uXXXX'.
- server=xxx -- the server to connect to, either an IP address or a fully qualified domain name, optionally followed by a port number after a colon.
- database=xxx -- optional path to a data source within the root file used by MANIFOLDSRV to connect to (eg, '[usa_east]::[new_york]'), if omitted, the connection is to the root file itself.
- user=xxx -- user to connect as.
- password=xxx -- password to connect with, ignored if user to connect as is not specified.
Unknown keys are ignored.
The data source for a MANIFOLDSRV database attempts to connect to the database in background. If the connection cannot be established for some time, the data source logs an error and stops attempts to connect. After the connection is established, the data source monitors it, and if the database breaks the connection unexpectedly, eg, because MANIFOLDSRV is being shut down, the data source logs an error and becomes disconnected. (In both cases above, the data source will not attempt to reconnect until you close and reopen the parent data source -- usually the MAP file. In the future we will allow reconnecting via the Refresh command.)
The data source for a MANIFOLDSRV database sends multiple requests to the server in parallel, a request does not have to wait until all previous requests have been processed by the server or even have been received by the server in full. The server processes multiple requests from the same connection in parallel, if there are worker threads available (not busy processing requests from other connections). Both sides split big data packets into parts and communicate with each other to avoid overwhelming the network with too much data sent at once. Both sides prioritize transferring small data packets (like 'get schema of a table X') before big ones (like 'fetch next N records of a big table Y'), to minimize the number of requests waiting in queues. Both sides put no limit onto the number of requests waiting in queues outside of available memory.
The data source for a MANIFOLDSRV database supports all features of spatial indexes, including those that aid rendering: access to intermediate levels for raster data, thinning for vector data and thinning for point cloud data used with LAS / LAZ files. (What this means is that if MANIFOLDSRV operates on a MAP file, rendering data from the MANIFOLDSRV database will use all smarts that rendering from a MAP file on a local system would use. This generally makes rendering perform much faster than rendering from a general-purpose database, such as PostgreSQL.)
Queries in a command window for a MANIFOLDSRV database run on the server. Queries in query components stored on the server run on the server. Script functions called from queries that run on the server also run on the server. Scripts in script components stored on the server currently run on the client. (In the future, scripts in script components stored on the server will also run on the server. Also, because scripts can access the file system, we are going to provide means to disable them on the server completely or partially, for security reasons.)
Connecting to a nested data source in a MANIFOLDSRV database creates a new connection to the server and tunnels data through that connection. For example, if MANIFOLDSRV operates on a MAP file which has a link to a JPEG2K file, attempting to read the contents of a JPEG2K file from a network client will ask the server to open the JPEG2K file and read its contents on behalf of the client. Web data sources work slightly differently: (a) connecting to a nested web data source in a MANIFOLDSRV database explicitly by specifying its name in the connection string will tunnel data through MANIFOLDSRV, but (b) connecting to a nested web data source in a MANIFOLDSRV database implicitly using the UI (by opening the data source in the Project pane, by opening a map which contains a layer from that data source, etc) will connect to the web data source directly. Direct connections to web data sources reduce the load on the server and are usually faster for clients as well.
Tunneling to nested data sources applies recursively from the root database operated by MANIFOLDSRV to all child data sources at all levels. Since each nested data source may potentially create a separate connection, if the data served by MANIFOLDSRV includes many nested data sources, consider increasing or removing the connection limit from the default 20.
Tunneling to a nested web data source in a MANIFOLDSRV database automatically turns off the option to save cached data between sessions, because the parent database used to store cached data might be read-only. If the web data source has been configured to cache data at all, the cache will still be used, but it will be cleared when MANIFOLDSRV exits.