MSNet Manual

MSNet is a simple seismic network data gathering program written in Perl. MSNet can collect data in near real time or on command from any data acquisition system or other data source, providing waveform data in SEISAN format on an ftp server. The data is collected into a SEISAN data base and is therefore ready for further processing, like automatic or manual phase pick and location. Currently MSNet is designed to work with SEISAN format files using SEISAN naming convention. The most common systems to connect to is therefore Windows Seislog, Linux Seislog and SEISAN data bases. Work is in progress to also include other data acquisition systems.

MSNet is designed to run on any platform supporting SEISAN (Windows, Linux and Solaris), but has only been tested on Windows.

Windows system requirements:
            - Internet connection
            - 500 mHz computer or better
            - Windows 2000 or better
            - Reasonably accurate time
            - Seisan
Perl interpreter and modules not included in the Perl distribution:
            - ActiveState Perl 5.8.3 or newer
            - DateTime::Precise (free at CPAN)
            - Seisan::Tools (comes with this distribution)

MSNet Installation

MSNet is written in Perl and will require ActiveState Perl for Windows to be installed first. Some additional modules for Perl must also be installed. MSNet also relies on the Seisan file structure to store the generated s-files and transferred waveform files. MSNet is distributed in a zip file which contains everything you need except Seisan and Perl.

1: Install ActiveState Perl 5.8.3 or newer

Download the ActiveState Perl installer at: (Download the free ActivePerl, not ASPN Perl which is not free.)

http://www.activestate.com/Products/Download/Register.plex?id=ActivePerl

Detailed information on the installation of ActiveState Perl on several platforms is available at: http://aspn.activestate.com/ASPN/docs/ActivePerl/install.html

Choose the MSI version and download it.

Just click on the ActiveState Perl installer and follow the defaults. In the following, it is assumed that Perl is installed under C:\perl

2: Install the DateTime module

The DateTime::Precise module is avialable from CPAN. http://search.cpan.org/ This module will allow for time and date arithmetic.

ActiveState Perl comes with a “perl packet installer” which can install some modules automatically. DateTime::Precise is one of them.

Follow these steps:

While connected to the internet go to the “Start” icon in the lower left.

Start > Programs > ActiveState ActivePerl > Perl Package Manager

This will open a command window; run ppm and you will get a ppm prompt.

(Alternalty you can open a command window where ppm.exe is located and continue below)

ppm>

At the prompt type “install DateTime::Precise” and return. Your line will look like this:
            ppm> install DateTime::Precise <enter>
The package manager (ppm) will go to the CPAN repository, get the module, and install it. Not all modules are this easy. Complete directions for installing other modules you might want some day are at: http://www.cpan.org/modules/INSTALL.html

3: Install MSNet

Create the folder MSNet in c:\seismo. Unzip the MSNet.zip file in c:\seismo\MNSnet and you have the following files and directories:
            MSNet.pl MSNet script
            MSNet.ini Start file with common parameters
            BRU2.station Example of station file
            POT2.station Example of station file
            BAG2.station Example of station file

You must place the station files in the folder defined in the .ini file. The most practical is to use the default directory \seismo\msnet.

In order to run MSNet from any directory, put the Msnet directory in the path, either by defining it in the autoex.bat file or using the environmental variables (see SEISAN installtion). You can also simply put Msnet into the \seismo\pro directory, which is already in the path.

4: Install SEISAN Perl tools

Now you need to install Seisan::Tools. Which comes with the MSNet distribuiton and is not at CPAN. This is even simpler. Just create a “Seisan” folder in C:\Perl\site\lib\ and place Tools.pm in it. The complete path for Tools.pm is:
            C:\Perl\site\lib\Seisan\Tools.pm

5: Setting up parameters

Please note that the paths defined in the “ini” must be the same as the ones on your system and the paths defined in the “station” files must be the same as on the remote systems. You must also have created a Seisan REA directory structure either with makerea.exe or manually.

Before running MSNet , all the parameter files must be set up to reflect the stations used in the network The system comes with some example parameter files, which can be modified to the local condition. In addition, a SEISAN data base must be set for the incoming data. A database means creating the S-file database with program makerea (see SEISAN manual). This data base can have any name, but the most practical is to use the same data base as used for the main network data. For details on how to set up parameter files, see below.

6: Running MSNet

You can run MSNet from the command line. While in a command window go to the MSNet directory and type in:
            perl msnet.pl

If the ini-file has a different name or located in a different directory than MSNet, the complete path to the ini-file must be given as a command line argument, such as:
            perl msnet.pl c:\seismo\dat\mynet.ini
where mynet.ini is the ini-file. (You could put this in a bat file, which is in path so call would only be MSNet.)

The program only runs once so if you want it to run periodically you must use either the Windows Task Scheduler or some other scheduling tool. I like nnCronLITE which is free and easy to use. It has the same format as the UNIX cron.

Quick description

MSNet goes and gets the “.ini” file and builds an array with all the variables it found there. It then goes to the directory, which contains the station files and makes an array of all the stations that it must connect to.

Each station is checked in turn, and the complete directory list from the remotes' Seislog/Events (the directory with triggered waveform files) is transferred. After all the stations have been connected to, there is a list with all the event names from all the stations. This list is checked against the existing s-files and if certain parameters are met, the name of the waveform file is added to the s-file and to the internal list of events that will be transferred. In addition, the list is checked for new events that are identified as triggers close in time. New events are declared and corresponding s-files are made. The corresponding waveform file names are added to the internal list of events that will be transferred.

To Summarize:

Call MSNet.pl with command line argument or use default.

MSNet gathers variables from the “.ini” file.

Waveform file names are gathered from the remote stations that have station files listed in the “station” folder (default MSNet).

A new event is declared when triggers are close in time and a new s-file is made.

  Stations missed in a previous connection are then added to an existing s-file.

Waveforms are collected based on s-files made and events added to existing s-files. (meaning waveform file nems are added to S-files). Many actions are logged.

Parameter Files

MSNet.ini

MSNet.ini is a plain text file containing the common network parameters. The order of the elements in this file is of no importance. Comments are allowed in the file. Any thing after a “#” is ignored. like in Perl itself. Minimally the file must contain:
            STATION_HOME = c:\seismo\MSNet
                STATION_EXTENSION = .station
                LOG = MSnet.log
                LOG_PATH = c:\seismo\logs
                REA_BASE = C:\seismo\REA\ANG2_
                EVENT_STORE = C:\seismo\WOR\
                TIMEOUT = 0
                VERBOSE = NO
                PROCESSING_DAYS = 5
                ARRAY_PROP_TIME = 30
                DECLARE_EVENT = 3

STATION_HOME is the full path to where MSNet will look for station files.

STATION_EXTENSION is the file name extension of the station files. Please note that the “.” is included in this variable.

LOG is the name of the log file

LOG_PATH is the full path to the directory into which the log file will be written.

REA_BASE is the path name to the Seisan database that MSNet will use to store the s-files. MSNet will add the year and month. If your REA_BASE value is C:\seismo\REA\ANG2_, then s-file for Dec, 25th, 2003 would be stored in C:\seismo\REA\ANG2_\2003\12

EVENT_STORE is where the waveform files, referred to in the s-fils, are stored. Normally it is in WAV. Presently only a flat file format is supported.

TIMEOUT is a variable used only rarely when you have a station in your system that needs a little more time to respond. The timeout is in seconds and starts just before the second round of station FTP's. The default is 0

PROCESSING_DAYS is the number of days back that MSNet will check to see if data has been collected and corresponding s-files have been created. The maximum is 30 days. If one of your stations is off line for a while then MSNet will look and see if there exists an s-file matching a waveform file within the time ARRAY_PROP_TIME (see below) and add the waveform name to the s-file and the waveform file to EVENT_STORE.

VERBOSE is YES if you want to have warnings and other information output to the console. Useful in debugging. There is also a VERY setting and soon I will add an EXTREME setting. All to help with finding problems. Default setting is “NO”. Most message are logged.

ARRAY_PROP_TIME this the the absolute value of the time that will be used to merge events and add events to an s-file. It is like the merge time in the program seisei.exe. Any events within plus or minus ARRAY_PROP_TIME will be made into or added to an s-file.

Station Files

Please note that the name of the station file MUST be the characters between the “.” and the first “_” in the Seisan file you want to get. For instance for files with names like 2004-02-16-0000S.BRU2_003 the name of the station file MUST be BRU2.station

Station files: Station files are in plain text. Each station to be check must have a “.station” file and MSNet will automatically find the file. To stop the MSNet from checking a station just remove the file from the folder or change the extension. Here is an example of one of my station files. The order of the elements is of no importance. Comments are allowed in the file. Any thing after a “#” is ignored like in Perl itself.
            # This station is BRU2 located in Volcan
            IP = 64.116.15.189
            LOGIN = user
            PASSWORD = password
            REMOTE_WAVE_FORM_DIR = /cdrive/Seislog/Events
            STATION_TYPE = seisan
            DELETE_FILES = 7
            MTTNE = 120

IP this is the IP address of the station it can also be a name that can be resolved by DNS

LOGIN this is the name of an allowed user by the remote FTP server

PASSWORD this is password associted with the above user by the remote FTP sever.

REMOTE_WAVE_FORM_DIR is the complete pathname on the remote system, to where the waveform files are stored. This will be used by the FTP “cwd” command. Remember not to get mixed up with the direction of \ and / in Windows and Unix systems. If you are having trouble with getting files from a particular station set VEROSE to VERY and run MSNet then check the log file. It will dump the directories from each remote that it has transferred and you can see if what you get is what you expect.

STATION_TYPE will be used to check for different station types. The only two values used right now are “seisan”. (QNX coming)

DELETE_FILES is the number of days waveforms on the remote stations will be left before they are deleted. The values can be decimals. For instance to delete waveform files that are older than 12 hours, on a remote station, use a value of “.5”. If DELETE_FILES is 99, no waveforms will be deleted. I think it is a good idea to leave the waveforms for at least several days to a particular station. The time will depend on how often a connection is made.

MTTNE is short for MINIMUM_TIME_TO_NEXT_EVENT. In my system, I sometimes get double triggers and I use this parameter to avoid this to happen.. The parameter will ensure that the second of any pair of triggers from the same station with time less than MTTNE is disregarded. A reasonable value to set is the sum of the pre and post event times on the acquisition system and shortest event length possible.

APPENDIX:

Seisan::Tools is a collection of Perl routines widely used by many programs I write. These subroutines could have been in the MSNet program proper just as easily. This module along with some others are needed in the proper place in the Perl folder.

Once you have Seisan::tools installed you can call the tools from any Perl program.

Known Issues and bugs

If an s-file exists but a file was missed at one of the remote stations then when it gets the new file to add it to the s-file, it does not check for the MTTNE variable. The effect is that you might get s-files with multiple triggers from one station.

Files on remotes are deleted only once a station is part of an s-file make or add. If a station is never part of an s-file construct it will never have its files deleted even if they are older than the DELETE_FILES variable. I have no plans to change this right now.

Presently, there is no way to get all files from a station every time it is visited.

Events are stored in the place of your choice but only in a flat manner. They are not nested as can be done with Seisan by year and month

The VERBOSE “extreme” toggle is not implemented; only “yes” and “very” are.

Presently, only Seisan files are dealt with, QNX is coming.

Questions and Comments

If you have any question or comments feel free to write me at angel@tierrasaltas.net

Feb 16, 2003