Surveys and data


Instruments


Support to other department sections


Support Dr. Scient. thesis


Contribution to "Scientific infrastructure"


Obsolete, kept for reference


edit SideBar

Last update:
16 May 2014

TAPE TRANSCRIPTION


1.  INTRODUCTION


Department of Earth Science possesses several thousand tape reels containing both raw seismic, gravity meter, magnetometer and navigation data collected from around 1972 and onwards, and processed (stacked) data. These tape reels occupy several cubic meters of storage place.

In 2005 we started a tape transcription project aiming to free space in storage area. The tape transcription consisted of these stages:

  1. Reading tape reels and copying content to computer disk, while simultaneously checking for data- and format errors.
  2. Converting data from original SEG-B and SEG-C format into SEG-Y
  3. Automatically generating near-trace plot for each line.
  4. Automatically generating summary web page for each line, including links to near-trace plot and data in their original format, and SEG-Y format.

Status as of 5 Dec. 2006:

  • SEG-B /SEG-C field data: Most tapes with this format has been read.
  • GECO processed data delivered 1980-1990: Tape format is now decoded. Must write some software to read these tapes.
  • Transcription of SEG-C to SEG-Y format, and navdata in tabular format to UKOOA P1/1990:
    • 1981 Svalbard Margin / Storfjorden:
      SEG-Y OK, Navdata OK
    • 1988 Jan Mayen:
      Many reels are missing - must search for these. Navdata not found.

2.  SURVEY OVERVIEW




2.1  Listing of folders, files


2.2  Details on individual surveys




3.  SOFTWARE


Software for tape-reading, conversion to SU/SEG-Y, QC

3.1  Read SEG-B formatted 9-track tape


Version written in bash: read-SEGB-from-9tr-tape.sh.txt

3.2  Read SEG-C formatted 9-track tapes


  • First version written as shell script: read-9tr-tapes.sh.txt
  • This is the original version written in C++: segcREAD.C
  • This is a version written in Python: tape_segc_read.py
    It was re-written as the old C++ version did not work on a newer Suse Linux installation.

3.3  Conversion from SEG-C to SU (Seismic Unix)


  • This program converts a single SEG-C file to a SU file: segc2su.C
  • Batch conversion of a range of SEG-C files to SU (Seismic Unix) files: redo-SEGY.py

3.4  SEG-Y and QC processing


This program operates on a user-specified range (i.e. a profile line) of SU files. It generates a range of files:

  • A complete SEG-Y file of this profile
  • A near-trace SU file
  • A JPEG image neartrace plot (both small and large version)
  • A MD5 checksum file of the main SEG-Y file, so integrity of this file can be verified later.

First make a list of profile line parameters in a file called file-list.txt. Parameters are one one line, in this sequence:

 Linename, source directory, target directory, min_reel, max_reel, min_FFID, max_FFID

Next, execute the script batch-collect.py. This script reads the file "file-list.txt" and calls a new script with parameters found on every line of this text file.

The script that does the actual work is called collect-SU-files.py

3.5  Make WEB page


This script operates on SEG-Y, SU and JPEG files files found in a specified directory. It is not used at the moment, as there are two methods of displaying wide JPEG near trace files on a web page (see below); the preferred method must be refined on order to be useful. The script is called make-web-page.py.

3.6  Displaying large JPEG image files on web page


Two methods were investigated in order to be able to display the very wide JPEG files representing the near trace plots.

METHOD A

Click to see tiling method example, like Google Maps
Click to see tiling method example. Use zoom control box in lower right corner.

Method A (preferred) uses Javascript to present a zoomable view of an image that has previously been tiled. This method mimics Google Maps. It provides a very smooth movement of the image. The drawback is that the tiling software script generates a quadratic image, whereas our images are typically very wide (20-40.000 pixels) but not very high (2-3000 pixels). The quadratic image will simply be too large for the tile-generating software library that is used (PIL = Python Imaging Library). The solution is to modify the Javascript so it is not dependent on a large quadratic tile image.

Javascript source: Giant Scalable Image Viewer (GSIV)

METHOD B (discarded)

Click to see method B
Click to see zoom box method. Use arrow keys to increase/decrease zoom rate and size of zoom box

Method B also uses Javascript to provide a zoom window that shows data details. The drawback is that the complete JPEG image - potentially very large - must be downloaded first; as a result, the zoom window moves sluggishly.

Javascript source: Javascript/CSS/DOM image zoomer - tjpzoom 2.0



4.  TAPE TRANSCRIPTION SERVICES




5.  GECO PROCESSED DATA


We have large volumes of GECO processed tape reels. The format is SEG-Y, but there are certain anomalies. Description of format discrepancies is here.

We conferred with Western-Geco i Stavanger, and they replied:

Email received 15 Nov 2006:

Hvis dine data ikke er SEG-Y, kan de kanskje være WGC-4 (WesternGeophysical Code-4).
Dette formatet ble brukt sent på 70-tallet men kanskje også litt tidlig på 80-tallet.
Da er det i tilfelle noen 80 bytes records med EBCDIC header fulgt av <eof> merke, og
så traser med en 100 byte header først i recorden.

Andre formater i bruk var SEG-D for data de-multiplexet on-board.
Internt Norseis format kan ha blitt levert i forbindelse med prosessering fra Oslo ?

Email sent 1 December 2006:

Jeg har laget en detaljert beskrivelse av de fem første blokkene i en typisk GECO tape med post-stack data:
    http://www.geo.uib.no/eworkshop/tape-transcription/GECO-format/

Email received 1 December 2006:

We have looked at the data files and screen dumps. It's SEG-Y format, and we have successfully read
and displayed the three traces.

The problem is that the SEG-Y binary header is blank. Thus all parameters in the binary header must
be entered manually in the SEG-Y reading program.

Made these assumptions:

   Input samples per trace = 1751
   Input sample Interval = 4
   Input sample format  = IBM_SINGLE_PRECISION (32 bit floating point)
   Measurement system = METERS
   Trace length = 7000 ms
   Trace sort code = trace (stack)

The display shows a normal seismic trace starting at around 1900 ms (fairly deep). The reason might
be that first traces on line are font muted ("taper on"), or that water depth is large.

It is also possible that the reason can be trace header size is supposed to be larger then standard
240 bytes; I believe it is highly unlikely, but it cannot be ruled out completely.

Hope this is helpful. The challenge here is proper coding of the empty binary header in the reading
program, as this section is missing (for unknown reasons).



6.  TIPS & TRICS


In Norwegian

6.1  Mounting of USB disk on Linux PC


Fra Jarles PC, 10. mai 2007:

***********************************
***    MOUNTING AV USB-DISK     ***
***********************************

    MAXTOR

1.  Plugg i, slå på USB-DISK
2.  I terminalvindu
	* Skriv dmesg (for å finne ut hva disken heter (sda1 el sda2)) 
	* Skriv mount/dev/sda1 /mnt/usb
3.  Skal da finnes under mnt/usb 
4.  Overfør ønskede filer ved hjelp av Konqueror eller gftp
5.  Unmounting (i terminalvindu)
	* Skriv unmount/dev/sda1 /mnt/usb
6.  Da skal 	
	>Is this correct (y/n) :y
	>Start reading : JanMayen1982/Reel-059/Reel-059.log contains 0 records
	>Append to total number of records: total skuddpunkt + 2 (f.eks 133)
7.  Sjekk loggen som er skrevet i mappen: /home/tape/DATA/DISK2/notOnBackup/JanMayen1982/Reel-xxx
	* Sjekk filstørrelsen 
	* Sjekk antall filer
	* Slett de 2 siste filene som er lagt til (er tomme)
8.  Skriv opp eventuelle mangler i loggen som føres i OpenOffice.org
9.  Hvis det mangler FILER les tapen en gang til (prøv 3-4 ganger)
10.  Når alle filene som går ann å lese er lest
	* Lag SU fil, i nytt terminalvindu skrives: 
	cd DATA/DISK2/notOnBackup/JanMayen1982
	[tape@pcom JanMayen1982]$ ../../../../SCRIPTS/segc2su.sh Reel-xxx
11.  Sjekk SU mappen at alle filene er konverterte
12. Når alt er OK, eventuelt annet som er å bemerke skrives i loggen.

6.2  Tape reading SEG-C files


***********************************
***    TAPE LESING .segc-fil    ***
***********************************

1.  Sett inn tapen
2.  Trykk på LOAD
3.  I terminalvindu gå til folder for tokt
	/home/tape/DATA/DISK2/notOnBackup/JanMayen1982
4.  [tape@pcom notOnBackup]$ ../../../SCRIPTS/segcREAD JanMayen1982/Reel-xxx (rull nr)
	>Start reading into : ./JanMayen1982/Reel-xxx
	>Is this correct (y/n) :y
	>Start reading : JanMayen1982/Reel-059/Reel-059.log contains 0 records
	>Append to total number of records: total skuddpunkt + 2 (f.eks 133)
5.  Sjekk loggen som er skrevet i mappen: /home/tape/DATA/DISK2/notOnBackup/JanMayen1982/Reel-xxx
	* Sjekk filstørrelsen 
	* Sjekk antall filer
	* Slett de 2 siste filene som er lagt til (er tomme)
6.  Skriv opp eventuelle mangler i loggen som føres i OpenOffice.org
7.  Hvis det mangler FILER les tapen en gang til (prøv 3-4 ganger)
8.  Når alle filene som går ann å lese er lest
	* Lag SU fil, i nytt terminalvindu skrives: 
	cd DATA/DISK2/notOnBackup/JanMayen1982
	[tape@pcom JanMayen1982]$ ../../../../SCRIPTS/segc2su.sh Reel-xxx
9.  Sjekk SU mappen at alle filene er konverterte
10. Når alt er OK, eventuelt annet som er å bemerke skrives i loggen.

6.3  Opening SEG-C files


***********************************
***    ÅPNE .segc-fil           ***
***********************************

ÅPNE .segc-fil (f.eks. når konventering av segc2su.sh ikke virker)

1.  Fin filen
2.  Åpnes i khexedit
3.  Header t.o.m. ff
	* Denne bakgrunnsinfo. kan være av ulik lengde (serier har som oftes lik header)
4.  Hvis konventering av segc2su.sh ikke virker er trolig headeren feil
	* Vanligste lengdene er 144 og 272 (m.a.o. hvor mange tall som står før ff i teksten)
5.  For å endre SCRIPTS
6.  Gå til /Home Folder og mappen SCRIPTS
	file:/home/tape/SCRIPTS
7.  Finn filen segc2su.C åpne denne i Kate
8.  Gå ned til header_size endre til 144 eller 272
9.  Kopier Compilation linjen (øverst (g++ -lm -o....))
10. LAGRE og lukk
11. Åpne nytt terminalvindu, bytt til rett folder cd SCRIPTS
12. I file:/home/tape/SCRIPTS kopieres Comilation linjen inn og kjør. (segc2su.C er klar til konventering fra .segc til .su)

6.4  Opening SeismicUnix files


***********************************
***    ÅPNE .su-fil             ***
***********************************

For å se en su-fil, plaser deg i folderen

1.  cd DATA/DISK2/... (sti)
2.  Skriv 
(f. eks. 0001.su) 
	* Bruk en perc mellom 0-100 (best m størst tall)

6.5  Terminal window commands


	
***********************************
***    TERMINAL KOMMANDOER      ***
***********************************

ls = list opp
ls-l
ls-if

cd = changes directory
cd .. = en mappe tilbake
../ = en mappe tilbake
pwd = se stien
Ctrl + c = avbryt handling
mt-f (stien) = få status
history | grep dd


KODER
På logg på maskin
tape
********
super user
su
********
(Ctrl+Alt+F2)

6.6  Reading SEG-Y files using segyread in Seismic Unix



Jeg har funnet ut hvordan den filen som vi laget med dd kan leses med segyread.
Problemet med filen er at den mangler informasjon om antall samples per trace
og formatkode i binær headeren. Hvis jeg bruker parameterene:

    endian=0 over=1 format=1 ns=1751

i segyread så leses den helt fint. Vi kan antakelig gjøre det samme når vi leser
fra tape.

Det er selvfølgelig litt dumt at man må gjette antall samples. Man bør kunne regne
ut dette tallet hvis man leser de 3 første recordene til separate filer, og sjekker
størrelsen på fil nr 3.

Bent 

7.  TAPE DRIVE HARDWARE


7.1  Tape drive 9-track 1/2" tape


Click images to enlarge.

Alternate tape drives:

7.2  Maintenance

Cleaning procedure - see section 6 in the User Manual:



7.3  Tape drive repair services



8.  NAVIGATION, GRAV/MAG DATA


8.1  Magnavox integrated navigation system, 1980


Block diagram Magnavox integrated navigation system, 1980.]]
Block diagram Magnavox integrated navigation system, 1980.


In this system, raw navdata was written continuously to 1/2" 9-track tape. There were different record types, according to this list:

  -------------------------------------------------------------------
  TYPE     DATA                FREQUENCY
  -------------------------------------------------------------------
  Type 1   Satellite Pass      Every satellite pass
  Type 2   Satellite Fix       Every satellite fix
  Type 3   Shot                Every shot
  Type 4   Operator Action     Every operator mnemonic
  Type 5   Gravity             Every gravity transfer
  Type 6   Operator Comment    Every use of CMNT mnemonic
  Type 7   Raw data            Every 10 second
  Type 8   Raw Satellite Data  During satellite pass
  Type 9   Update              Every system update
  Type 10  Sail line data      Every sail line setup and line change
  Type 11  Position fix data   At completion of all CPFX position
                               fixes and at shot/DR printout time
                               if the automatic position fix feature
                               is used.

Excerpt from "Operator's Manual":

  A Type 1 record is produced at the end of each satellite pass. These records contain
  the processed data collected during the pass. This data includes the majority voted
  message set, the navigator's table, and the doppler data.

  A Type 2 record is produced at the end of the satellite fix process. These records
  contain the results of the fix computation, including the update values.

  A Type 3 record is produced with each shot. These records record the systems position
  at shot time as well as data from attached sensors.

  Type 4 records are produced when the operator enters mnemonics. These records may
  be used to trace the operator's action on the system.

  Type 5 records are produced whenever gravity data is transmitted to the system.

  Type 6 records result from the use of the comment (CMNT) mnemonic. The operator may
  place up to 64 characters of information in a record to annotate the mag tape for
  subsequent processing.

  Type 7 records are produced every ten seconds unless supressed with the MTRD mnemonics,
  These records contain ten seconds of data collected at a rate of once per second from 
  each nav aid.

  Type 8 records are written during a satellite pass and contain the raw data from the
  satellite receiver. These records are provided to enable fix recomputation for post
  processing.

  A Type 9 record is written each time the system position is updated by a satellite
  update, operator entry of a new latitude or longitude, use of the DLAT, DLON, PFIX,
  URPF mnemonics, etc.

  A Type 10 record is written at the end of the sail line initialization sequence,
  when an automatic line change occurs and when the NLIN mnemonic is used. It contains
  sail line parameters.

  A Type 11 record is written after all periodic nav aid position fixes have been completed
  or after a CPFX computation. It contains information concerning what nav
  aids and stations were used and the results of each position fix.

  When a mag tape is written, the data is first collected in a write buffer.
  It is then written to tape and the mag tape status is checked to ensure that the
  hardware read-after-write detected no errors. If errors were detected, the records
  is repeated up to three times, after which the deck is placed off line and rewound.
  The second deck is placed in service to continue mag tape operations.



8.2  Decoding records on Magnavox nav system 1980 tapes


Unfortunately we don't have a detailed description of how the individual varables are placed in each record, and their encoding. So we have to do some reverse-engineering, based on hex dumps of different records, and listings of the Fortran programs that did the tape reading and decoding in the 1980-ties.

Here are hex dumps of a Type 5 record. The upper window shows byte content of record (type 5 is indicated in byte no. 3 & 4, which is a two-byte integer in big-endian format). The lower window shows the same record (and more) after EBCDIC-to-ASCII conversion. A segment of data (a gravity record, containing date, time, spring tension, gravity etc.) now becomes legible. So a record can contain a mix of integer (two-byte, big-endian), real, ASCII and EBCDIC variables. Format seems to be fixed-position.

Decoding tape record Type 5, using Hex dumps
Decoding tape record Type 5, using Hex dumps. Upper window = Plain Hex dump. Lower window = Assumed EBCDIC encoding, to make gravity data segment legible.



In this particular record, we suggest following decoding of variables (all numbers are big-endian):

  -----------------------------------------------------------------------------------------------------
  OFFSET  TYPE  SIZE   VALUE     INTERPRETATION       COMMENT
  -----------------------------------------------------------------------------------------------------
  00-01   INT     2    0087h     Record size (words)  0087h = 135 16-bit words = 270 bytes = 010Eh
  02-03   INT     2    0005h     Type                 Record Type 5
  04-05   INT     2    0005h     Record counter       Incrementing for each record
  06-07   INT     2    0000h     Repeat write         If errors found, system could re-write three times.
  08-19   CHAR   12              User text field      "BERGEN81005 "
  20-21   INT     2    00ECh     Day of year          00ECh = 236 => Day of Year, date = 810824 found below
  22-23   INT     2    020Ah     Minutes of day       020Ah = 522 => 8 hours 42 minutes, found below
  24-25   INT     2    000Bh     Seconds              000Bh = 11. So system time was 08:42:11.
                                                      Makes sense as gravity record has timestamp 08:42:10.
  26 ..          20   Probably LAT/LON/SPEED/COURSE 
  .....   EBCDIC encoded string of Gravity data and also Magnetometer, including date and time



Listing of Fortran software written in the 1980-ties to read and decode nav data tapes:


Listing of Fortran software written in the 1980-ties to read and decode nav data tapes.



8.3  Transit/GPS receivers


  • Ca. 1985: MX 4102 Transit Satellite Navigator
  • Ca. 1987: Trimble model 4000A GPS Locator
Magnavox MX 4400 GPS brochure
Magnavox MX 4400 GPS brochure. Click to download PDF version (3.1 MByte).

12.12.2011. Ref telco with RV "Håkon Mosby" electr. engineer 1988-1997 (JVN):

  • Transit based receivers, fix every 90 minute, more often in polar region
  • Dead reckoning interpolation between fixes
  • Shooting seismics on time, not distance
  • Integrated Magnavox navigation system, with interface to:
    • Transit sat receiver
    • Gyro, log
    • Gravitymeter
    • Magnetometer
    • Seismic recording system (DFS V)
    • Tape drive 1/2" 9-track. Data written sequentially to tape. Different type of records.
  • System should receive FFID / shooting confirmation from seismic recording system.



8.4  Tape transcription services


9.  TAPE STORAGE, STATUS 12.12.2011


  • Approx 3000 tapes read, mostly in SEG-C and SEG-B format.
  • Remaining tapes in storage room no 1 shown below. They are mainly in SEG-Y (processed data) format, or nav/grav/mag tapes written on Magnavox integrated navigation system.
  • "Mobile Search 1987" tapes are in storage room no. 2.

Click to see large version.

Click to see large version.

Click to see large version.

Nav. grav/mag data tapes on right side. Click to see large version.



10.  STATUS 4 JANUARY, 2012


  1. Processed navdata received from A.G. in December 2011. Generated UKOOA P1/90 and KML (for import in Google Earth) files.
    • Lofoten 1982 % 1988
    • Sognefjorden 1985 & 1987
    • Mobile Search 1987, 25 ea profiles (there are more in other areas)
  2. UKOOA and KML files are here.
  3. Contacted Geco, Scripps, Lamont, Off. of Naval Research re. tape record format Magnavox integrated navigation system (see above) - but to no avail

Mobile Search part A, red lines, 1987. Click to see large version.

Lofoten 1982 (yellow) and 1988 (red). Click to see large version.

Sognefjord 1985 (red) and 1987 (yellow). Click to see large version.

Tape depot as of 4 january 2012. Click to see large version.
Department of Earth Science
University of Bergen
N O R W A Y