 |
|
|
||||
| USGS Earth Resources Observation and Science Center (EROS) |
1.0 Purpose
The purpose of this procedure is to define the responsibilities and operational requirements when operating the AVHRR Data Acquisition and Processing Systems (ADAPS).
2.0 Scope
This procedure applies to those Branch of Computer Services personnel during operation of the ADAPS system for data acquisition, archive generation, software development, and hardware maintenance. The Branch of Data Services personnel are responsible for acquisition scheduling, data base management, and product generation.
3.0 General
The U.S. Geological Survey (USGS), together with the National Oceanic and Atmospheric Administration (NOAA), has developed a system at the EDC Data Center (EDC) to process and archive Advanced Very High Resolution Radiometer (AVHRR) data received directly from NOAA TIROS polar-orbiting satellites with nadir coverage possible over an 1,800 mile radius of EDC. Foreign data is also routinely collected via domestic satellite (DOMSAT) from either Wallops Island, Virginia or Fairbanks, Alaska.
Scientific investigations have indicated that significant global earth science information can be derived from 1 kilometer AVHRR image data. NASA has also identified this data set as critical to the preparatory studies for the Earth Observing System (EOS). As a result, NASA headquarters has asked USGS at EDC to coordinate the preparation of such a data set as part of its role as the EOSDIS/Land Processes Distributed Active Archive Center (DAAC).
AVHRR data collection efforts by NASA, NOAA, USGS, the European Space Agency (ESA), and 26 international AVHRR ground receiving stations facilities are providing a global 1-km AVHRR data set over all land surfaces and coastal zones using NOAA's TIROS "afternoon" polar-orbiting satellites on a daily basis.
ADAPS consists of three computer systems tied together via local area network, included are the MicroVax 3800, MicroVax 3900, and Silicon Graphics Challenge series. The MicroVax 3800 consists of an antenna, receiver subsystem, high resolution video display monitor, and a high density tape recorder (HDTR).
The MicroVax 3800 computer system performs all satellite acquisition requirements for EDC HRPT, and DOMSAT HRPT and LAC data received.
The Silicon Graphics does all AVHRR data ingest processing, archive generation, and product generation tasks. Software utilizes Land Analysis System (LAS) modules and EDC AVHRR Satellite Geometric, Radiometric, and Atmospheric correction modules developed by the staff at the EDC Data Center. Peripherals include 3480, 8mm, 4mm, and 9 track tape drives.
During data processing a positive print is made to evaluate acquisition image quality and cloud cover assessment for continued on-going projects within the EDC Data facility.
The MicroVax 3900 is used to produce laser paper/film products on a Harris recorder. Microfiche products are produced on a MDA Colorfire film recorder.
After acquisitions are received and processed, the AVHRR data is copied to archive quality 3480 computer tape for permanent storage in an environmentally controlled area.
Acquisition data are catalogued, metadata are entered into an information management system (IMS), and a reduced resolution browse image is created from each observation. Data users worldwide can access the IMS to query, browse, and order any observation at the marginal cost of reproduction. Participants in the project receive data free of charge in the quantity equal to their contribution.
4.0 References
Computer Operations Section Memorandums:
OP-019 "3480 Procedures"
Operator's Manual for Harris Laserfax Microprocessor Recorder Operation and Maintenance Instructions for Model 410WA Basic Receive Unit, March 1987, Scientific-Atlanta, Inc.
5.0 Responsibilities and Procedures
The successful operation of ADAPS involves coordination and cooperation between the three primary user groups and the Computer Operations Section. ADAPS can be defined into four major functions, each function the responsibility of a different user or support group.
Customer Services (CS) Section is responsible for data acquisition, scheduling, and order entry functions.
Digital Data Production (DDP) Section is responsible for product generation.
Computer Operations (CO) Section is responsible for data acquisition, processing, and archiving.
Data Management (DM) Section is responsible for the administration of metadata and data base management for information management systems. AVHRR data, received at EDC via mail services from participating international ground receiving facilities, shall be scheduled for processing and coordinated by DM.
5.1 Overall Operations
A design goal of ADAPS was to develop a system which could perform all tasks with minimum operator intervention, to provide an unattended data acquisition capability, and to provide a user friendly front-end to the applications software and associated processing tasks. All of these goals were met in ADAPS. ADAPS operates under the umbrella of the Transportable Applications Executive (TAE). This allows the man-machine interface to ADAPS to be menu driven. The ADAPS operator selects the processing function to be performed using the ADAPS MAIN MENU.
The remainder of this document is organized into functional elements of the processing flow, as related to the ADAPS MAIN MENU. Detailed documentation for each MENU selection is found in appendices.
5.2 Orbital Element Updating
The AVHRR data acquired, processed, and archived by ADAPS is received from NOAA TIROS polar-orbiting satellites, therefore, ADAPS must be programmed to track the satellites while in view of the EDC earth station.
Two separate forms of satellite ephemeris data are routinely used to establish proper satellite to earth orientation. One-line data elements received and updated daily from Naval Space Surveillance (NAVSPASUR) are necessary to establish the earth reference in order to achieve accurate map overlays.
Two-line data elements received daily from the North American Defense Command (NORAD) are necessary to properly position the antenna during tracking of the satellites in their orbiting configuration. The Antenna Control Unit (ACU) is part of the MicroVax 3800 system configuration and performs satellite signal receiving capabilities.
5.2.1 Antenna Control Unit Ephemeris Data
The two-line ephemeris data is received daily, but current procedure requires the use of the Thursday data only. The procedure for loading the Thursday ephemeris data is called Weekly ACU Updating.
Inherent errors will occur when converting the two-line data into ACU parameters due to limited memory and rounding methods. These inherent errors tend to degrade the satellite signal strength during receive mode. A procedure called Daily ACU Updating is performed after the Weekly ACU Update to resolve the inconsistencies.
5.3 Acquisition Scheduling
AVHRR data acquisition scheduling is the responsibility of the Customer Services Section. A Customer Service Representative will update the schedule, producing a listing of all possible acquisitions. After this listing is created the Customer Service Representative updates the Master Schedule with the selected passes that fall within the specified criteria. The Customer Service Representative will use a program called SCHEDIT-MODIFY to update the Master Schedule as to which scenes have orders placed against them. The Master Schedule, which includes the next seven days acquisitions at minimum, must be generated by close of business on Friday each week. The Master Schedule may be updated at any time to respond to customer orders. The Master Schedule has room for 600 entries, including both local and domsat entries to be acquired and those entries desired not to be acquired. Operations will use SCHED-DELETE to delete any entries over three days old, so the entry limit will not be exceeded.
5.4 EDC HRPT Direct Receive Data Acquisition
There are two methods used to acquire HRPT direct receive acquisition AVHRR data. The two methods are Normal Mode and Manual Mode. The Normal mode of operation is used Monday through Sunday when operations personnel are either on duty or off duty during weekends and holidays. The Manual mode is used is used when the MicroVax 3800 computer system or critical components become inoperable.
5.4.1 ADAPS Pre-acquisition Checklist
It is important that prior to the start of ADAPS operations the equipment be certified. It is recommended that the ADAPS operator go through the checklist at the beginning of each shift. The checklist must be performed immediately after a maintenance period or engineering tests.
5.4.2 Normal Operation Mode
The acquisition of data is an automatic real-time procedure that is driven by the Master Schedule. During Normal Operation Mode, the data is received and transferred directly to disk. While data is being acquired and written to disk it is also displayed on a color display to provide the computer operator with a real-time indication of scene quality and antenna performance.
A design goal of ADAPS was to provide a backup capability of the acquired data to high density tape (HDT). The computer operator must update the Master Schedule to indicate to the ACQUIRE program that the data is to be written to HDT rather than disk only. The operator has the flexibility to change the mode of operation, i.e. dish to tape, as well as other options required to completely achieve necessary results.
5.4.2.1 HDTR Setup Procedure
Do not manually position the HDTR at the desired footage setting or file position. Use the symbol DRIVE under the [POPS] account on the MicroVax 3800. If it is necessary to reposition the HDT use DRIVE to clear any possible glitches. The operator has the option to dismount the current HDT and mount a new tape or position the HDT at a desired file and track. When the HDT needs to be repositioned and the MicroVax 3800 computer system is not available, the MicroVax 3900 can be used by switching a BLACKBOX to the MicroVax 3900 configuration.
5.4.3 Manual Acquisition Procedure
The Antenna Control Unit on ADAPS is a micro-processor controlled device. The ACU can still run in a stand-alone mode to track the satellites when the remote computer system is not operational. This allows the data to be recorded on HDT even though the CPU is unavailable. When using the Manual Acquisition Procedure, the operator must be present to ensure proper set- up of the HDTR. The operator must remain on site to monitor the acquisition and then to document on the AVHRR Inventory Log form, the date and ZULU start time of the acquisition. This information is required for future data acquisition from HDT.
5.4.4 Acquire Data from HDT
Before any acquisition can be copied from HDT to disk the Master Schedule must be updated with the proper status codes by using the Schedule Editor. The Schedule Editor provides the option to change the mode of operation, i.e. HDT to disk, as well as other options.
The computer operator must be cognizant of any real-time passes scheduled to disk when transferring data from HDT to disk. Once all passes have been acquired from HDT, the computer operator must reposition the HDT to desired file position or footage selection using 'DRIVE' command. It may be necessary to forward the HDT to the Digital Archive for degaussing and subsequent reuse.
5.5 DOMSAT HRPT/LAC Data Reception
All HRPT and LAC data that is acquired by Fairbanks, Alaska and Wallops Island, Virginia ground stations is transmitted to NOAA's satellite operation control for relay to EDC via DOMSAT using a relay/reception system. The data is handled in the same manner as an EDC direct receive data acquisition, the data is transferred directly to disk.
5.6 Global Earth Science Facility AVHRR Data Contribution
Participating earth science facilities send their AVHRR data to EDC by mail services or electronically for processing. These data provide daily Global 1-KM Land Coverage. The software can handle various types of input magnetic media, i.e. 9 track, 3480, 8mm, and 4mm computer tape.
5.7 Network File Transfer from Receiving System to Processing System.
All direct EDC HRPT and DOMSAT received acquisitions are transferred from the MicroVax 3800 acquisition computer system to the Silicon Graphics processing computer system using a high speed Multinet transfer program with transfer rate of 430 kb per second.
A command procedure, LOCALCOPY.COM, copies acquisitions from the acquisition disk to a different disk directory structure, where NETCOPY.COM command procedure will use Multinet to transfer the files from the acquire computer system to the processing system. Two LOCALCOPY.COM command procedures are always active on the acquisitions system. One is used for EDC direct HRPT acquisition and Wallops Island, Virginia DOMSAT passes. The other services Fairbanks, Alaska DOMSAT acquisitions.
5.8 AVHRR Data Acquisition Processing
Data processing encompasses those tasks necessary to process an acquisition from one of several input formats and media types, including direct receive HRPT at EDC, domsat receive HRPT or LAC at EDC, electronically transferred data from another direct receive facility, or HRPT data mailed to EDC on magnetic media from a global earth science information participant. A routine called TPACQUIRE is used to extract raw data from magnetic media to disk files on the processing computer system.
Data processing will include ingesting, archiving the acquisition in the standard EDC archival format, create positive facsimile print, generate microfiche product, browse image, and metadata.
Batch jobs are continually scanning directory structures for incoming data to perform processing requirements to complete the acquisition workflow process.
TPACQUIRE acquires AVHRR images from tape media to disk. A sequential or random set of images may be selected from each tape.
CKTRANS checks for images that have been transferred over the network from specified remote receiving stations. Data is verified for valid format requirements before data is moved to the appropriate disk structure for processing. Several participants are Louisiana State University (LSU), Buenos Aires, Argentina (through Miami University), and the National Snow and Ice Data Center (NSIDC).
INGEST_DAEMON ingests raw data from the acquire directory and writes to the ingest directory in a standard EDC archival format disk file and deletes the raw image from the acquire directory.
REFAID_DAEMON converts in the image in the ingest directory to LAS format, maps the data, overlays line work, and adds annotation blocks, a grey level wedge, timestamp, and generates a facsimile image disk file, browse image disk file, microfiche image disk file, transfers a copy of the archive image to the StorageTek NEARLINE mass storage system for future processing requirements, and initiates the creation of the metadata data base record.
FAXTRANS_DAEMON will transfer facsimile image disk files to the Vax 3900 computer for facsimile print generation on the Harris recorder.
DBTRANS transfers completed metadata data base records to the GLIS server for information management systems (IMS).
BWTRANS_DAEMON will transfer the browse image disk file to the GLIS Server for IMS.
MICRO_DAEMON will merge twenty-four microfiche disk images into one large disk file, copy the large disk file to 3480 computer tape for processing on the MDA Colorfire Film Recorder, and create microfiche metadata data base records.
MICTRANS transfer completed microfiche data base records to the GLIS Server for IMS.
UPNAV batch job will automatically processes ephemeris date and provides the operator with a printout to perform Daily and Weekly ACU updates.
AUTOSTACK is a batch job submitted by the operator as needed to copy processed images to ansi-labeled 3480 archive tape. The archive files are deleted from disk after a successful archive tape is created. An update occurs to the metadata modifying the storage location field within the metadata data base record signifying completion of processing for this scene.
PRODGEN_DAEMON determines if an AVHRR archive images meets the criteria for any standing order and generates the necessary products. This eliminates the need for human intervention when an order is placed for an area of interest over an extended period of time. An example of this is, the National Snow and Ice Data Center has requested all acquisitions that have any coverage data over the North and South Poles.
AUTOCOPY is a batch job submitted by the operator to continually check for acquisitions that need to be copied to 8mm tape for the European Space Agency (ESA). ESA will get a copy of all data they did not provide to the global 1-km AVHRR data set. REFAID processing will determine whether a copy of the processed acquisition needs to made for ESA.
HALFORB processing is the mechanism of combining all daytime acquisitions of the same orbit of the earth together. The daytime viewing period is equal to a half orbit, the other half of the earth is dark. The combination of scenes is referred to as stitching. An orbital stitch list provides a group of scenes that will be retrieved from the mass storage device, otherwise commonly known as the Unitree Central File Manager (UCFM). Each scene will be processed by LNDETECT to remove any spot noise and to drop bad lines. The scenes are then stitched together to form an image containing a half orbit of data. The stitched image is copied to the mass storage system and is also copied to 3480 archive tape. A browse image and metadata data base record are also produced. A production data base source link is generated to reflect which scenes have been stitched into a half orbit.
ORBTRANS transfer completed HALFORB data base records to the GLIS Server for IMS.
At 2300 each night LOGGERS and all metadata data base transfers will be run.
5.9 Product Generation
Two standard products can be produced; LEVEL-1B and Geo-registered CCT digital computer tapes. The system generates computer tapes identical to the LEVEL-1B tapes produced by NOAA; these tapes are for users that require raw data. Users may also order tapes containing a scene which has been registered, calibrated, and processed to a projection supported by National Mapping Division's (NMD) General Cartographic Transformation Package (GCTP). Users ordering this product may also request a normalized difference band and registered World Data Bank 2 (WDB2) line work. The line work may be embedded into the image data or be included as a separate band.
The Computer Operation Section in support of the Digital Data Production Section will provide tape mounting and console operation services on other computer systems for product generation as well as on the AVHRR processing system. A product for example may include operations support for NALC, North American Land Coverage. Generally, the products will be created on canister tapes and tape labels are automatically produced on the label printer using information provided by the users.
HALFORB stitched scenes are used as input for Digital Data Production Section project work. These scene can be retrieved by the UCFM on the mass storage system. The scenes that reside on the mass storage system do not need any operation intervention. Data will remain on the mass storage system for ninety days, then will automatically be deleted. This provides for a efficient data organization management scheme of the data being placed in the mass storage system.
5.10 Processing LOGS
A software design requirement of ADAPS was to pre-determine the SCENE-ID of a given acquisition based upon the information available when generating the Master Schedule. The approximate SCENE-ID is known before a pass is acquired and is then converted to the exact SCENE-ID after INGEST. This SCENE-ID will be used for all subsequent processing and logging during each processing step. The SCENE-ID can be cross referenced to query pertinent information about each phase of processing. For example, TPACQUIRE, INGEST, and REFAID log files can be checked for errors, missed lines, program completions, and time stamps. The log file is an ASCII text file and can be manipulated for management reports and statistics.
5.11 Recovery Procedures
Detailed recovery procedures for each process is found in the ADAPS recovery procedures manual.
5.12 Preventive Maintenance
Operator and engineer preventive maintenance schedules have been defined in CSB-OPS directives.
5.13 Powerdown Procedures
Defined in CSB-OPS directive.
5.14 Antenna Stowing
Although the AVHRR antenna is rated by the manufacturer to withstand (at 32 degrees Fahrenheit) a wind velocity of up to 85 mph while tracking, and 110 mph while stowed, it may become necessary to stow the antenna under wind conditions less than 85 mph. The long-term benefits from such a conservative policy may be apparent 5-8 years from now, as a safeguard to reduce wear and tear on bearing surfaces and gear faces.
5.15 Basic Guidelines for Bringing the System Down
On the acquisition computer system (MicroVax 3800) be prepared to manually record EDC scenes to HDT. Fairbanks and Wallops scenes will be missed. Before shutting down the system, check frame syncs to see if a pass is locked up. Let the current acquisition finish.
On the processing computer system (SG4), INGEST, REFAID, BWTRANS, UPNAV, and FAXTRANS can be aborted, although some intermediate files may need to be cleaned up. Use STOPMIC, STOPSTACK, STOPCOPY to allow these processes to terminate gracefully. After batch jobs and interactive processes are down; a SHUTDOWN can be initiated.
| AccessibilityFOIAPrivacyPolicies and Notices | |
  |
|
