1.0 INTRODUCTION

1.1 BACKGROUND

1.2 SYSTEM OVERVIEW

1.3 USER KNOWLEDGE

1.4 DOCUMENT OVERVIEW

1.5 DOCUMENT CONVENTIONS

1.6 FREQUENTLY ASKED QUESTIONS

2.0 GETTING STARTED

2.1 WHAT YOU NEED

2.2 INSTALL DATAWISE®

2.2.1 INSTALLING DATAWISE® AS A SERVICE

2.3 RUN DATAWISE®

2.4 CONFIGURE DATAWISE®

2.4.1 DEFINE DATA COLLECTION ACTIVITIES

2.4.1.1 PROCESS DESCRIPTIONS

2.4.1.2 PROCESS DESCRIPTIONS - GOES DATA COLLECTION METHODS

2.4.1.3 HOT STAND-BY/TASK LISTS TO EDIT

2.4.1.4 CUSTOM COMMANDS

2.4.1.5 STEPS FOR CONFIGURING DATA COLLECTION ACTIVITIES

2.4.2 DEFINE SENSORS

2.4.3 DEFINE SIMPLE SENSOR ALARMS

2.4.4 DEFINE MULTIPLE ALARMS

2.4.5 ADD / EDIT ALARM ACTIONS

2.4.6 DEFINING CUSTOM ALARM TEXT FILES

2.4.7 DEFINE STATIONS

2.4.8 SHEF CONFIGURATION

2.4.8.1 SHEF ENCODING CONFIGURATION

2.4.8.2 SHEF DECODING CONFIGURATION

2.4.8.3 MANUAL SHEF CONFIGURATION

2.4.9 ODBC / RDB CONFIGURATION

2.4.9.1 CONFIGURING DATAWISE® TO STORE DATA IN A RELATIONAL DATABASE

2.4.9.2 RELATIONAL DATABASE STRUCTURE

2.4.9.3 ENABLING THE PROCESS TO POPULATE THE RDB

2.4.10 SCHEDULE FUNCTIONS

2.4.11 IFLOWS CONFIGURATION

2.4.11.1 IFLOWS NETWORK SENSOR CONFIGURATION

2.4.11.2 ALERT SENSOR CONFIGURATION FOR AN IFLOWS NETWORK

2.4.11.3 ADDING THE IFLOWS BACKBONE COMMUNICATIONS FUNCTIONALITY

2.4.12 USGS DATA CONFIGURATION

3.0 DATABASE MANAGEMENT

3.1 SENSOR MANAGEMENT

3.2 EQUATIONS MANAGEMENT

3.2.1 REAL-TIME SENSOR EQUATIONS

3.2.2 STATISTICAL EQUATIONS

3.2.3 HSE VIRTUAL ICE SENSOR

3.3 RE-CALIBRATING SENSOR DATA

3.4 RATING TABLES

3.5 EXPORTING TO A COMMERCIAL DATABASE

4.0 PAGER

4.1 CONFIGURING PAGERS

4.2 SELECTING SENSORS FOR PAGING

4.3 SENDING A TEST PAGE

4.4 VIEWING A LOG OF PAGER ACTIVITY

5.0 DATA DISPLAY

5.1 SINGLE SENSOR DATA DISPLAY

5.2 TABULAR REPORTS

5.2.1 STATISTICAL TABULAR REPORTS

5.2.2 CUSTOM TABULAR REPORTS

5.3 DATA ARCHIVING

5.4 MAPS

5.4.1 DEFINING MAPS

5.4.2 DISPLAYING SENSOR MAPS

5.4.3 DISPLAYING CONTOUR MAPS

5.5 TIME SERIES PLOTS

5.5.1 TIME SERIES PLOTS IN OLD STYLE

5.5.2 TIME SERIES PLOTS IN NEW STYLE

5.6 VIEWING ANIMATED GRAPHICS

5.7 PRECIPITATION REPORTS

5.7.1 CREATE AN HOURLY (ANNUAL) REPORT

5.7.2 CREATE AN ANNUAL MAXIMUM RAINFALL REPORT

5.7.3 CREATE A DAILY PRECIPITATION REPORT

5.7.4 CREATE AN HOURLY (MONTH) REPORT

5.7.5 CREATE A YEARLY (MULTIPLE) REPORT

5.7.6 CREATE A DAYS WITH RAIN REPORT

5.8 VIEWING POLLING LOG

5.9 VIEWING SYSTEM LOG

6.0 HYDROLIC ANALYSIS AND FORECASTING

6.1 HEADWATER FORECASTING

6.2 CHANNEL/RESERVOIR FORECASTING

6.3 PRECIPITATION SCENARIOS

7.0 ALARM MANAGEMENT

7.1 VIEWING ALARMS

7.2 EMAIL ALARMS CONFIGURATION

7.3 CONTACT CLOSURE ALARMS

7.4 TESTING ALARMS

7.5 DEFINE COMPLEX ALARMS

8.0 WEB TOOLS

8.1 BROWSER INTERFACE

9.0 TOOLS

9.1 START/STOP SYSTEM - See Section 2.3 Run DataWise®

9.2 SET DATABASE SERVER

9.3 MAP SERVER ACCESS

9.4 GENERAL SYSTEM DIAGNOSTICS

9.5 ALERT STATISTICS

9.6 GOES DIAGNOSTICS

9.7 DISPLAY INCOMING GOES MESSAGES

9.8 NESDIS INTERFACE MODULE

9.8.1 INTRODUCTION

9.8.2 ISSUING RETRANSMIT COMMANDS BY LIST

9.8.3 ISSUING RETRANSMIT COMMANDS BY PDT

9.8.4 MANUAL UPDATE OF A SINGLE PDT

9.8.5 BATCH UPDATING OF PDTS

9.8.6 RETRIEVAL OF NETWORK LISTS

9.8.7 EDITING OF NETWORK LISTS

9.8.8 UPDATING OF NETWORK LISTS

9.9 CHECK AND REPAIR DATABASE

9.10 TEST ALARMS - See Section 7.4 Testing Alarms

9.11 GRAPHIC SCREEN DESIGN

9.11.1 GETTING STARTED

9.11.2 IMPORTING AND PLACING BITMAP IMAGES

9.11.3 VIEWING SELECTED IMAGE COMPONENTS

9.11.4 SETTING COLORS FOR DRAWN STATIC COMPONENTS

9.11.5 ADDING A SENSOR READOUT

9.11.6 MODIFYING OR DELETING A SENSOR READOUT

9.11.7 MOVING A SENSOR READOUT

9.11.8 DRAWING COMMANDS

9.11.9 SAVING CHANGES

9.11.10 CREATING ANIMATED GRAPHICS AND MAP BACKGROUNDS FROM DFX FILES

9.12 SEND FAKE DATA

9.13 DRIFT RECALIBRATION

9.14SET USER LEVELS

9.15 EXPERT TOOLS

10.0 MANUAL ACTIVITIES

10.1 MANUAL INTERROGATION

10.2 PROGRAM RTU'S

10.3 IMPORT ASCII DATA

10.4 RE-PROCESS LOGGED GOES MESSAGES

10.5 HSE LIGHT/GATE CONTROLS

10.6 DISPLAY INCOMING DATA

10.7 SET TIME ZONE

11.0 ADDENDUM

11.1 MODIFICATION TO SYSTEMRUNNING

 

 

1.0 INTRODUCTION

1.1 BACKGROUND

DEC Data Systems develops environmental data acquisition and control software for the 32-bit Windows environment. The basic DataWise® software package is able to collect, analyze, and manipulate data from most types of telemetry currently on the market. Also, custom programming and system programming can be done to incorporate new data acquisition modules and analyses, and to display routines and screens to suit your unique needs.

DEC Data Systems personnel have been working in the data acquisition and control industry for over 25 years, with expertise in meteorology, hydrology, and real-time computer programming. Their hands-on experience includes hydrologic modeling for watersheds, channel routing, and reservoir management, using the SSMA model and various other hydrologic routing models. DEC Data Systems understands the complex system requirements for successful environmental data acquisition and database management. This expertise is reflected in the DataWise® software product.

 

1.2 SYSTEM OVERVIEW

The DataWise® software general-purpose package was designed for and optimized to run under all 32 bit Windows operating systems. DataWise® software provides you with the following functions, which are further described in their respective Chapters:

 

1.3 USER KNOWLEDGE

To use the DataWise® system, the primary user should have a good working knowledge of Windows-based, personal computer systems/workstations, particularly Windows NT or Windows 2000. The user should also have a good knowledge of the types of functions you want the DataWise® database to perform. That is, you should have a plan, which defines the following:

To help you (where needed), each chapter will start by listing data you must have on hand to perform the steps in that chapter.

 

1.4 DOCUMENT OVERVIEW

This document is a Users’ Manual with explanation of DataWise® and steps on how to work through and with the program. The document is organized initially by startup and set up activities and by major functions. Following that is progressively more detailed information, such as detailed explanations of specific areas and system diagnostics. This is a "Foundation" Users’ Manual. For each customer’s specific implementation, DEC Data Systems engineers can provide an addendum to this document for unique implementation. Your system may have such an addendum or you may request an addendum be developed for you by contact DEC Data Systems. Specifically, information may be found as described below.

CHAPTER 2 How to install, set up, and get started using DataWise®, including configuring sensors and stations.

CHAPTER 3 Database Management

CHAPTER 4 Pager

CHAPTER 5 Data Display

CHAPTER 6 Hydrologic Analysis and Forecasting

CHAPTER 7 Alarm Management

CHAPTER 8 Web Tools

CHAPTER 9 Tools

CHAPTER 10 Manual Activities

 

1.5 DOCUMENT CONVENTIONS

This Users’ Manual is set up to be easy to use and easy for you to understand the commands and functions you are asked to perform. To aid in this goal, the following standard set of conventions is used.

 

 

1.6 FREQUENTLY ASKED QUESTIONS

 

Are there any issues transferring data from the Datawise native binary format to another database in the event of a system migration?

All data in the DataWise native database can be easily transferred to any relational database in which an ODBC connection can be made. Known databases that can be used are MS-SQL, Oracle, Access, and MySQL. All data in the DataWise native database can also be easily transferred to a HEC-DSS. Additionally, data can be exported to text files in a variety of formats suitable for importing to various databases.

 

What is involved in configuring the Datawise system to perform back up to an external DB?

The following steps are all performed from a menu-driven, DataWise Dialog:

1) Specify an ODBC data source name.

2) Specify the sensors for which it is desired to store data into the external DB.

3) Specify the form in which the data is to be stored. That is, a) in raw data form as it is received, b) in processed form after it has been converted to engineering units, c) whether period values for rainfall should be stored (e.g, 15-minute rainfall, hourly rainfall, etc.) Note that data for a single sensor can be stored in multiple ways.

Once the above steps are performed, data transfer to an external DB is automatic and performed in real-time unless the user specifies that it should be done periodically at some user-specified interval or on demand.

 

How often do you need to migrate data to an external DB that is used for queries?

DataWise can be configured to:

1) “migrate” data to an external DB in real-time so that whatever data is in the native DataWise data base is also in the external DB at all times.

2) “migrate” data to an external DB at any user-specified interval.

3) “migrate” data to an external DB on demand.

The preferable way is usually to populate an external DB in real-time so that at any time the external DB is queried it will contain up to date data.

 

Why does Datawise store data to a binary database? (Advantages and Disadvantages)

Advantages:

1) Relational databases are not efficient in storing un-evenly spaced time series data, which is what data in an ALERT / IFLOWS system is. This becomes especially true as the time series becomes long. The native DataWise database is highly optimized for rapid handling of long time series data for multiple sensors. As a point of interest, even though the U.S. Army Corps of Engineers has invested heavily in optimizing the Oracle RDB for storing time series data for its hydrologic operations, the non-RDB HEC-DSS still outperforms the optimized Oracle RDB by nearly two orders of magnitude.

2) The DataWise native database has been optimized for detecting and correcting erroneous data (which can and does occur in ALERT systems).

3) The DataWise native database is optimized for detecting sensor data that has gone into user-specified alarm conditions and for performing some external function when alarm conditions occur.

Disadvantages:

Clearly, accessing the native DataWise database cannot be performed directly using standard RDB methods. If DataWise did not populate external databases, this could be considered a significant disadvantage. However, since DataWise can easily maintain a “parallel” data set in a RDB, the disadvantages are relatively minor.

 

Who helps us establish proper backup procedures for the Datawise base station?

DEC DataSystems personnel provide this service as part of the support provided when DataWise is purchased.

 

Could you review the process for creation and modification of display maps for the base station?

1) Select the type of map background for placing sensor data on. Up to 100 different maps can currently be supported in DataWise. Map backgrounds can either be raster graphics (e.g., .bmp, .jpg, .gif files) or vector graphics. If using a raster graphic map background, the user must specify the coordinates (latitude / longitude) of the map boundaries.

2) For each map to be displayed, specify the sensors to be displayed on the map. Sensor positions are initially determined by their coordinates although sensors can be re-positioned on a map-by-map basis if desired.

 

Review the analysis capabilities of the base station. What applications will we be able to transfer data to and what special configuration requirements will need to be met to make them work?

Analysis capabilities include, but are not limited to:

1) Tabular data in fixed-format displays (static or realtime)

2) Tabular data in user-specified format displays (static or realtime)

Note that a wide range of statistical analysis can be performed on any tabular data value before it is displayed.

3) Tabular reports displaying statistical analysis of rainfall data including maximum observed intensities for various time periods, hourly, daily, monthly, and yearly rainfall reports.

4) Time series plots of data from single or multiple sensors. Plots can be either static or dynamic.

5) “Animated graphics” displays in which users can design their own graphic displays (maps, schematics of weather stations, etc.) These screens can contain a wide range of “widgets” including gages, dials, thermometers, stream cross-sections, time series plots, simple text values. All displays update in realtime.

6) Extensive mapping capabilities including isohyetal maps which display rainfall similar to radar intensity maps.

Note that all displays that are also available remotely either through the DataWise client software or via a standard browser.

 

What is required to integrate QPF data into the base station for the generation of alerts?

A connection to a data stream (e.g. NOAA-port) that contains NWS-QPF data.

 

How does the station handle visual alerts? What do they look like?

DataWise generated alarms (alerts) can be delivered in numerous ways, from visual alerts on computer screens to text messages to controls of an external device. Station visual alerts can be delivered to not only the master DataWise base station but also to every computer on the network running the DataWise client software or every computer with a browser pointed to the DataWise Web server. Visual alarms can be customized or standard. In standard mode, an alarm display will pop up over the top of any and all applications currently running. The display provides a tabular list of all sensors in alarm condition and clicking on a specific sensor in the list will show the reason for the alarm condition. If a map is being displayed and the map contains a sensor that is in alarm condition, the sensor data will turn red. Note that an audible sound can be associated with an alarm. The sound can be customized to execute any desired .wav file on a sensor / condition by sensor / condition criterion or just a standard alarm tone.

 

Does the station log both meteorological and administrative alerts that are triggered?

All alarms (alerts) are logged.

 

How do we access the logs?

Through standard DataWise displays.

 

NWS alerts what is required to configure the station to ingest data from the NWS? Would there be firewall issues associated with collection or transfer of data from outside sources?

Through the TCP/IP socket mechanism over the internet using NOAA-port or something equivalent. DataWise comes standard with software to ingest such information. A port will need to be opened through the firewall.

 

Integration with web and GIS.

DataWise comes standard with a Web server which allows access to all displays, both tabular and graphic (including realtime updating graphic) displays, using a standard browser. The DataWise Web server code can be modified / customized by any web programmer if desired. GIS integration has been done through a RDB. Best to talk about this on the phone or in person.

 

 

2.0 GETTING STARTED

This chapter explains what data you need to get started, how to install DataWise® and how to create a database and how to configure the database for your system. It also tells you how to start and run the program. Because this is the foundation for DataWise® successfully running with your system and providing the critical functions you need, it is very important that you set up the system completely and accurately. Before ever installing DataWise® or working with the database, you will need to know what DataWise® needs to operate successfully and to know that your system meets these needs. You must also have data about your system with you as you sit down to configure DataWise®. Section 2.1 explains what you will need to install and configure DataWise® successfully. Throughout the remainder of this chapter, for each task, the User’s Manual reiterates what data you will need to perform tasks in that section.

 

2.1 WHAT YOU NEED

Before getting started, and to configure DataWise® for use, you will need to have the following:

In addition, DataWise® is distributed on CD-ROM or Zip disks (herein called the install disk). Therefore, you must have a CD-ROM or Zip Drive.

To follow the steps in this section for installing, running and configuring DataWise®, you will need the following data about your system close at hand.

     

    2.2 INSTALL DATAWISE®

    DataWise® can be loaded from a CD-ROM or Zip disk (herein called Installation Disk).

    To install DataWise®, place your Installation Disk in the appropriate drive then follow the steps below to install DataWise®.

    INSTALL DataWise®.

    CHOOSE Start >Run.

    SELECT the appropriate Drive letter for your CD-ROM or other drive which contains your DataWise® installation disk.

    DOUBLE CLICK on the file labeled dwinstall.exe.

     

     

     

     

     

     

     

     

    NOTE if you get this message saying " TZ Environment Variable Not Set ”, you may continue with the installation, then follow instructions in Section 2.3 for setting the TZ Environment Variable.

     

    SELECT the Drive where you want DataWise® installed.

     

    CLICK on New DataWise® Install.

     

    CLICK on Yes to continue with the installation.

     

     

     

    DataWise® will then be installed on your system.

    The DataWise® installation program does not automatically create a shortcut. To create a shortcut, follow the steps below.

     

    CREATE A SHORTCUT
    to the Main DataWise® Menu

    GO TO c:\dw\utils\menu.exe

     

    CLICK the right mouse button on the menu.exe file.

     

    SELECT New Shortcut. A Shortcut to the menu file is created at the bottom of the folder.

     

    CLICK & DRAG the Shortcut to your desktop.

     

     

     

    2.2.1 INSTALLING DATAWISE® AS A SERVICE

    INSTALLing DataWise® as a Service

    From Start / Stop Screen

    CLICK the on the Start/Stop System icon on the Main DataWise® screen. The Start/Stop screen is displayed.

    SELECT the desired DataWise® Service State:

    Automatic if you want DataWise® to start at system boot up and continue to run whether or not users are logged in.

    Manual if you prefer to start the service manually.

    Disabled if you want to disable the service.

    Not Installed if you do not want DataWise® to run as a service.

     

     

    An information screen will appear saying, "Service Put into Automated Mode" if you selected Automatic. DataWise® will now start up at system boot.

    "or"

    "Service Put into Manual Mode" if you selected Manual. In this case you will need to start the service manually.

     

    SELECT the "When running DataWise® As a Service" method desired.

    The preferred method should be to run under the Local System account. There are times when this is not possible, and those cases are if DW must connect to a relational database OR must access network resources. In that case, if DataWise® is running as a service it must run under an account that has access to the required resources.

    CLICK on the "Save" button to save your selection.

    INSTALLing DataWise® as a Service

    Manual Setup

    Bring up a "command prompt" and change directories to \DW\Utils

    EXECUTE "installdwservice -install [d=drive]"

    Where drive is the drive on which DataWise® is installed (default drive is C:\)

     

    CHOOSE Administrative Tools then Services from the Control Panel.

     

    RIGHT CLICK on "DataWise" to set the properties to be:

    • Startup type to Automatic
    • Logon as Local System
    • Allow the service to interact with the desktop

    DataWise® will now be installed as a service. It will start at system boot up time and continue to run whether or not users are logged in. Alarms will be properly generated and displayed.

     

     

    2.3 RUN DATAWISE®

    To run DataWise® the first time, follow the steps below. If running under Windows NT, XP, or 2000 this step is unnecessary provided the TZ Environment Variable has been set. If it has not been set then go to Control Panel > System > Advanced. Click on Environment Variables. Under System Variables, click on New. For Variable name enter TZ. For Variable Value enter time zone variable in this format. First 3 characters are the time zone name (ex. PST for Pacific Standard Time). Next is the # of hours earlier that your time zone is from Greenwich Time. Next if you want to automatically change to and from Daylight Savings time, you must enter the abbreviation for your local daylight savings time zone, ex. PDT for Pacific Daylight Time, EDT for Eastern Daylight Time or MDT for Mountain Daylight time, etc. For example, if you are on Pacific Standard Time and want to change automatically to and from Daylight Savings Time, you would enter PST8PDT for the variable value. If running under any other operating system follow the steps below.

    RUN DataWise® for the first time.

    DOUBLE CLICK on the menu Shortcut you created. This will open the program to the Set Time Zone Variable screen.

    CLICK on Yes. the Set Time Zone screen will be displayed.

     

    ENTER your correct Time Zone Variable. This allows DataWise® software to compute the difference between GMT and local time. .

     

    After this first setup, to run DataWise® at any time, follow the steps below.

    RUN DataWise®.

    DOUBLE CLICK on the menu Shortcut you created. This opens the program to the Main DataWise® Menu, shown below. On the menu is a series of 11 icons. The last icon looks like an alarm clock. This is the Start/Stop System icon.

     

    CLICK the on the Start/Stop System icon. The Start/Stop DataWise® screen is displayed.

    CLICK on the Push To Start button. You get the following message if done correctly:

    DataWise® is Running.

    CLICK on Exit, but DataWise® continues to run even after exiting.

    NOTE

    To stop DataWise®, you must go back into the Main DataWise® Menu, CLICK on Stop/Start icon; and SELECT Push to Stop.

     

     

    2.4 CONFIGURE DATAWISE®

After DataWise® has been installed, there are five basic tasks (each with an associated set of steps) to initially configure DataWise® to function with your system.

Steps for these five basic tasks are provided in the following sections. Each section will tell you what data you will need available to configure the system for that task.

       

      2.4.1 DEFINE DATA COLLECTION ACTIVITIES

Defining your collection sources correctly is critical to make DataWise® work effectively. You also will need to become familiar with the Real-time Data Collection Task Editor, shown below, which is the primary tool for defining data collection activities.

DEFINE Data Collection Sources.

From the Main DataWise® Menu Choose Configure > Data Collection Activities.

This starts the Real-time Data Collection Task Editor.

Before giving you the steps needed to define data collection sources (in section 2.4.1.5), the following subsections breakdown and give you an overview of the major functions of the Real-time Data Collection Task Editor. These include:

Subsection 2.4.1.5 contains the steps for using the information in subsections 2.4.1.1 through 2.4.1.4 to configure DataWise® for data collection activities for your system.