Sonde data from the Portneuf River Monitoring Project

Data in these files were collected as part of the Portneuf River Monitoring
Project (PRMP). Support for the PRMP has been provided by many agencies, 
including ID DEQ, the City of Pocatello, the Pocatello Wastewater Treatment
Plant, J.R. Simplot Company, Three Rivers RC&D, the U.S. EPA, and Idaho State
University.

Any publications that make use of these data should acknowledge the PRMP.

The file includes ‘filtered’ data, which have been screened to exclude values
that the PRMP Technical Committee has deemed unacceptable. The file also includes 
the original data reported by the sonde and QA codes (generated by comparing
readings from a sonde that was in place with a newly calibrated sonde) so
that the user can judge whether the filtering was appropriate.

Filtering algorithms
Raw temperature data were assigned as missing (-99) if they exceeded 50.
Raw NTU data were assigned as missing (-99) if they exceeded 1000. This upper
limit was chosen based on observations made during high flow periods. We believe
that values greater than 1000 are the result of fouling of the turbidity
sensor or air bubbles associated with turbulence. 
Raw NTU data were assigned as missing (-99) if the value exceeded the average of
the previous 5 readings (including only readings that were accepted) by more
than 1.75 and also by more than 10 NTU units.
Raw NTU data were assigned as missing (-99) if they were less than zero.
All data for a given sample time were assigned as missing (-99) if the values
for each parameter were identical to the values recorded for the previous time.
There were periods of time when a sonde was not connected or initiated
properly, resulting in strings of identical data.

The file contains the following variables, in this order.

1. SITE - Site code (2-digit number)
2. YEAR - Calendar year
3. JDAY - Julian day
4. HRMIN - hour (2 digits) and minute (2 digits) when readings were taken
5. JDAYFR - Julian day + fraction that indicates time of day
6. BATTVLT - battery voltage read from sonde
7. TEMPR - Temperature recorded by sonde (R denotes ‘raw data’) 
8. TEMPQA - Temperature QA code
9. TEMPF - Filtered temperature data (degrees Celsius)
10. SpCondR - Specific Conductivity recorded by sonde
11. SPCQA - Specific Conductivity QA code
12. SPCONDF - Filtered Specific Conductivity data
13. DO%R - Dissolved Oxygen (%) recorded by sonde
14. DOmgplR - Dissolved Oxygen (mg per l) recorded by sonde
15. DOQA - Dissolved Oxygen QA code
16. DO%F - Filtered Dissolved Oxygen (%) data
17. DOmgplF - Filtered Dissolved Oxygen (mg per l) data
18. pHR - pH data recorded by sonde
19. pHQA - pH QA code
20. pHF - Filtered pH data
21. NTUR - NTU (turbidity) data recorded by sonde
22. NTUQA - NTU QA code
23. NTUF - Filtered NTU data

QA codes: 
     1. Excellent       4. Poor
     2. Good            5 or greater. Unacceptable
     3. Fair

Data that are missing or deemed unacceptable (because of low quality or because
they were excluded by the NTU data filter) were assigned a value of -99.

Result of filtering file Ficht03.CSV

             # LINES READ =  54797 
# DUPLICATE LINES DROPPED =  12374 ( 0.226 OF TOTAL)
     # NTU VALUES DROPPED =    453 ( 0.008 OF TOTAL)