Background
Data in Water has varying levels of precision that is needed. For example, if we're measuring the depth on a 20 ft deep tank, measuring down to the millimeter probably isn't needed, but down to a tenth of a foot is probably more appropriate. Similarly, if we're measuring pH, usually one or two decimals (i.e. 7.1 or 6.97) is probably more appropriate than 4 decimals (i.e. 7.12935). The reasons for this depend on the Detection Limits of instruments, the physical limitations of things like tank gauges and scales, as well as the requirements from regulators or downstream calculations that aggregate one or more value.
Additionally, best practices in math and science dictate that when we are doing math on values to calculate something else, we typically want to keep the highest level of precision that we have available through the calculation. This article explores some of the concepts involved in both deciding what level of precision is appropriate to store, as well as how you can change the level of precision for downstream calculations taking into account the various level of significant figures in your upstream calculations. We will unpack how we can display and report precision different than we store precision.
Full Precision Data Handling Explained
Taking “Reasonable” Readings
Sometimes we see operators taking readings that are beyond the “natural precision” of an instrument. For example, if you are trying to read the level of a 4 ft (or similar) diameter chemical tank like shown here, chances are, you can make the reading within maybe a half inch or so. For a small, <1ft diameter tank, you would probably want quarter inches measured. It also depends on how quiescent (settled) the tank is…tanks that have faster addition or withdrawals will sometimes have a liquid level that will “slosh” a bit and make very precise readings difficult.Or maybe you can take a measurement of a tenth of a foot. You cannot, reasonably, take a measurement down to a hundredth of a foot. Additionally, sometimes we see SCADA integrators and other instrumentation professionals who will configure a sensor to show you hundredths of a pound when you typically are measuring dozens or hundreds of pounds of usage each day. In cases like this, we encourage users to consider the reasonableness of the precision that is selected. Following is Waterly’s list of typical recommended levels of precision when it comes to measuring typical plant metrics in the water/wastewater industry:
- Flow (plant flow meters >4”): MGal to 3 decimals (whole kGal)
- Flow (small diameter meters): Whole Gallons
- Distances: Tenths of a Foot or Half Inches or Centimeters
- Weights: Tenths of Pounds
- Time: Whole Seconds
Waterly's Implementation of Precision
Displayed Data, Stored Data, Reported Data
To enhance readability, Waterly presents a rounded version of your data in data entry and on reports. This rounding is purely for display purposes, and all underlying calculations utilize the complete, unrounded Full Precision data.
Waterly Calculated Metric Example:
Consider this example scenario where Waterly is configured to display only 3 digits for a Metric that gets its data from a SCADA system.
Metric Value Input from SCADA: 1.4234
Waterly Displays: 1.423
When there is Calculation Metric that multiply Value by 2
Calculation: 1.4234 x 2 = 2.8468
Waterly Displays: 2.847
Here, Waterly uses the exact figure of 1.4234 for calculations but displays a rounded figure of 1.423 as configured, but when multiplied by 2 (to display an intermediate value) the actual result is 2.8468, but displayed as 2.847 as configured. Had we rounded the value without full precision before multiplying by 2, we would get an answer of 2.846. You may be saying that the difference is minuscule, but if you then are looking for the minimum, maximum, or average of these…say daily numbers, that error can propagate further. By maintaining full precision behind the scenes, a calculator (or spreadsheet, or Waterly) minimizes the downstream impact (inaccuracy) of rounding. Here is a visual example in Waterly that shows the data above.
Click HERE learn how to adjust the number decimals to display in Waterly
When Precision can get Confusing in Watelry
When you have an INPUT set to one precision and a calculation that is set to another, data can look confusing. See the example below where the Source Metric says 1.511 and the Max says 1.51 (that's pretty easy to understand), but now see #2, where it shows 2 because its precision is set to 0. Then look at the "Above Value x 2" result, which one might expect would say 4, but it says 3.022. That's because it is multiplying 1.511 (Source Metric) by 2, which results in 3.022 and NOT 4.
Summary
Accuracy: Your interim calculations are based on Full Precision data, ensuring the reliability of outcomes.
Clarity: By displaying rounded numbers, Waterly makes data easier to read and interpret.
Consistency: Full precision calculations help to assure consistent and reliable data handling across all of Waterly.
Similarly, when we calculate the daily/monthly/annual minimums, maximums, totals, and averages, we are using the Full Precision of each day's numbers, which means that we are as accurate as the inputted data.
Regulatory Realities
We have seen in some states (specifically, Indiana and Pennsylvania last we checked in summer 2024) where they have an Excel template that rounds numbers for the day PRIOR to monthly aggregation. We believe this practice to be wrong but may be a reality. If your state follows this practice in a template, we encourage you READ YOUR REGULATIONS, as we have also seen where the templates utilize incorrect rounding but the regulations do NOT specify that rounding intermediate calculations is required. In these cases, we recommend you contact your local inspector with a specific example for direction in writing and also encourage them to making note of the discrepancy in their templates.
Fortunately, Waterly does support the scenario where we will enable our customers to round (truncate) intermediate values prior to downstream calculations, even though we don’t recommend it unless your state truly requires it. If your state specifically requires this, all you have to do is to create a support ticket, reference your state’s regulations, and we are happy to enable the functions to enable truncating numbers for you.
References and Related Articles
- A very old but relevant article from the EPA on rounding and precision for environmental values (air): https://www.epa.gov/emc/technical-information-document-024-memo-rounding-and-significant-figures
- Changing the Range, # of Decimals, and Archiving Metrics in Waterly
- Discussion on Water/Wastewater Precision in Reddit: https://www.reddit.com/r/Wastewater/s/rVy3QvxC7e
- IEEE Standard for Floating Point Arithmetic: https://ieeexplore.ieee.org/document/4610935
- Pennsylvania Reference on eDMR, which includes examples on the 6th page of the PDF showing full precision carrying forward in intermediate calculations: https://files.dep.state.pa.us/Water/FactSheets/Operations/3800-BK-DEP3047.pdf