Noisy Sensor Data: Moving Average, Median and EMA Filters
See the effect of mutation, real change and invalid data with a Python test; Selection of the combined filter according to the delay and retention of the error warning.
Contents
Sensors Laboratory • 18 Mehr 1405
See the effect of mutation, real change and invalid data with a Python test; Selection of the combined filter according to the delay and retention of the error warning.
1. Before smoothing the chart, see why
A fluctuating output may be measurement noise, real ambient variation, saturation, or distorted response. Filtering is not a common cure for all of these cases. First, keep the raw data and check whether the change coincides with a single power-on, cable displacement, or communication error.
In this paper, we generate dummy data for comparison; The test result graph is not a physical sensor. The goal is to see the behavior of the algorithms against a jump and a real step change. Put the filter after the data validity check, not as a way to turn a corrupted response into a smooth number.
2. moving average; Simple with the cost of delay
The moving average divides the sum of the window samples by their number. A larger window usually shows the rapid swing more slowly, but follows the fresh change more slowly. The moving average chapter in the DSP manual explains the behavior of this filter.
For a five-sample window, a single jump contributes to the next few outputs to exit the window. If the sample interval varies, the five-sample window does not always cover the same amount of time. So record the sampling time.
3. middle Suitable for examining single mutations
The median of a window is the middle value of the sorted data. In the five-sample example, a very large value among four close values does not shift the result like the average. This feature is useful for examining single mutations, but it is not a sign of the absolute superiority of the median.
If an actual event is short, the median may remove it. For quick hit or event detection, removing a peak can remove exactly the information a project needs. The choice should start from the type of event and its duration.
4. EMA; Low memory and clear coefficient
The exponential filter uses the previous value and the new sample. In the formula below, alpha is chosen between zero and one. A smaller value reduces the contribution of new samples and makes the response slower. Start the first output with the first valid instance so that an arbitrary initial value does not skew the result.
y = alpha * x + (1 - alpha) * previous_yIf the sampling interval is not constant, a constant alpha does not necessarily give a constant time response. For advanced design, the coefficient can be adjusted according to the sample time; In the example of this article, the distance between the samples is intentionally assumed to be equal to keep the initial comparison clear.
5. Test that can be carried out with the article
The filters_demo.py file runs on Python 3 without additional libraries. Creates a completely dummy sequence with a step change and a jump and saves the raw output, moving average, median and EMA to CSV. The five-sample window and alpha of 0.2 are training choices, not recommended settings for all sensors.
Plot the CSV file and ask three questions: Which output shows the least amount of mutation? Which reaches the new level later? If the alarm must be triggered quickly, what delay is acceptable? Compare the answer with the project requirement and then change the parameters.
6. Invalid data and time gap
Don't convert disconnection, NaN, and fragment out-of-contract response to zero. Zero may be a valid physical value. The accompanying code does not accept a non-infinite value and considers an empty window as an error; This behavior helps not to hide the corrupted input.
In the product, specify the state of the data, the time of the last valid sample, and the filter restart policy. After a long break, resuming the old memory may slow down or make the display confusing. Sometimes it is appropriate to reset with the first new example; Write this decision clearly and test it.
7. Selection criteria and limits of application
| method | Advantage in educational example | Important limitation |
|---|---|---|
| moving average | Simple and traceable | Plays the jump in the window |
| the middle | Single mutation resistance | It may omit the actual short event |
| EMA | Low memory and simple implementation | The delay depends on the coefficient and sample time |
For the control decision, sum the filter delay with the sensor delay and the sampling time. A calm chart does not necessarily give a correct decision. In the warning path, the error and fast event behavior should be tested independently.
It also does not remove averaging, fixed offset, fitting error, or reference constraints. Keep the raw data for troubleshooting and record the filter parameter along with the date and software version.
Exercise files and the next step
Get the practice sheet and accompanying files (ZIP)
The files are prepared for the training exercise. Software testing does not take the place of testing your actual parts and wiring.
Frequently asked questions
Is the median always better than the average?
no The type of noise and the actual event are decisive; The median can remove the short event.
Does the filter guarantee the accuracy of the sensor?
no The fixed offset filter does not correct for saturation and misalignment.
Should we insert zero into the filter when the sensor is disconnected?
no Specify invalid data with error status and time of last valid reading.
Official sources
Technical resources reviewed on 10/10/2026. For hardware implementation, match the model and version of your part's documentation.
Question and answer article
Ask your technical question
Questions and answers are published after moderation. Authors can see their own pending drafts.
No confirmed questions have been registered for this article yet.