Hi guys,
a new plugin is now available – data quality control. It is intended especially for you as a webmaster to find nonsense and potentially suspicious or incorrect values. The plugin is divided into several sections and includes for example:
- graphical and tabular overview of gaps in data
- graphical and tabular overview of spikes (rapidly changing values, likely to be errors)
- graphical and tabular overview of outliers (numbers outside normal expected range)
- graphical and tabular overview of nonsense values (numbers which cannot be right, eg. negative wind speed etc.)
- direct link to correct these values via your admin password
Those of you who have been using Meteotemplate for longer may still remember the dbInfo page, which unfortunately I had to discontinue because of the errors it was showing.
I have thought about how to overcome these. The problem was that querying the whole database will often exhaust the memory allocated to PHP. This can be set in the server settings, however most webhosting providers have this preset and it cannot be changed and even the command you can use to increase the memory, cannot override this setting (as is the case of my webhosting too…). After lot of thinking and testing various things I came up with a workaround. The query is first split into chunks, which are analyzed separately and then finally the results are aggregated again. However – please take this feature as still experimental, I need more people to confirm this is working ok.
This plugin can potentially be very useful, but it is very important that you really take the time and read the install instructions, how it works and then set it up correspondingly.