0
\$\begingroup\$

I have two arrays A and B. I want to do color mapping of their difference A - B. So far, I am doing this:

1) calculate d = A - B
2) find min / max in d
3) linear mapping of d from (min, max) to (0, 1)
4) convert (0, 1) to color

The problem is, that sometimes, there are noise values in my data. For example, several values in B are too large (like 100 times bigger than the rest) and it leads to enormous difference and increased min or max. After mapping to (0, 1) all other values are "wiped". How can I solve this?

Improve this question
\$\endgroup\$
4
  • \$\begingroup\$ Are those noise values incorrect data or good data - i.e. do you want to "remove"/ignore them in result or do you want to include them? \$\endgroup\$
    – wondra
    Aug 7, 2016 at 18:37
  • \$\begingroup\$ @wondra I want to "ignore" them in a sort... ie. I want to show them as max / min error. \$\endgroup\$
    – Martin Perry
    Aug 7, 2016 at 18:49
  • \$\begingroup\$ Are those data trending or of "continuous" function(read from a sensor)? \$\endgroup\$
    – wondra
    Aug 7, 2016 at 19:12
  • \$\begingroup\$ @wondra they are continuous in a way.. not read from sensor, but calculated from a discretized function with some added noise and compared with values calculated from the function directly. \$\endgroup\$
    – Martin Perry
    Aug 7, 2016 at 19:53

1 Answer 1

Reset to default
2
\$\begingroup\$

I would suggest two options:

  1. use percentile instead of min/max (e.g. 95/5%, you need to experiment here), with linear mapping clamped to (0,1) range. To do so sort the d array and pick (N - 1) * percentile + 1th member as minimum/maximum, map as normal and clamp r = Min(1.0, Max(0.0, r)) the results in (0,1) range
  2. Use non linear mapping, for example the logarithmic scale is often used in scientific applications d = log (d)

The option 1 attempts to filter the data from erroneous values rather than displaying them, the quality heavily relies on on percentile settings, which is (input)data-dependent. The option 2 tries to use non-linear mapping in order to display meaningful values even for erroneous data, but requires users to understand logarithmic scale.

Improve this answer
\$\endgroup\$
2
  • \$\begingroup\$ For #1 I would suggest a special color designating extreme values instead of clamping them. This would require passing a flag of some sort or using a sentinel value if the color mapping only takes one input. \$\endgroup\$
    – MooseBoys
    Aug 8, 2016 at 1:26
  • \$\begingroup\$ @MooseBoys good idea, but it brings us to the original problem again - how do you tell which values are extreme(say 1000) and which ones are just a bit above(say 1.0001) percentile? You could combine it with #2, but it makes it even harder to read. \$\endgroup\$
    – wondra
    Aug 8, 2016 at 10:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .