EVALUATE: predict across the range of an input variable

[gjwgit]

Split out from #1174, where this was point (1). Point (2) of that issue,
scoring a larger dataset, is done and is tracked in #1175.

What is wanted

A table, and a chart, that shows how the prediction changes as one input
variable is varied across its range, with the other inputs held at values the
user chooses. One row per value of the varied variable, each with the
prediction and its probability.

In the weather dataset, for example: enter a min temp, a max temp and a cloud
cover, then vary the humidity from its lowest to its highest value in the
data, and see the predicted likelihood of rain tomorrow at each step.

This is a natural extension of the Interactive button on the EVALUATE tab
(6.5.39), which already collects a value for every input variable and predicts
that one observation with each ticked model. The step beyond it is to nominate
one variable to vary, build a grid rather than a single row, and plot the
result.

Known elsewhere as an ICE curve, or predictions at representative values, and
it is what the at = argument of the
prediction
package produces.

Notes towards building it

No new R dependency is needed. Every evaluate_model_*.R script sets
model and defines pred_ra() and prob_ra() for it, and those work on a
data frame of any number of rows just as well as on the single row the
Interactive popup builds today. The prediction package has 67 methods, but
not randomForest, xgboost, ada, ctree/cforest or neuralnet, so it
would cover fewer of our models than we already cover ourselves.

The shape of the curve depends on the model, and this needs designing for.
Measured over the weather data:

• Decision tree, varying humidity_3pm: a single step, 0.06 below about 75%
  humidity and 0.85 above it. A tree is piecewise constant, so its curve is
  always a step function.
• Linear model, same variable: a smooth curve from 0.0007 to 0.99.
• Decision tree, varying rainfall: a flat line, because that tree never
  splits on rainfall.

A flat line is a truthful answer, the model genuinely ignores that variable,
but it will be reported as a bug unless the display says so. Defaulting the
varied variable to one the model actually uses, and showing several models on
the one chart, would both help.

Varying a derived variable can produce impossible observations. In the
weather data rain_today follows from rainfall, so sweeping rainfall
while holding rain_today at No asks the model about combinations that cannot
occur. Worth a note in the tooltip at least.

The example in #1174 reads two ways. Sweeping today's rainfall from 0 to
its maximum and reporting the predicted likelihood of rain tomorrow is
straightforward with the models Rattle builds. Reporting the likelihood of
each possible volume of rain tomorrow is a different question, needing a
distributional or quantile model, and is not something any current Rattle
model can answer. Worth settling which is meant before building.

[gjwgit]

@TonyNolan assigning this to you for the one question that decides what gets
built, from the last note above.

Your example in #1174 was: enter a min temp, max temp and cloud cover, then
show "zero rain to the maximum rain in the dataset, and gave a prediction for
each observation, the likely hood of that volume of rain the next day".

That reads two ways, and they are quite different jobs.

(a) Vary today's rainfall, predict rain tomorrow. Each row is a value of
today's rainfall, from 0 up to the 44.8mm maximum in the weather data, and
the prediction is the likelihood of any rain tomorrow at that value, with
min temp, max temp and cloud held at what you entered:

rainfall_today   predicted   probability
          0.0          No         0.354
          5.0          No         0.391
         10.0          No         0.429
         ...
         44.8         Yes         0.692

This is straightforward with the models Rattle already builds, and is what
#1178 is written up as.

(b) Predict how much rain falls tomorrow, as a distribution. Each row is a
possible volume of rain tomorrow, and the prediction is the likelihood of
that particular volume:

rain_tomorrow_mm   likelihood
             0.0        0.61
             5.0        0.14
            10.0        0.07
             ...

This one none of Rattle's models can answer as they stand. They predict either
a yes/no with a probability, or a single expected number of millimetres, not a
spread of likelihoods over possible amounts. It would need a different kind of
model, a quantile or distributional one, which would be a much larger piece of
work and worth its own issue.

Which did you have in mind? If it is (b), it would help to know what you would
do with the answer, as there may be a simpler way to get at it, for example
predicting the expected millimetres with a regression model and reporting the
error around it.