1(ch_regressor)= 2 3# PetscRegressor: Regression Solvers 4 5The ``PetscRegressor`` library provides a framework for the scalable solution of 6regression and classification problems. Methods are available for 7 8- {any}`sec_regressor_linear` 9 10Note that by regressor, we mean an algorithm or implementation 11used to fit a regression model, following notation from machine-learning community. 12Regressor here does NOT mean independent (or predictor) variable, as it often does in the 13statistics community. 14 15(sec_regressor_usage)= 16 17## Basic Regressor Usage 18 19Here, we introduce a simple example to demonstrate `PetscRegressor` usage. 20Please read {any}`sec_regressor_solvers` for more in-depth discussion. 21The code presented {any}`below <regressor-ex3>` solves ordinary linear 22regressoion problem, with various regularization options. 23 24In the simplest usage of the regressor solver, the user simply needs to 25provide target matrix (`Mat`), and a target vector (`Vec`) to fit 26the regressor against. With fitted regressor, then the user can obtain 27predicted value vector. 28 29PETSc's default method for solving regression problem is ordinary least squares, 30`REGRESSOR_LINEAR_OLS`, which is a sub-type of linear regressor, 31`PETSCREGRESSORLINEAR`. 32 33Note that data creation, option parsings, and cleaning stages are omiited for 34display purposes. The complete code is available in {ref}`ex3.c <regressor-ex3>`. 35 36(regressor-ex3)= 37:::{admonition} Listing: `src/ml/regressor/tests/ex3.c` 38```{literalinclude} /../src/ml/regressor/tests/ex3.c 39:prepend: '#include <petscregressor.h>' 40:start-at: int main 41:end-at: PetscFinalize 42:append: return 0;} 43``` 44::: 45 46To create a `PetscRegressor` solver, one must first call `PetscRegressorCreate()` 47as follows: 48 49``` 50PetscRegressorCreate(MPI_Comm comm, PetscRegressor *regressor); 51``` 52 53To choose a solver type, the user can either call 54 55``` 56PetscRegressorSetType(PetscRegressor regressor, PetscRegressorType type); 57``` 58 59or use the option `-regressor_type <method>`, where details regarding the 60available methods are presented in {any}`sec_regressor_solvers`. 61The application code can take complete control of the linear and nonlinear 62techniques used in the Newton-like method by calling 63 64``` 65PetscRegressorSetFromOptions(regressor); 66``` 67 68This routine provides an interface to the PETSc options database, so 69that at runtime the user can select a particular regression solver, set 70various parameters and customized routines. With this routine the user 71can also control all inner solver options in the `KSP`, and `Tao` 72modules, as discussed in {any}`ch_ksp`, {any}`ch_tao`. 73 74After having set these routines and options, the user can fit the problem 75by calling 76 77``` 78PetscRegressorFit(PetscRegressor regressor, Mat X, Vec y); 79``` 80 81where `X` is training data, and `y` is target values. 82Finally, after fitting the regressor solver, the user can compute 83prediction, that is, perform inference, using a fitted regressor. 84 85``` 86PetscRegressorPredict(PetscRegressor regressor, Mat X, Vec y_predicted); 87``` 88 89Finally, after the user is done using predicting the regressor solver, 90the user should destroy the `PetscRegressor` context with 91 92``` 93PetscRegressorDestroy(PetscRegressor *regressor); 94``` 95 96Note that the user should not destroy `y_predicted` from previous section, 97as this is done internally. 98 99(sec_regressor_solvers)= 100 101## Regression Solvers 102 103One can see the list of regressor solver types in Table 104{any}`tab-regressordefaults`. Currently, we only support one type, 105`PETSCREGRESSORLINEAR`. 106 107```{eval-rst} 108.. list-table:: PETSc Regressor 109 :name: tab-regressordefaults 110 :header-rows: 1 111 112 * - Method 113 - PetscRegressorType 114 - Options Name 115 * - Linear 116 - ``PETSCREGRESSORLINEAR`` 117 - ``linear`` 118``` 119 120If the particular method that the user is using supports regularizer, 121the user can set regularizer's weight via 122 123``` 124PetscRegressorSetRegularizerWeight(PetscRegressor regressor, PetscReal weight); 125``` 126 127or with the option `-regresor_regularizer_weight <weight>`. 128 129(sec_regressor_linear)= 130 131## Linear regressor 132 133The method `PETSCREGRESSORLINEAR` (`-regressor_type linear`) 134constructs a linear model to reduce the sum of squared differences 135between the actual target values in the dataset and the target 136values estimated by the linear approximation. By default, 137this method will use bound-constrained regularized Gauss-Newton 138`TAOBRGN` to solve the regression problem. 139 140Currently, linear regressor has three types, which are described 141in Table {any}`tab-lineartypes`. 142 143```{eval-rst} 144.. list-table:: Linear Regressor types 145 :name: tab-lineartypes 146 :header-rows: 1 147 148 * - Linear method 149 - ``PetscRegressorLinearType`` 150 - Options Name 151 * - Ordinary 152 - ``REGRESSOR_LINEAR_OLS`` 153 - ``ols`` 154 * - Lasso 155 - ``REGRESSOR_LINEAR_LASSO`` 156 - ``lasso`` 157 * - Ridge 158 - ``REGRESSOR_LINEAR_RIDGE`` 159 - ``ridge`` 160``` 161 162If one wishes, the user can (when appropriate) use `KSP` to solve the problem, instead of `Tao`, 163via 164 165``` 166PetscRegressorLinearSetUseKSP(PetscRegressor regressor, PetscBool flg); 167``` 168 169or with the option `-regressor_linear_use_ksp <true,false>`. 170 171The user can also compute the intercept, also known as the bias or offset), via 172 173``` 174PetscRegressorLinearSetFitIntercept(PetscRegressor regressor, PetscBool flg); 175``` 176 177or with the option `-regressor_linear_fit_intercept <true,false>`. 178 179After the regressor has been fitted and predicted, one can obtain intercept and 180a vector of the fitted coefficients from a linear regression model. 181 182``` 183PetscRegressorLinearGetCoefficients(PetscRegressor regressor, Vec *coefficients); 184PetscRegressorLinearGetIntercept(PetscRegressor regressor, PetscScalar *intercept); 185``` 186