如何在 Python 中进行多元非线性回归？

Question

Let's say my actual equation is y = a * b + c

So my data set looks like

a b c y
2 5 4 14
3 7 2 23
1 7 4 15
4 1 7 11
3 2 1 7
1 2 3 5

And so forth. What module do I use in order to have an output that tells me "y = a * b + c"? Is this even possible?

How about y = a * a + b? Any pointers to documentation or explanation of what I should try would be great.

Edit:

The duplicate is clearly a different scenario. In that example there is a single formula that describes a line; in my example it is many variables that mostly fit a result. That other one does not talk about squared terms.

Answer 1

There is no module. Your general problem is "what simple function best fits this data?" There is no general solution, as "simple" requires proper definition and restriction to yield a meaningful answer.

A basic theorem of algebra shows that a data set on N points can be fitted by a polynomial of degree no more than N-1 . Restricting more than this requires that you define search space and explore within that definition.

Yes, there exist methods to set a maximum degree and work within that; you can write a loop to increase that degree until you find an exact solution.

I suggest that you look at the curve-fitting methods of Scikit and employ those in a solution of your own devising. You may need to work through all combinations of your chosen degree, adding new terms each time you increase the degree. You may also need to write the exploration to consider those terms in the order of your defined complexity.

---

Response to OP comment:

I see; you're somewhat following in the footsteps of FiveThirtyEight.com, best known for accuracy with baseball and elections in the USA. Depending on the accuracy you want, this problem gets nasty very quickly. You get terms such as ((MY_OFF-OPP_DEF) ^ 1.28 + 2.1 - sqrt(OPP_GK)) / BLAH .

In any case, you're likely into a deep learning regression application, somewhat more complex than a "simple" sum-of-products scenario. You might get acceptable results with "mere" machine learning, but be prepared for disappointment in even the simpler task of predicting the winner.

Answer 2

Have you thought about giving the scikit-learn Gradient Boosting Regressor a try? Please refer to the user guide for code examples of how this method can be used on regression problems.

Please also note that the documentation states that

scikit-learn 0.21 introduces two new experimental implementations of gradient boosting trees, namely HistGradientBoostingClassifier and HistGradientBoostingRegressor, inspired by LightGBM. These histogram-based estimators can be orders of magnitude faster than GradientBoostingClassifier and GradientBoostingRegressor when the number of samples is larger than tens of thousands of samples.