关于逻辑回归和线性回归的问题

Question

I'm trying to understand logistic and linear regression and was able to understand the theory behind it (doing andrew ng course).

We have X -> given features -> matrix of (m , n+1) where m - no. of cases and n- features given (excluding x0)

We have y - > the label to predict -> matrix of (m,1)

Now while I'm implementing it from scratch in python, I'm confused as to why we use transpose of theta in the sigmoid function.

Also we use theta transpose X for linear regression too.

We do not have to perform matrix multiplication anywhere while coding, its straight element to element coding, what's the need for the transpose or is my understanding wrong and we need to take matrix multiplication during implementation.

My main concern is that I'm very confused as to where we do matrix multiplication and where we do element wise multiplication in logistic and linear regression

Answer 1

You are a bit off topic for this area, but the piece you appear to be hung up on is the treatment of x and Theta .

In the use cases you describe, x is a vector of inputs, or the "feature vector". The Theta vector is the vector of coefficients. Both are usually expressed as column vectors and of course, must be of the same dimension.

So to "make a prediction" you need the inner product of these two, and the output needs to be a scalar (by definition for inner product) so you need to transpose the theta vector in order to properly express that operation, which is a matrix multiplication of two vectors. Make sense?

Answer 2

For matrix multiplication, the number of Columns in the first element must equal the number of rows in the second element . Since one of the elements your multiplying has either one column or one row, it does not appear to be matrix multiplication due to it's simplicity. But it still is matrix multiplication

Let me provide an example,

Let A be (m,n) matrix We can perform scalar multiplication, for some fixed a in the real numbers If we want to multiply A to some vector, x , we need to meet some restrictions. Here it is common to mistake the dot product for matrix multiplication, but they serve completely different purposes.

So our restrictions for multiplying an (m,n) matrix, A by a vector x is that x has the same number of entries as A has columns To do this in your example, one of the elements needed to be transposed.