如何在 GPU 中运行机器学习算法

Question

I have used backward elimination algorithm to reduce features. But the point is with large amount of features and samples, it run on CPU and pretty slow.

How could i run it multithreading on GPU like i train Deep Learning. This is my code

import pandas as pd
data = pd.read_csv('data.csv')
X = data.drop(['Path','id','label'], axis=1)
y = data['label']

from mlxtend.feature_selection import SequentialFeatureSelector as sfs
from sklearn.linear_model import LinearRegression
lreg = LinearRegression()
new_sfs = sfs(lreg, k_features=1600, forward=False, verbose=1, scoring='neg_mean_squared_error')
new_sfs = new_sfs.fit(X, y)
feat_names = list(sfs1.k_feature_names_)

Answer 1

Have you considered using JAX ? You can easily use numpy routines (written in jax) to offload your calculations to GPU (or any opther device like TPU). Here's an example of basic linear regression in JAX:

import jax.numpy as jnp
import jax
import matplotlib.pyplot as plt

@jax.jit
def gpu_linear(x,y):
    return jnp.linalg.lstsq(x, y, rcond=None)[0]

# Setting key for random number generation
key_ = jax.random.PRNGKey(seed=1)

# Generating toy example
key_, subkey_ = jax.random.split(key=key_)
x = jax.random.normal(key=subkey_, shape=(100000,))
x = jnp.vstack([x,jnp.ones(len(x))]).T

key_, subkey_ = jax.random.split(key=key_)
y = 1.6*x[:,0] + jax.random.normal(key=subkey_, shape=(100000,))


m, c = gpu_linear(x,y)

plt.plot(x[:,0], y, 'o', label='original data')
plt.plot(x[:,0], m*x[:,0] + c, 'r', label='fitted line')
plt.legend()
plt.show()

I timed the above implementation using %timeit :

%timeit gpu_linear(x,y)

243 µs ± 691 ns per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

Here's the same thing implemented in numpy (just for reference).

import numpy as np
import matplotlib.pyplot as plt

def cpu_linear(x,y):
    return np.linalg.lstsq(x, y, rcond=None)[0]

# Generating toy example
x = np.random.normal(0, 1, 100000)
x = np.vstack([x,jnp.ones(len(x))]).T

y = 1.6*x[:,0] + np.random.normal(0, 1, 100000)


m, c = cpu_linear(x,y)

plt.plot(x[:,0], y, 'o', label='original data')
plt.plot(x[:,0], m*x[:,0] + c, 'r', label='fitted line')
plt.legend()
plt.show()

%timeit cpu_linear(x,y)

1.52 ms ± 22.3 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each)

As you can see there's a speed up when I'm using JAX as all my calculations are offloaded to GPU. Even if you're not using GPU, JAX can cache your functions (when I decorate the function using @jax.jit ) and you'll see speedups on CPU as well (compared to naive numpy or scikit learn).