Can you write a simple Python function to implement a basic machine learning algorithm

 Can you write a simple Python function to implement a basic machine learning algorithm?

Sure! Let's implement a basic linear regression algorithm from scratch using Python. Linear regression is a fundamental algorithm in machine learning used for predicting a continuous target variable based on one or more input features.

Here's a simple Python function to perform linear regression using gradient descent:

import numpy as np

def linear_regression(X, y, learning_rate=0.01, epochs=1000):

    """

    Perform linear regression using gradient descent.

    Parameters:

    X: numpy array of shape (m, n), where m is the number of samples and n is the number of features.

    y: numpy array of shape (m, 1), where m is the number of samples.

    learning_rate: float, the learning rate for gradient descent.

    epochs: int, the number of iterations for training.

    Returns:

    theta: numpy array of shape (n, 1), the learned parameters.

    """

    m, n = X.shape

    X_b = np.c_[np.ones((m, 1)), X]  # Add bias term (column of ones) to X

    theta = np.random.randn(n + 1, 1)  # Initialize parameters randomly

    for epoch in range(epochs):

        gradients = 2 / m * X_b.T.dot(X_b.dot(theta) - y)

        theta = theta - learning_rate * gradients

    

    return theta

# Example usage:

if __name__ == "__main__":

    # Generate some synthetic data for demonstration

    np.random.seed(42)

    X = 2 * np.random.rand(100, 1)

    y = 4 + 3 * X + np.random.randn(100, 1)

    # Train the linear regression model

    theta = linear_regression(X, y)

    print("Learned parameters:", theta)

    

    # Predicting new data

    X_new = np.array([[0], [2]])

    X_new_b = np.c_[np.ones((2, 1)), X_new]  # Add bias term

    y_predict = X_new_b.dot(theta)

    print("Predictions:", y_predict)

Explanation:

1. Data Preparation: We create synthetic data for demonstration purposes. X represents the input features, and y represents the target values.
2. Add Bias Term: We add a column of ones to the input features X to account for the bias term in linear regression.
3. Parameter Initialization: Initialize the parameters theta randomly.
4. Gradient Descent: Perform gradient descent for a specified number of epochs to minimize the cost function. The gradients are calculated and used to update the parameters.
5. Return Parameters: Return the learned parameters theta.

Usage:

• We generate synthetic data with some noise.
• Train the linear regression model using the linear_regression function.
• Print the learned parameters and use them to make predictions on new data.

This is a simple implementation for educational purposes. In practice, you'd typically use a library like Scikit-learn for linear regression and other machine learning algorithms, as it provides optimized and well-tested implementations.