Python Code Public Examples

My Deep Learning 'Hello World'

Juan D. Correa - Software Developher @ Veracruz,MX & California,USA https://www.astropema.com/ astropema@gmail.com

The notebook now has comprehensive content, with structured documentation for each example and key reflections added at the end. Here's a final summary of the improvements and structure:

Neural Network Models:

Both TensorFlow and PyTorch models were demonstrated and fully tested. The models were designed to handle both simple linear and more complex non-linear relationships. Model Components and Functionality:

Layers, activations, and parameter extraction were explained and demonstrated. Both forward propagation and model parameter analysis were included for further insights. Graphical Visualization:

For the TensorFlow example, training data, predictions, and a plot of the learned function were visualized. Documentation and Reflections:

Clear summaries of each code block. A concluding reflection that ties together the learning goals, challenges, and outcomes of the exercise.

Challenge Summary

The challenge in this exercise was to design and understand neural networks for both simple and complex relationships between input and output data. We explored two main models:

A TensorFlow-based linear model that learned a quadratic function.

A PyTorch-based feedforward neural network capable of more complex non-linear approximations.

Both examples demonstrated the core principles of neural networks, including:

Forward propagation: Passing input through layers to compute outputs.

Activation functions: Introducing non-linearity to learn more complex patterns.

Weight and bias analysis: Understanding how the network learns and encodes functions internally.

Outcome Reflection

Through these exercises, we:

Successfully defined, trained, and tested models with varying complexity.

Visualized and extracted key model parameters (weights and biases) to understand the learned functions.

Demonstrated the importance of designing appropriate network architectures for the task at hand.

This exploration lays the groundwork for further studies in neural network optimization, data preparation, and more advanced tasks such as image recognition or time series forecasting.

# Library for creating data paths
import os

# Library for randomly selecting data points
import random

# Library for performing numerical computations
import numpy as np

# Library for creating and showing plots
import matplotlib.pyplot as plt

# Library for reading and showing images
import matplotlib.image as mpimg

import tensorflow as tf

### Neural Network Model Setup Documentation

Basic Example

The first example demonstrates how to define and use a neural network to approximate a quadratic relationship between input and output data.

Summary

• The model consists of one dense layer with a single input and output unit, ideal for learning simple linear functions.
• The training process maps inputs to outputs by minimizing the mean squared error over multiple epochs.
• After training, the model can predict values, approximating a function like y = x^2 + 2x + 1.

This introduces the concept of training a neural network on basic data before moving on to more complex cases.

---

Model Definition

The SimpleNet model is a neural network with multiple layers designed to handle more complex relationships between input and output data.

Structure Overview

• Input Layer: Accepts input data with 10 features.
• Hidden Layer: Consists of 5 neurons, followed by a ReLU activation function.
• Output Layer: Reduces the hidden layer's output to a single value.

Data flows through the network via forward propagation, passing through the layers and activation functions to produce an output.

---

Example Usage

We tested the model by generating random input data and passing it through the network. The forward pass generated a random output, verifying the proper functioning of the layers and activation.

import tensorflow as tf
import numpy as np

# Define the model
model = tf.keras.Sequential([
    tf.keras.Input(shape=(1,)),
    tf.keras.layers.Dense(units=1),
])

# Compile the model
model.compile(optimizer='sgd', loss='mean_squared_error')

# Training data
xs = np.array([-2.0, -1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
ys = xs**2 + 2 * xs + 1

# Train the model with suppressed output
history = model.fit(xs, ys, epochs=500, verbose=0)

# Display the final training loss
final_loss = history.history['loss'][-1]
print(f"Final loss after training: {final_loss:.4f}")

# Make a prediction for input 10.0
prediction = model.predict(np.array([10.0]))
print("Prediction for 10.0:", prediction)

---

Reflections and Outcome

Challenge Summary

The challenge in this exercise was to design and understand neural networks for both simple and complex relationships between input and output data. We explored two main models:

1. A TensorFlow-based model that approximated a quadratic function.
2. A PyTorch-based feedforward neural network capable of more complex non-linear approximations.

Both examples demonstrated the core principles of neural networks, including:

• Forward propagation: Passing input through layers to compute outputs.
• Activation functions: Introducing non-linearity to learn more complex patterns.
• Weight and bias analysis: Understanding how the network learns and encodes functions internally.

Outcome Reflection

Through these exercises, we:

• Successfully defined, trained, and tested models with varying complexity.
• Visualized and extracted key model parameters (weights and biases) to understand the learned functions.
• Demonstrated the importance of designing appropriate network architectures for the task at hand.

This exploration lays the groundwork for further studies in neural network optimization, data preparation, and more advanced tasks such as image recognition or time series forecasting.

The final takeaway is that neural networks can approximate a wide range of functions, but their success depends heavily on the architecture, data quality, and training process.

import tensorflow as tf
import numpy as np

# Define the model
model = tf.keras.Sequential([
    tf.keras.Input(shape=(1,)),
    tf.keras.layers.Dense(units=1),
])

# Compile the model
model.compile(optimizer='sgd', loss='mean_squared_error')

# Training data
xs = np.array([-2.0, -1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
ys = xs**2 + 2 * xs + 1

# Train the model with suppressed output
history = model.fit(xs, ys, epochs=500, verbose=0)

# Display the final training loss
final_loss = history.history['loss'][-1]
print(f"Final loss after training: {final_loss:.4f}")

# Make a prediction for input 10.0
prediction = model.predict(np.array([10.0]))
print("Prediction for 10.0:", prediction)

2025-02-08 14:32:03.141667: W tensorflow/tsl/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz

Final loss after training: 12.0000
1/1 [==============================] - 0s 19ms/step
Prediction for 10.0: [[44.001762]]

Reflections and Outcome

Challenge Summary

The challenge in this exercise was to design and understand neural networks for both simple and complex relationships between input and output data. We explored two main models:

A TensorFlow-based linear model that learned a straight-line function.

A PyTorch-based feedforward neural network capable of more complex non-linear approximations.

Both examples demonstrated the core principles of neural networks, including:

Forward propagation: Passing input through layers to compute outputs.

Activation functions: Introducing non-linearity to learn more complex patterns.

Weight and bias analysis: Understanding how the network learns and encodes functions internally.

Outcome Reflection

Through these exercises, we:

Successfully defined, trained, and tested models with varying complexity.

Visualized and extracted key model parameters (weights and biases) to understand the learned functions.

Demonstrated the importance of designing appropriate network architectures for the task at hand.

This exploration lays the groundwork for further studies in neural network optimization, data preparation, and more advanced tasks such as image recognition or time series forecasting.

The final takeaway is that neural networks can approximate a wide range of functions, but their success depends heavily on the architecture, data quality, and training process.

Example of a Simple Neural Network

Here’s how neurons in a layer are defined and used in Tensorflow:

# Get the model's weights (coefficients and bias)
weights, biases = model.layers[0].get_weights()

# Display the learned weights and bias
print(f"Learned weight: {weights[0][0]}")
print(f"Learned bias: {biases[0]}")

# Reconstruct the function
print(f"The learned function is approximately: y = {weights[0][0]:.4f} * x + {biases[0]:.4f}")

Learned weight: 4.000306129455566
Learned bias: 3.998703718185425
The learned function is approximately: y = 4.0003 * x + 3.9987

# Define the data
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt

xs = np.array([-2.0, -1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=float)
ys = xs**2 + 2 * xs + 1

# Define and train the model
model = tf.keras.Sequential([
    tf.keras.layers.Dense(units=1, input_shape=(1,))
])
model.compile(optimizer='sgd', loss='mean_squared_error')
history = model.fit(xs, ys, epochs=1000, verbose=0)

# Make predictions for plotting
predicted_ys = model.predict(xs)
new_prediction = model.predict(np.array([10.0]))

# Extract weights and biases for the learned function
weights, biases = model.layers[0].get_weights()

# Plot the training data and the model's predictions
plt.figure(figsize=(10, 5))

# Plot the actual data points
plt.scatter(xs, ys, color='blue', label='Training Data (Quadratic)')

# Plot the learned linear function approximation
plt.plot(xs, predicted_ys, color='red', label='Learned Linear Approximation')

# Plot the new prediction for x = 10.0
plt.scatter(10.0, new_prediction, color='green', s=100, label='Prediction for x = 10.0')

# Add title and labels
plt.title("Quadratic Data with Linear Approximation (TensorFlow)")
plt.xlabel("Input (x)")
plt.ylabel("Output (y)")
plt.axhline(0, color='black', linewidth=0.5, linestyle='--')
plt.axvline(0, color='black', linewidth=0.5, linestyle='--')
plt.legend()

# Display the graph
plt.show()

# Display the learned linear function
print(f"The model learned a linear approximation: y ≈ {weights[0][0]:.4f} * x + {biases[0]:.4f}")

1/1 [==============================] - 0s 11ms/step
1/1 [==============================] - 0s 5ms/step

The model learned a linear approximation: y ≈ 4.0000 * x + 4.0000

Example 2

The goal of this code is to define and test a simple neural network architecture. Specifically, we achieved the following goals step-by-step:

---

Goal 1: Define a neural network structure

We designed a basic feedforward neural network (SimpleNet) with:

• Input layer (10 input features),
• Hidden layer (5 neurons),
• Activation function (ReLU for non-linearity),
• Output layer (1 output neuron).

This structure can be adapted for various tasks like:

• Regression (predicting a continuous value),
• Binary classification (yes/no predictions).

---

Goal 2: Test forward propagation (inference)

• By generating random input data (torch.randn(1, 10)), we simulated how an input flows through the network.
• The network successfully processed the input and produced an output, indicating that the layers and activation functions were working correctly.

---

Goal 3: Lay the foundation for training

• While this code only performed a forward pass (inference), it's a crucial step before training.
• In a training loop, we would:
  1. Pass real input data (e.g., features like height, weight, age, etc.) through the network.
  2. Compare the network's output to actual labels using a loss function.
  3. Update the weights and biases using an optimizer (e.g., SGD or Adam) to minimize the loss.

---

Goal 4: Explore and extract internal components

• We later extended the code to extract and visualize:
  • Weights and biases from the layers,
  • The mathematical function the network learned.

This helps us understand what the network is doing under the hood.

---

Practical Example: Predicting house prices

Imagine you have 10 features about houses (e.g., square footage, number of bedrooms, location, etc.) and you want to predict the house price. This network could:

1. Take the 10 input features,
2. Pass them through the hidden and output layers,
3. Output a predicted price.

By adjusting the architecture, data, and training process, you could tailor this network to any number of tasks in machine learning.

---

In this case, the primary goal was to build, test, and understand a simple neural network to serve as a foundation for more complex machine learning tasks. Would you like to continue and train this network on a sample task? 😊

'''
torch: Core PyTorch library for tensors and operations on them.
torch.nn: Contains modules for building neural networks (e.g., layers, activation functions).
torch.optim: Provides optimization algorithms for training (e.g., SGD, Adam).
'''

import torch
import torch.nn as nn
import torch.optim as optim

'''
Inheritance: We define a class SimpleNet that inherits from nn.Module, which is the base class for all neural networks in PyTorch.
Layers:
self.fc1 = nn.Linear(10, 5): A fully connected (dense) layer with 10 input features and 5 neurons in the hidden layer.
self.relu = nn.ReLU(): A ReLU (Rectified Linear Unit) activation function that introduces non-linearity to the network.
self.fc2 = nn.Linear(5, 1): Another fully connected layer, mapping the 5 hidden neurons to 1 output neuron.

The forward method defines how data flows through the network.
The input x is processed in the following order:
First layer (fc1): The input is transformed by the fully connected layer fc1.
Activation (ReLU): The output of fc1 is passed through the ReLU activation function to introduce non-linearity.
Second layer (fc2): The activated output is passed through the second fully connected layer, which produces the final output.
The final output is returned.
'''

# Define a neural network with one hidden layer
class SimpleNet(nn.Module):
    def __init__(self):
        super(SimpleNet, self).__init__()
        self.fc1 = nn.Linear(10, 5)  # Input layer (10 neurons) -> Hidden layer (5 neurons)
        self.relu = nn.ReLU()        # Activation function
        self.fc2 = nn.Linear(5, 1)   # Hidden layer (5 neurons) -> Output layer (1 neuron)


    def forward(self, x):
        x = self.fc1(x)            # Apply first linear layer
        x = self.relu(x)           # Apply activation
        x = self.fc2(x)            # Apply second linear layer
        return x


# Instantiate and test the network
# We create an instance of the SimpleNet class, which initializes the network with the defined layers and structure.
model = SimpleNet()

'''
We generate a random input tensor with shape (1, 10).
1: The batch size (i.e., one sample at a time).
10: The number of input features, matching the input size of the first layer (fc1).
'''

input_data = torch.randn(1, 10)    # Batch size = 1, input features = 10

'''
We pass the input data through the network by calling model(input_data), which internally invokes the forward method.
The output is a tensor with shape (1, 1) (since the final layer has one output neuron).
We print the output to verify that the network processes the input correctly.
'''

output = model(input_data)
print(output)

tensor([[-0.0427]], grad_fn=<AddmmBackward0>)

Summary of the flow:

The input tensor of size (1, 10) is fed into the network. It is transformed by the first fully connected layer (fc1), resulting in a tensor of size (1, 5). The ReLU activation function applies non-linearity to the output of fc1. The activated output is passed to the second fully connected layer (fc2), producing a final output tensor of size (1, 1). We print the output tensor to verify the network's behavior. This setup demonstrates how to define, initialize, and test a simple feedforward neural network in PyTorch

# Assuming you have a sequential model defined like this
model = nn.Sequential(
    nn.Linear(10, 5),
    nn.ReLU(),
    nn.Linear(5, 1)
)

# Access layers by index
weights_fc1 = model[0].weight.detach().numpy()
biases_fc1 = model[0].bias.detach().numpy()

# Display weights and biases
print("Weights of the first layer (Sequential):", weights_fc1)
print("Biases of the first layer (Sequential):", biases_fc1)

# If you want to get weights from the second linear layer:
weights_fc2 = model[2].weight.detach().numpy()
biases_fc2 = model[2].bias.detach().numpy()
print("Weights of the second layer:", weights_fc2)
print("Biases of the second layer:", biases_fc2)

Weights of the first layer (Sequential): [[ 0.2697888   0.20213635  0.19023721 -0.1468712  -0.3114496   0.170151
   0.03030304  0.12703265  0.21450496  0.01119327]
 [ 0.17015229 -0.23759213  0.03548381 -0.18301584 -0.2475193   0.04969203
  -0.06004484 -0.2940334   0.1590394   0.04275385]
 [-0.16047859 -0.2922929  -0.1300369   0.12639414 -0.12328725  0.14820968
   0.1178977   0.28436846 -0.1023896   0.03928133]
 [ 0.20487216 -0.2571995  -0.24848299 -0.06312094  0.18579602  0.1153989
  -0.31189385  0.30386925 -0.16431992  0.06752644]
 [-0.20599976 -0.24898365 -0.25330406 -0.00270033  0.27830166  0.06941654
  -0.2620376  -0.21067841  0.16468433  0.21272394]]
Biases of the first layer (Sequential): [ 0.17527376  0.00829762  0.30459145 -0.19042133  0.2523093 ]
Weights of the second layer: [[-0.4097851  -0.15117815  0.30260804  0.39809036  0.00216425]]
Biases of the second layer: [-0.2643417]

How to interpret the results: First Layer (Input -> Hidden):

Weights shape: (5, 10) – This means the layer has 5 neurons, each with 10 input connections. Biases shape: (5,) – Each neuron in the hidden layer has its own bias term. Second Layer (Hidden -> Output):

Weights shape: (1, 5) – The output layer has 1 neuron, with 5 input connections (from the hidden layer). Biases shape: (1,) – The single output neuron has one bias.

# Convert weights and biases to NumPy arrays if needed
hidden_weights = weights_fc1  # Shape (5, 10)
hidden_biases = biases_fc1    # Shape (5,)
output_weights = weights_fc2  # Shape (1, 5)
output_bias = biases_fc2      # Shape (1,)

# Assuming input is a 10-dimensional vector `x`
print("The final output function can be expressed as:")
print(f"y = {output_weights.flatten()} * relu({hidden_weights} * x + {hidden_biases}) + {output_bias[0]:.4f}")

The final output function can be expressed as:
y = [-0.4097851  -0.15117815  0.30260804  0.39809036  0.00216425] * relu([[ 0.2697888   0.20213635  0.19023721 -0.1468712  -0.3114496   0.170151
   0.03030304  0.12703265  0.21450496  0.01119327]
 [ 0.17015229 -0.23759213  0.03548381 -0.18301584 -0.2475193   0.04969203
  -0.06004484 -0.2940334   0.1590394   0.04275385]
 [-0.16047859 -0.2922929  -0.1300369   0.12639414 -0.12328725  0.14820968
   0.1178977   0.28436846 -0.1023896   0.03928133]
 [ 0.20487216 -0.2571995  -0.24848299 -0.06312094  0.18579602  0.1153989
  -0.31189385  0.30386925 -0.16431992  0.06752644]
 [-0.20599976 -0.24898365 -0.25330406 -0.00270033  0.27830166  0.06941654
  -0.2620376  -0.21067841  0.16468433  0.21272394]] * x + [ 0.17527376  0.00829762  0.30459145 -0.19042133  0.2523093 ]) + -0.2643

class CustomLayer(nn.Module):
    def __init__(self, input_size, output_size):
        super(CustomLayer, self).__init__()
        self.weight = nn.Parameter(torch.randn(input_size, output_size))
        self.bias = nn.Parameter(torch.randn(output_size))

    def forward(self, x):
        return torch.matmul(x, self.weight) + self.bias

# Example usage
custom_layer = CustomLayer(10, 5)
output = custom_layer(torch.randn(1, 10))
output

tensor([[-3.4886,  0.8400,  1.8178,  0.4445,  0.1836]], grad_fn=<AddBackward0>)

import torch
import torch.nn as nn
import numpy as np
import matplotlib.pyplot as plt

# Define data (Quadratic relationship) and convert to PyTorch tensors
xs = torch.tensor([-2.0, -1.0, 0.0, 1.0, 2.0, 3.0, 4.0], dtype=torch.float32)
ys = xs**2 + 2 * xs + 1

# Define a simple model in PyTorch
class SimpleNet(nn.Module):
    def __init__(self):
        super(SimpleNet, self).__init__()
        self.fc1 = nn.Linear(1, 1)  # Single-layer linear model

    def forward(self, x):
        return self.fc1(x)

# Initialize the model, loss function, and optimizer
model = SimpleNet()
criterion = nn.MSELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

# Reshape data for PyTorch (batch, feature)
xs = xs.view(-1, 1)  # Ensure the shape is compatible with PyTorch input
ys = ys.view(-1, 1)

# Train the model
epochs = 1000
for epoch in range(epochs):
    optimizer.zero_grad()
    output = model(xs)
    loss = criterion(output, ys)
    loss.backward()
    optimizer.step()

# Make predictions
with torch.no_grad():
    predicted_ys = model(xs).numpy()  # Convert to numpy for plotting
    new_prediction = model(torch.tensor([[10.0]], dtype=torch.float32)).item()

# Extract weights and bias
weights = model.fc1.weight.item()
biases = model.fc1.bias.item()

# Plotting the results
plt.figure(figsize=(10, 5))

# Plot the training data
plt.scatter(xs.numpy(), ys.numpy(), color='blue', label='Training Data (Quadratic)')

# Plot the learned linear approximation
plt.plot(xs.numpy(), predicted_ys, color='red', label='Learned Linear Approximation')

# Plot the new prediction point
plt.scatter(10.0, new_prediction, color='green', s=100, label='Prediction for x = 10.0')

# Add title and labels
plt.title("Quadratic Data with Linear Approximation (PyTorch)")
plt.xlabel("Input (x)")
plt.ylabel("Output (y)")
plt.axhline(0, color='black', linewidth=0.5, linestyle='--')
plt.axvline(0, color='black', linewidth=0.5, linestyle='--')
plt.legend()

# Display the graph
plt.show()

# Display the learned linear function
print(f"The model learned a linear approximation: y ≈ {weights:.4f} * x + {biases:.4f}")

The model learned a linear approximation: y ≈ 4.0000 * x + 4.0000

Neural Network Model Setup Documentation

Basic Example

The first example in this exercise demonstrates how to define and use a neural network to learn a linear relationship between input and output data.

Summary

• The model consists of one dense layer with a single input and output unit, ideal for approximating simple non-linear functions.
• The training process maps inputs to outputs by minimizing the mean squared error over multiple epochs.
• After training, the model can predict values, learning functions such as y=x^2+2x+1.

This introduces the concept of training a neural network on basic data before moving on to more complex cases.

---

Model Definition

The SimpleNet model is a neural network with multiple layers designed to handle more complex relationships between input and output data.

Structure Overview

• Input Layer: Accepts input data with 10 features.
• Hidden Layer: Consists of 5 neurons, followed by a ReLU activation function.
• Output Layer: Reduces the hidden layer's output to a single value.

Data flows through the network via forward propagation, passing through the layers and activation functions to produce an output.

---

Example Usage

We tested the model by generating random input data and passing it through the network. The forward pass generated a random output, verifying the proper functioning of the layers and activation.

---

Reflections and Outcome

Challenge Summary

The challenge in this exercise was to design and understand neural networks for both simple and complex relationships between input and output data. We explored two main models:

1. A TensorFlow-based model that approximated a quadratic function.
2. A PyTorch-based feedforward neural network capable of more complex non-linear approximations.

Both examples demonstrated the core principles of neural networks, including:

• Forward propagation: Passing input through layers to compute outputs.
• Activation functions: Introducing non-linearity to learn more complex patterns.
• Weight and bias analysis: Understanding how the network learns and encodes functions internally.

Outcome Reflection

Through these exercises, we:

• Successfully defined, trained, and tested models with varying complexity.
• Visualized and extracted key model parameters (weights and biases) to understand the learned functions.
• Demonstrated the importance of designing appropriate network architectures for the task at hand.

This exploration lays the groundwork for further studies in neural network optimization, data preparation, and more advanced tasks such as image recognition or time series forecasting.

The final takeaway is that neural networks can approximate a wide range of functions, but their success depends heavily on the architecture, data quality, and training process.