PyTorch, a product of Facebook's AI Research lab, is anopen-source machine learning library built on the strengths of dynamic computation graphs. Its features and workflow have made it a popular choice for researchers and developers alike.

Unlike TensorFlow, which primarily utilizes static computation graphs, PyTorch offers dynamic computational capabilities. This equips it to handle more complex architectures and facilitates an iterative, debug-friendly workflow. Moreover, PyTorch's dynamic nature naturally marries with Pythonic constructs, resulting in a more intuitive development experience.

PyTorch is known for its streamlined, Pythonic interface. This makes the process of building and training models more accessible, especially for developers coming from a Python background.

PyTorch excels in harnessing the computational strength of GPUs, reducing training times significantly. It also enables seamless multi-GPU utilization.

Another standout feature is the ability to integrate Python control structures, such as loops and conditionals, giving developers more flexibility in defining model behavior.

PyTorch integrates with libraries likematplotlib and offers a suite of debugging tools, namelytorch.utils.bottleneck,torch.utils.tester, andtorch.utils.gdb.

• Research-Oriented Projects: Especially those requiring dynamic behavior or experimental models.
• Prototyping: For a rapid and nimble development cycle.
• Small to Medium-Scale Projects: Where ease of use and quick learning curve are crucial.
• Natural Language Processing (NLP) Tasks: Many NLP-focused libraries and tools utilize PyTorch.

The choice between TensorFlow and PyTorch depends on the specific project requirements, the team's skills, and the preferred development approach.

Many organizations use ahybrid approach, leveraging the strengths of both frameworks tailored to their needs.

In PyTorch,Tensors serve as a fundamental building block, enabling efficient numerical computations on various devices, such as CPUs, GPUs, and TPUs.

They are conceptually similar tonumpy.arrays while benefiting from hardware acceleration and offering a range of advanced features for deep learning and scientific computing.

• Automatic Differentiation: Tensors keep track of operations performed on them, allowing for immediate differentiation for tasks like gradient descent in neural networks.

• Computational Graphs: Operations on Tensors construct computation graphs, making it possible to trace the flow of data and associated gradients.

• Device Agnosticism: Tensors can be moved flexibly between available hardware resources for optimal computation.

• Flexible Memory Management: PyTorch dynamically manages memory, and its tensors are aware of the computational graph, making garbage collection more efficient.

• Float16, Float32, Float64: Tensors support various numerical precisions, with 32-bit floats as the default.

• Sparse Tensors: These are much like dense ones but are optimized for tasks with lots of zeros, saving both memory and computation.

• Quantized Tensors: Designed especially for tasks that require reduced precision to benefit from faster operations and lower memory footprint.

• Per-Element Operations: PyTorch is designed for parallelism and provides a rich set of element-wise operations, which can be applied in various ways.

PyTorch is equipped with multiple inbuilt helper methods that you can utilize for monitoring tensors during training. These include:

• Variables: These have been deprecated in favor of directly using tensors, as modern versions of PyTorch have automatic differentiation capabilities.

• Gradients: By setting therequires_grad flag, you can specify which tensors should have their gradients tracked.

You can visualize the computation graph using a tool liketensorboard or directly within PyTorch using the following methods:

importtorch# Define tensorsx=torch.tensor(3.,requires_grad=True)y=torch.tensor(4.,requires_grad=True)z=2*x*y+3# Visualize the graphz.backward()print(x.grad)print(y.grad)

PyTorch was initially developed around the concept of dynamic computation graphs, which are updated in real time as operations are applied to the network. The introduction ofAutograd brought about theVariable. However, in more recent versions,Variable has been made obsolete, and utility has been integrated into the mainTensor class.

PyTorch 0.4 and earlier versions had bothTensors andVariables.

• Operations using Tensors: The operations performed onVariables were different from those onTensors.Variable relied onAutomatic Differentiation to determine gradients and update weights, whileTensors did not.

• Backward Propagation:Variable implementedbackward() functions for gradient calculation.Tensors had to be detached using the.detach method before backpropagation, so as not to compute their gradients.

PyTorch, starting from version 0.4, combined the functionalities ofVariable andTensor. This amalgamation streamlines the tensor management process.

WithAutograd automatically computing gradients, all PyTorch tensors are now gradient-enabled; they possess both data (value) and gradient attributes.

For differentiation-related operations,context managers, such astorch.no_grad(), serve to govern whether gradients are considered or not.

importtorchfromtorch.autogradimportVariable# Create a Variabletensor_var=Variable(torch.Tensor([3]),requires_grad=True)# Multiply with another Tensorresult=tensor_var*2# Obtain the gradientresult.backward()print(tensor_var.grad)

importtorch# Create a tensortensor=torch.tensor([3.0],requires_grad=True)# Multiply with another Tensorresult=tensor*2# Obtain the gradientresult.backward()print(tensor.grad)

Converting aNumPy array to a PyTorchTensor involves multiple steps, and there are different ways to carry out the transformation.

Thetorch.Tensor function acts as a bridge, allowing for direct transformation from a NumPy array:

importnumpyasnpimporttorchnumpy_array=np.array([1,2,3,4])tensor=torch.Tensor(numpy_array)

PyTorch provides a dedicated function,torch.from_numpy(), which is more efficient thantorch.Tensor:

tensor=torch.from_numpy(numpy_array)

However, it crucially binds the resulting tensor to the original NumPy array. Therefore, modifying theNumPy array also changes the associatedTensor. Any further modifications requireclone() ordetach().

In PyTorch, the.grad attribute inTensors serves a critical function by trackinggradients duringbackpropagation, ultimately enablingautomatic differentiation. This mechanism is fundamental for trainingNeural Networks.

• Gradient Accumulation: When set toTrue,requires_grad enables the accumulation of gradients for the tensor, thereby forming the backbone of backpropagation.

• Computational Graph Recording: PyTorch establishes a linkage between operations and tensors. Theautograd module records these associations, facilitating backpropagation for taking derivatives.

• Defining Operations in Reverse Mode:.backward() triggers derivatives computation through the computational graph in the reverse order of the function calls.

  Key Consideration: Only tensors withrequires_grad set toTrue in your computational graph will have their gradients computed.

For scenarios where you don't require gradients, it is advantageous to disable their computation.

• Code Efficiency: By omitting gradient computation, you can streamline code execution and save computational resources.

• Preventing Gradient Tracking: Settingno_grad() is useful if you don't want a sequence of operations to be part of the computational graph nor affect future gradient calculations.

Here's a Python example that illustrates the intricacies of tensor attributes and their roles inautomatic differentiation:

importtorch# Input tensorsx=torch.tensor(2.0,requires_grad=True)y=torch.tensor(3.0,requires_grad=True)# Operationz=x*y# Gradientsz.backward()# Triggers gradient computation for z with respect to x and y# Accessing gradientsprint(x.grad)# Prints 3.0, which is equal to yprint(y.grad)# Prints 2.0, which is equal to xprint(z.grad)# None, as z is a scalar. Its gradient is replaced by grad_fn, the function that generated z.# Code efficiency example with no_grad()withtorch.no_grad():# At this point, any operations within this block are not part of the computational graph.a=x*2print(a.requires_grad)# Falseb=a*yprint(b.requires_grad)# False

In the context of gradient-enablement,Tensors prove versatile, enabling fine-grained control, synaptic strength among their complex neural network operations.

CUDA (Compute Unified Device Architecture) is an NVIDIA technology that delivers dramatic performance increases for general-purpose computing on NVIDIA GPUs.

In the context of PyTorch, CUDA enables you toleverage GPU acceleration for deep learning tasks, reducing training time from hours to minutes or even seconds. For simple examples, PyTorch automatically selects whether to use the CPU or GPU. However, for more complex work, explicit device configuration may be needed.

Here is the Python code:

importtorch# Checks if GPU is availableiftorch.cuda.is_available():device=torch.device("cuda")# Sets device to GPUtensor_on_gpu=torch.rand(2,2).to(device)# Move tensor to GPUprint(tensor_on_gpu)else:print("GPU not available.")

For more complex use-cases, like multi-GPU training, explicit device handling in PyTorch becomes essential.

Here is the multi-GPU setup code:

device=torch.device("cuda:0"iftorch.cuda.is_available()else"cpu")model.to(device)# Moves model to GPU device# Data Parallelism for multi-GPUiftorch.cuda.device_count()>1:# Checks for multiple GPUsmodel=nn.DataParallel(model)# Wraps model for multi-GPU training

The code can be broken down as follows:

1. Device Variable: This assigns thefirst GPU (index 0) as the device if available. If not, it falls back to the CPU.

2. Moving the Model to the Device: Theto(device) method ensures the model (neural network here) is on the selected device.

3. Multi-GPU scenario: Checks if more than one GPU is available, and if so, wraps the model fordata parallelism usingnn.DataParallel. This method replicates the model across all available GPUs, divides batches across the replicas, and combines the outputs.

PyTorch allows the use of GPUs within acontext manager.

Here is the Python code:

# Executes code within the contextwithtorch.cuda.device(1):# Choose GPU device 1tensor_on_specific_gpu=torch.rand(2,2)print(tensor_on_specific_gpu)

As a best practice, it's essential to understand that while GPUs provide massive parallel processing power, they also have high latency and limited memory compared to CPUs.

Therefore, it's critical totransfer data (tensors and models) to the GPU only when it's necessary, to minimize this overhead.

Automatic differentiation in PyTorch, managed by itsautograd engine, simplifies the computation of gradients in neural networks.

1. Tensor: PyTorch's data structure that denotes inputs, model parameters, and outputs.
2. Function: Operates on tensors and records necessary information for computing derivatives.
3. Computation Graph: Formed by linking tensors and functions, it encapsulates the data flow in computations.
4. Grad_fn: A function attributed to a tensor that identifies its origin in the computation graph.

1. Tensor Construction: When a tensor is generated from data or through operations, it acquires arequires_grad attribute by default unless specified otherwise.

2. Computation Tracking: Upon executing mathematical operations, the graph's relevant nodes and edges, represented by tensors and functions, are established.

3. Local Gradients: Functions within the graph determine partial derivatives, providing the local gradients needed for the chain rule.

4. Backpropagation: Through a backwards graph traversal, the complete derivatives with respect to the tensors involved are calculated and accumulated.

Here is the Python code:

importtorch# Step 1: Construct tensors and operationsx=torch.tensor(3.,requires_grad=True)y=torch.tensor(4.,requires_grad=True)z=x*y# Step 2: Perform computationsw=z**2+10# Let's say this is our loss function# Step 3: Derive gradientsw.backward()# Triggers the full AD process# Retrieve gradientsprint(x.grad)# Should be x's derivative: 8 * x => 8 * 3 = 24print(y.grad)# Should be y's derivative: 6 * y => 6 * 4 = 24

Here are the steps to create aneural network in PyTorch:

Define the architecture based on the number of layers, types of functions, and connections.

• Prepare input and output data along with data loaders for efficiency.
• Data normalization can be beneficial for many models.

Define a class to represent the neural network usingtorch.nn.Module. Use pre-built layers fromtorch.nn.

Choose a loss function, such as Cross-Entropy for classification and Mean Squared Error for regression. Select an optimizer, like Stochastic Gradient Descent.

• Iterate overbatches of data.
• Forward pass: Compute the model's predictions based on the input.
• Backward pass: Calculate gradients and update weights to minimize the loss.

After training, assess the model's performance on a separate test dataset, typically using accuracy, precision, recall, or similar metrics.

Use the trained model to make predictions on new, unseen data.

Here is the Python code:

importtorchimporttorch.nnasnnimporttorch.optimasoptim# Architecture Designinput_size=28*28# For MNIST imagesnum_classes=10hidden_size=100# Data Preparation (Assume MNIST data is loaded in train_loader and test_loader)# ...# Model ConstructionclassNeuralNet(nn.Module):def__init__(self):super(NeuralNet,self).__init__()self.fc1=nn.Linear(input_size,hidden_size)# Fully connected layerself.relu=nn.ReLU()# Activation functionself.fc2=nn.Linear(hidden_size,num_classes)defforward(self,x):# Define the forward passout=self.fc1(x)out=self.relu(out)out=self.fc2(out)returnoutmodel=NeuralNet()# Loss and Optimizer Selectioncriterion=nn.CrossEntropyLoss()optimizer=optim.SGD(model.parameters(),lr=0.001,momentum=0.9)# Training Loopnum_epochs=5forepochinrange(num_epochs):forbatch, (images,labels)inenumerate(train_loader):images=images.reshape(-1,28*28)# Reshape images to vectorsoptimizer.zero_grad()# Zero the gradientsoutputs=model(images)# Forward passloss=criterion(outputs,labels)# Compute lossloss.backward()# Backward passoptimizer.step()# Update weights# Model Evaluation (Assume test_loader has test data)correct,total=0,0withtorch.no_grad():forimages,labelsintest_loader:images=images.reshape(-1,28*28)outputs=model(images)_,predicted=torch.max(outputs,1)total+=labels.size(0)correct+= (predicted==labels).sum().item()accuracy=correct/totalprint(f'Test Accuracy:{accuracy}')# Inference# Make predictions on new, unseen data

Both theSequential model and theModule class in PyTorch are tools for creatingneural network architectures.

• Builder Functions:Sequential employs builder functions (e.g.,nn.Conv2d) for layer definition, whileModule enables you to create layers from classes directly (e.g.,nn.Linear).

• Complexity Handling:Module presents more flexibility, allowing for branching and multiple input/output architectures. In contrast,Sequential is tailored for straightforward, layered configurations.

• Layer Customization: WhileModule gives you finer control over layer interactions, the simplicity ofSequential can be beneficial for quick prototyping or for cases with a linear layer structure.

Here is the full code:

importtorchimporttorch.nnasnn# Define Sequential Modelseq_model=nn.Sequential(nn.Linear(12,8),nn.ReLU(),nn.Linear(8,4),nn.ReLU(),nn.Linear(4,1))# Define Module Model (equivalent with flexible definition)classModuleModel(nn.Module):def__init__(self):super(ModuleModel,self).__init__()self.fc1=nn.Linear(12,8)self.relu1=nn.ReLU()self.fc2=nn.Linear(8,4)self.relu2=nn.ReLU()self.fc3=nn.Linear(4,1)defforward(self,x):x=self.fc1(x)x=self.relu1(x)x=self.fc2(x)x=self.relu2(x)x=self.fc3(x)returnxmod_model=ModuleModel()# Use of the Modelsinput_data=torch.rand(10,12)# Sequentialoutput_seq=seq_model(input_data)# Moduleoutput_mod=mod_model(input_data)

Custom layers in PyTorch are any module or sequence of operations tailored to unique learning requirements. This may include combining traditional operations in novel ways, introducing custom operations, or implementing specialized constraints for certain layers.

1. Subclassnn.Module: This forms the foundation for any PyTorch layer. Thenn.Module captures the state, or parameters, of the layer and its forward operation.

2. Define the Constructor (__init__): This initializes the layer's parameters and any other state it might require.

3. Override theforward Method: This is where the actual computation or transformation happens. It takes input(s) through one or more operations or layers and generates an output.

Here is the Python code:

importtorchimporttorch.nnasnnimporttorch.nn.functionalasFclassCustomLayer(nn.Module):def__init__(self,in_features,out_features,custom_param):super(CustomLayer,self).__init__()self.weight=nn.Parameter(torch.Tensor(out_features,in_features))self.bias=None# Optionally, describe custom parametersself.custom_param=custom_paramself.reset_parameters()# Example: Initialize weights and optional parametersdefreset_parameters(self):# Init codes for weights and optional parametersnn.init.kaiming_uniform_(self.weight,a=math.sqrt(5))defforward(self,x):# Custom computation on input 'x'x=F.linear(x,self.weight,self.bias)returnx

Theforward method is foundational toPyTorch's Module. It links input data to model predictions, embodying the core concept ofcomputational graphs.

PyTorch usesdynamic computational graphs that are built on-the-fly.

1. Back-Propagation: PyTorch creates the graph throughout the forward pass and then uses it for back-propagation to compute gradients. It efficiently optimizes this graph for the available computational resources.

2. Flexibility: Model structure and input data don't need to be predefined. This dynamic approach eases modeling tasks, adapts to varying dataset features, and accommodates diverse network architectures.

3. Layer Connections: Theforward method articulates how layers or units are organized within a model in sequence, branches, or complex network topologies.

4. Custom Functions: Alongside defined layers, custom operations using PyTorch tensors are integrated into the graph, making it profoundly versatile.

A PyTorchModule—whether aModule itself or a derived model like aSequential,ModuleList, orModel—utilizes theforward method for prediction and building the computational graph.

1. Predictions: When you call the model with input data, for example,output = model(input), you're effectively executing theforward method to produce predictions.

2. Graph Construction: As the model processes the input during theforward pass, the dynamic graph adapts, linking the various operations in a sequential or parallel fashion based on the underlying representation.

Here is the PyTorch code:

importtorchimporttorch.nnasnn# Define the neural networkclassSimpleNN(nn.Module):def__init__(self):super(SimpleNN,self).__init__()self.linear1=nn.Linear(10,5)self.activation=nn.ReLU()self.linear2=nn.Linear(5,1)defforward(self,x):x=self.linear1(x)x=self.activation(x)x=self.linear2(x)returnx# Create an instance of the networkmodel=SimpleNN()# Call the model to perform a forward passinput_data=torch.randn(2,10)output=model(input_data)

Optimizers play a pivotal role in guidinggradient-based optimization algorithms. They drive the learning process by adjusting model weights based on computed gradients.

In PyTorch, optimizers likeStochastic Gradient Descent (SGD) and its variants, such asAdam andRMSprop, are readily available.

1. SGD

   • Often used as a baseline for optimization.
   • Adjusts weights proportionally to the average negative gradient.

2. Adam

   • Adaptive and combines aspects of RMSprop and momentum.
   • Often a top choice for tasks across domains.

3. Adagrad

   • Adjusts learning rates for each parameter.

4. RMSprop

   • Adaptive in nature and modifies learning rates based on moving averages.

5. Adadelta

   • Similar to Adagrad but aims to alleviate its learning rate decay drawback.

6. AdamW

   • Essentially Adam with techniques to improve convergence.

7. SparseAdam

   • Efficient for sparse data.

8. ASGD

   • Implements the averaged SGD algorithm.

9. Rprop

   • Specific to its parameter update rules.

10. Rprop

• Great for noisy data.

11. LBFGS
    • Particularly useful for small datasets due to numerically computing the Hessian matrix.

• Learning Rate (lr): Determines the size of parameter updates during optimization.

• Momentum: In SGD and its variants, this hyperparameter accelerates the convergence in relevant dimensions.

• Weight Decay: Facilitates regularization. Refer to the specific optimizer's documentation for variations in its implementation.

• Numerous Others: Each optimizer offers a distinct set of hyperparameters.

1. Instantiation: Create an optimizer object and specify the model parameters it will optimize.

2. Backpropagation: Compute gradients by backpropagating through the network using a chosen loss function.

3. Update Weights: Invoke the optimizer to modify model weights based on the computed gradients.

4. Periodic Adjustments: Optional step allows for optimizer-specific modifications or housekeeping.

Here is the Python code:

importtorchimporttorch.nnasnnimporttorch.optimasoptim# Instantiate a Model and Specify the Loss Functionmodel=nn.Linear(10,1)criterion=nn.MSELoss()# Instantiate the Optimizeroptimizer=optim.SGD(model.parameters(),lr=0.01,momentum=0.9)# Inside the Training Loopforinputs,targetsindata_loader:# Zero the Gradient Buffersoptimizer.zero_grad()# Forward Passoutputs=model(inputs)# Compute the Lossloss=criterion(outputs,targets)# Backpropagationloss.backward()# Update the Weightsoptimizer.step()# Include Additional Steps as Necessary (e.g., Learning Rate Schedulers)# Remember to Turn the Model to Evaluation Mode After Trainingmodel.eval()

In PyTorch,zero_grad() is used toreset the gradients of all model parameters to zero. It's typically employed before a new forward and backward pass in training loops, ensuring that any pre-existing gradients don't accumulate.

Internally,zero_grad() performsbackward() on the model to deactivate gradients for all parameters followed by setting them all to zero. This approach is more efficient for many deep learning models since it avoids the overhead of maintaining gradients when unnecessary.

Here is the Python code:

importtorchimporttorch.optimasoptim# Define a simple model and optimizermodel=torch.nn.Linear(1,1)optimizer=optim.SGD(model.parameters(),lr=0.01)# Initialize input data and targetinputs=torch.randn(1,1,requires_grad=True)target=torch.randn(1,1)# Starting training loopfor_inrange(5):# run for 5 iterations# Perform forward passoutput=model(inputs)loss=torch.nn.functional.mse_loss(output,target)# Perform backward pass and update model parametersoptimizer.zero_grad()# Zero the gradientsloss.backward()# Compute gradientsoptimizer.step()# Update weights

Learning Rate Scheduling in PyTorch adapts the learning rate during training to ensure better convergence and performance. This process is particularly helpful when dealing with non-convex loss landscapes, as well as to balance accuracy and efficiency.

PyTorch'storch.optim.lr_scheduler module provides several popular learning rate scheduling techniques. You can either use them built-in or customize your own schedules.

The common ones are:

• StepLR: Adjusts the learning rate by a factor everystep_size epochs.
• MultiStepLR: Like StepLR, but allows for multiple change points where the learning rate is adjusted.
• ExponentialLR: Multiplies the learning rate by a fixed scalar at each epoch.
• ReduceLROnPlateau: Adjusts the learning rate when a metric has stopped improving.

Here is the Python code:

importtorchimporttorch.optimasoptimimporttorch.nnasnnimporttorch.optim.lr_scheduleraslr_scheduler# Instantiate model and optimizermodel=nn.Linear(10,2)optimizer=optim.SGD(model.parameters(),lr=0.1)# Define LR schedulerscheduler=lr_scheduler.StepLR(optimizer,step_size=5,gamma=0.5)# Inside training loopforepochinrange(20):# Your training code hereoptimizer.step()# Step the schedulerscheduler.step()

In this example:

• We create a StepLR scheduler withstep_size=5 andgamma=0.5. The learning rate will be halved every 5 epochs.
• Inside the training loop, after each optimizer step, we callscheduler.step() to update the learning rate.

• Start with a Fixed Rate: Begin training with a constant learning rate to establish a baseline and ensure initial convergence.
• Tune Scheduler Parameters: Thestep_size,gamma, and other scheduler-specific parameters greatly influence model performance. Experiment with different settings to find the best fit for your data and model.
• Monitor Loss and Metrics: Keep an eye on the training and validation metrics. Learning rate schedulers can help fine-tune your model by adapting to its changing needs during training.

When to use learning rate scheduling:

• Sparse Data: For data with sparse features, scheduling can help the model focus on less common attributes, thereby improving performance.

• Slow and Fast-Learning Features: Not all features should be updated at the same pace. For instance, in neural networks, weights from the earlier layers might need more time to converge. Scheduling can help pace their updates.

• Loss Plateaus: When the loss function flattens out, indicating that the model is not learning much from the current learning rate, a scheduler can reduce the rate and get the model out of the rut.

Backpropagation is a foundational process indeep learning, enabling neural network models to update their parameters.

In PyTorch, backpropagation is implemented with autograd, a fundamental feature that automatically computes gradients.

1. Tensor:torch.Tensor forms the core data type in PyTorch, representing multi-dimensional arrays. Each tensor carries information on its data, gradients, and computational graph context. Neural network operations on tensors get recorded in this computational graph to enable consistent calculation and backward passes.

2. Autograd Engine: PyTorch's autograd engine tracks operations, enabling automatic gradient computation for backpropagation.

3. Function: Every tensor operation is an instance ofFunction. These operations form a dynamic computational graph, with nodes representing tensors and edges signifying operations.

4. Graph Nodes: Represent tensors containing both data and gradient information.

1. Forward Pass: During this stage, the input data flows forward throughout the network. Operations and intermediate results, stored in tensors, are recorded on the computational graph.

# Forward passoutput=model(data)loss=loss_fn(output,target)

2. Backward Pass: After calculating the loss, you call thebackward() method on it. This step initiates the backpropagation process where gradients are computed for every tensor that hasrequires_grad=True and can be optimized.

# Backward passoptimizer.zero_grad()# Clears gradients from previous iterationsloss.backward()# Uses autograd to backpropagate and compute gradients# Gradient descent stepoptimizer.step()# Adjusts model parameters based on computed gradients

3. Parameter Update: Finally, the computed gradients are used by the optimizer to update the model's parameters.

Here is the code:

importtorchimporttorch.nnasnnimporttorch.optimasoptim# Create a simple neural networkclassNet(nn.Module):def__init__(self):super(Net,self).__init__()self.fc=nn.Linear(1,1)# Single linear layerdefforward(self,x):returnself.fc(x)# Instantiate the network and optimizermodel=Net()optimizer=optim.SGD(model.parameters(),lr=0.01)# Fake datasetfeatures=torch.tensor([[1.0], [2.0], [3.0]],requires_grad=True)labels=torch.tensor([[2.0], [4.0], [6.0]])# Training loopforepochinrange(100):# Forward passoutput=model(features)loss=nn.MSELoss()(output,labels)# Backward passoptimizer.zero_grad()# Clears gradients to avoid accumulationloss.backward()# Computes gradientsoptimizer.step()# Updates model parametersprint("Trained parameters: ")print(model.fc.weight)print(model.fc.bias)