SharpNeuralNetwork

A lightweight neural-network implementation in C#/.NET 8 with inspectable mathematics, reproducible initialization, backpropagation, and numerical verification.

SharpNeuralNetwork implements a sequential dense neural network directly in C#. Its computation path is explicit—from vector and matrix operations through forward propagation, loss gradients, backpropagation, parameter updates, and evaluation—without relying on a prebuilt deep-learning framework.

The current scope is a compact neural-network engine for single-sample binary classification. It is not yet a general-purpose or production-ready ML framework.

What is implemented?

• Linear algebra: vectors, matrices, vector dot products, matrix multiplication, and transpose.
• Activation functions: ReLU, Sigmoid, and Tanh, including derivatives.
• Dense layers: $z = Wx + b$, optional element-wise activation, and gradients for weights, biases, and inputs.
• Weight initialization: seeded He normal initialization for ReLU layers and Xavier/Glorot uniform initialization for other layers.
• Learning: Binary Cross-Entropy, backpropagation, and gradient-descent parameter updates.
• Verification: automated xUnit tests, finite-difference gradient checks, and an executable training/evaluation demonstration.
• Evaluation: predictions and accuracy on a small held-out synthetic classification set.

The core library has no dependency on ML.NET, TensorFlow, PyTorch, TorchSharp, or another ready-made deep-learning framework.

Network in the demonstration

The sample uses a sequential fully connected network for binary classification:

4 input features
      |
      v
 Dense(4 -> 8) + ReLU
      |
      v
 Dense(8 -> 8) + ReLU
      |
      v
 Dense(8 -> 6) + ReLU
      |
      v
 Dense(6 -> 4) + ReLU
      |
      v
 Dense(4 -> 1) + Sigmoid
      |
      v
 Probability in [0, 1] -> binary class

The sample deliberately keeps the architecture small so its execution is easy to inspect in a debugger.

Weight initialization

New dense layers start with zero weights and zero biases. Before training, initialize a network with a fixed seed:

network.InitializeWeights(seed: 42);

The network applies He normal initialization to ReLU layers and Xavier/Glorot uniform initialization to other layers. The seed makes the resulting weights repeatable on the same .NET runtime, which helps reproduce examples and debug training behavior. Biases remain zero.

How learning works

For one input vector $x \in \mathbb{R}^{n}$, a dense layer with $m$ outputs computes:

$$ z = Wx + b, \qquad a = \phi(z) $$

where $W \in \mathbb{R}^{m \times n}$, $b \in \mathbb{R}^{m}$, and $\phi$ is an activation function.

Given the gradient arriving from the next operation, backpropagation applies the chain rule:

$$ \delta = \frac{\partial L}{\partial a} \odot \phi'(z) $$

$$ \frac{\partial L}{\partial W} = \delta x^T, \qquad \frac{\partial L}{\partial b} = \delta, \qquad \frac{\partial L}{\partial x} = W^T\delta $$

Gradient descent then updates each parameter:

$$ W \leftarrow W - \eta\frac{\partial L}{\partial W}, \qquad b \leftarrow b - \eta\frac{\partial L}{\partial b} $$

The implementation exposes these calculations directly in DenseLayer. See Mathematical Foundations for dimensional analysis, the loss derivative, and finite-difference gradient checking.

Verification

The test suite covers:

• vector and matrix operations, including incompatible dimensions;
• activation-function values and derivatives;
• Binary Cross-Entropy values, derivatives, and invalid input handling;
• dense-layer forward propagation and weight, bias, and input gradients;
• network layer-shape validation and backward-pass preconditions;
• analytical weight and bias gradients compared with central finite differences;
• a gradient-descent step that reduces the loss for a simple example.
• seeded initialization, including He variance, Xavier/Glorot bounds, layer-strategy selection, and zero-bias preservation.

The console demonstration also performs an end-to-end training run and evaluates a held-out synthetic test set. The test set is intentionally small and simple: 100% accuracy on it is only a pipeline sanity check, not evidence of generalization or real-world model quality.

GitHub Actions restores and builds the solution, runs the automated test project, and executes the console demonstration.

Run locally

Requires the .NET 8 SDK.

From the repository root:

dotnet restore SharpNeuralNetwork.sln
dotnet build SharpNeuralNetwork.sln --configuration Release --no-restore
dotnet test tests/SharpNeuralNetwork.Tests/SharpNeuralNetwork.Tests.csproj --configuration Release --no-build --no-restore
dotnet run --project examples/SharpNeuralNetwork.Console/SharpNeuralNetwork.Console.csproj --configuration Release --no-build

The console demo reports its seed, hyperparameters, training-loss change, predictions, and held-out test accuracy. It exits with an error if training fails to reduce loss or the small synthetic test set does not meet its sanity-check threshold.

Repository layout

.
├── src/SharpNeuralNetwork/
│   ├── Mathematics/          # Vector and Matrix
│   ├── ActivationFunctions/  # ReLU, Sigmoid, Tanh
│   ├── LossFunctions/        # Binary Cross-Entropy
│   └── NeuralNetworks/
│       ├── DenseLayer.cs
│       ├── NeuralNetwork.cs
│       └── Initialization/    # He normal and Xavier uniform
├── tests/SharpNeuralNetwork.Tests/
│   ├── MathematicsTests.cs
│   ├── NeuralNetworkLearningTests.cs
│   └── WeightInitializationTests.cs
├── examples/SharpNeuralNetwork.Console/
│   └── Program.cs            # Executable training and verification demo
├── docs/
│   ├── MATHEMATICAL-FOUNDATIONS.md
│   └── architecture/adr/
└── .github/workflows/ci.yml

Implementation scope and limitations

This is a compact neural-network implementation, not yet a general-purpose or production-ready ML framework.

• It currently supports sequential fully connected layers and one sample at a time.
• Its matrix operations use straightforward managed C# loops rather than optimized numerical kernels.
• The training demonstration uses gradient descent; optimizers such as Adam, batching, model serialization, GPU execution, and production deployment are outside the current scope.
• The synthetic dataset is designed to make the training pipeline easy to inspect, not to serve as a meaningful benchmark.

The current implementation keeps the full computation path visible:

Linear algebra -> Forward pass -> Loss -> Backpropagation -> Parameter update -> Evaluation

License

MIT. See LICENSE.