A proof of convergence and equivalence for 1D finite element methods and ReLU neural networks

Authors

  • Ronald Katende Department of Mathematics, Kabale University, Uganda
  • Henry Kasumba Department of Mathematics, Makerere University, Uganda
  • Godwin Kakuba Department of Mathematics, Makerere University, Uganda
  • John M. Mango Department of Mathematics, Makerere University, Uganda

DOI:

https://doi.org/10.56947/amcs.v25.392

Keywords:

Numerical Analysis, Rectified Linear Unit Function, Finite Element Method, Neural Networks, Equivalence

Abstract

 This paper investigates the convergence and equivalence properties of the Finite Element Method (FEM) and Rectified Linear Unit Neural Networks (ReLU NNs) in solving differential equations. We provide a detailed comparison of the two approaches, highlighting their mutual capabilities in function space approximation. Our analysis proves the subset and superset inclusions that establish the equivalence between FEM and ReLU NNs for approximating piecewise linear functions. Furthermore, a comprehensive numerical evaluation is presented, demonstrating the error convergence behavior of ReLU NNs as the number of neurons per basis function varies. Our results show that while increasing the number of neurons improves approximation accuracy, this benefit diminishes beyond a certain threshold. The maximum observed error between FEM and ReLU NNs is 10-4, reflecting excellent accuracy in solving partial differential equations (PDEs). These findings lay the groundwork for integrating FEM and ReLU NNs, with important implications for computational mathematics and engineering applications.

Downloads

Download data is not yet available.

Downloads

Published

2024-11-16

Issue

Section

Articles