On the Space of Coefficients of a Feedforward Neural Network

We establish the conditions for 'equivalent neural networks' - neural networks with different weights, biases, and threshold functions which result in the same associated function. We prove that given a neural network N with piecewise linear activation, the space of coefficients describing all equivalent neural networks is given by a semialgebraic set. This result is obtained by studying different representations of a given piecewise linear function using the Tarski-Seidenberg theorem. Given a neural architecture and an initial neural network N-0, specified by coefficients, we give an algorithm to compute inequalities defining the corresponding semiaglebraic sets. These algorithms are based on the rules of max-plus algebra.