HIGH-ORDER NETWORKS THAT LEARN TO SATISFY LOGIC CONSTRAINTS

Logic-based problems such as planning, formal verification and inference, typically involve combinatorial search and structured knowledge representation. Artificial neural networks (ANNs) are very successful statistical learners; however, they have been criticized for their weaknesses in representing and in processing complex structured knowledge which is crucial for combinatorial search and symbol manipulation. Two high-order neural architectures are presented (Symmetric and RNN), which can encode structured relational knowledge in neural activation, and store bounded First Order Logic (FOL) constraints in connection weights. Both architectures learn to search for a solution that satisfies the constraints. Learning is done by unsupervised "practicing" on problem instances from the same domain, in a way that improves the network-solving speed. No teacher exists to provide answers for the problem instances of the training and test sets. However, the domain constraints are provided as prior knowledge encoded in a loss function that measures the degree of constraint violations. Iterations of activation calculation and learning are executed until a solution that maximally satisfies the constraints emerges on the output units. As a test case, block-world planning problems are used to train flat networks with high-order connections that learn to plan in that domain, but the techniques proposed could be used more generally as in integrating prior symbolic knowledge with statistical learning.