Firewall,as a mechanism of compulsory access control between the network or system,is an important means to ensure the network security.Firewall can be a very simple filter,but also it can be a carefully targeted gateway.But the principle is the same,which is monitoring and filtering all the information exchanged in internal and external networks.Linux as an open source operating system,is famous for it's stability and security.netfilter/iptables is a firewall system based on linux which has a great function.This thesis first analysed the working principle of iptables,then introduced iptables rule set,and last proposed an effective algorithm to optimize the rules set which is implemented based on linux system.In the part of implementation,some key code of the algorithm are given.