In this digital era, electronic circuit is the key component and its design is tested and validated through simulator. Simulator uses mathematical model to replicate circuit behavior. All electronic designs rely truly on simulation software. But even though simulation is cost-effective; large circuit simulation is relatively time consuming. Also various iterations in transient analysis may make simulation slower. Fast simulator is the basic requirement for large circuit simulation. In this paper, we have addressed parallel computing approach using Graphics Processing Unit(GPU) to accelerate simulation. As GPU is many core processor, compute intensive functions are redesigned to execute on GPU. Matrix operations, linear- nonlinear equations, integration, differential equations, numerical methods are some of the very basic operations required in circuit analysis. Mathematical operations are redesigned to get clusters of sufficient size. Forming clusters of circuit components and mathematical procedures proves to be crucial, for reliable mapping to graphics processor. Loop replacement, data-code partitioning, parallel data mapping, reductions, fast memory access are the strategies adopted for parallel processing on GPU. More than 40% speed gain is achieved on circuit having at least four components and transient analysis for more than thousand iterations.