We propose an algorithm to compute the spectral set of a polytope of polynomials. The proposed algorithm offers several key guarantees that are not available with existing techniques. It guarantees that the generated spectral set: (i) contains all the actual points, (ii) is computed to a prescribed accuracy, (iii) is computed reliably in face of all kinds of computational errors, and (iv) is computed in a finite number of algorithmic iterations. A further merit is that the computational complexity of the proposed algorithm is O(n) in contrast to O(n(2)) for existing techniques, where n is the degree of the polynomial. The algorithm is demonstrated on a few examples.