This paper explores the plausibility of using the GPU for numerical simulations on structured grids (lattices). The paper (1) reviews previous work on using GPUs for non-graphics applications, (2) implements probability-based simulations on the GPU, namely the Ising and percolation models, (3) implements vector operation benchmarks for the GPU, and (4) compares CPU and GPU performance. The original contribution of this work is implementing Monte Carlo type simulations on the GPU. Such simulations have a wide area of applications. They are computationally intensive and, as shown in the paper, lend themselves naturally to implementation on GPUs, providing a computational speedup. A general conclusion from the results obtained is that moving computations from the CPU to the GPU is feasible, yielding good time and price performance for certain lattice computations. Preliminary results also show that it is feasible to use GPUs in parallel. (S.Tomov, M.McGuigan, R.Bennett, G.Smith, J.Spiletic. Benchmarking and Implementation of Probability-Based Simulations on Programmable Graphics Cards, to appear in Computers & Graphics.)