This paper describes a framework for the implementation of linear algebra operators on GPUs, providing the building blocks for the design of more complex numerical algorithms. The framework takes advantage of sparse and banded matrices in particular. The paper demonstrates the approach by implementing direct solvers for sparse matrices with application to multi-dimensional finite difference equations, i.e. the 2D wave equation and the incompressible Navier-Stokes equations. (Linear Algebra Operators for GPU Implementation of Numerical Algorithms. Jens Krüger and Rüdiger Westermann. To appear in the proceedings of SIGGRAPH 2003.)