In order to use Python for mathematics, you need four products.
Windows users might find it easiest to subscribe to and download the Enthought Python Distribution. This includes Numpy, Scipy, as well as Matplotlib and the iPython interactive interpreter. It is a fairly complete distribution.
Linux users can probably get all these things pretty easily also. Ubuntu users would
sudo apt-get install iPython Numpy Scipy Sympy Matplotlib
while Fedora or CentOS users can become root and type
yum install iPython Numpy Scipy Sympy Matplotlib
Both distributions ordinarily install Python as a matter of course.
Note that there are other packages that can do these things - writing mathematical packages for Python is a cottage industry nowadays, and anyone can do it. One well-distributed and supported package is called Sage. It is an alternative to all the packages above, and shares many syntax elements and characteristics with them (because they are all based in Python). However, it is fair to say that Numpy and Scipy have become standard in the mathematical community - Sympy is much less so. We won't discuss this further.
Note that there are currently two streams of Python interpreters. The historical Python version follow the 2.x stream. This is the version we discuss here, for the most part. However, Python is undergoing a major code rewrite, to make many parts more functional and to do some things better technically. This stream has numbers of form 3.x. Both of these streams are fully supported. As of this writing, the current version of the older stream is 2.7, while that for the newer stream is 3.4. These streams are not completely compatible. The reason we use the older form is that there are more packages that comply fully with that. Many packages have not yet been updated to function with the 3.x versions. These upgrades are happening rapidly, though, and we expect support for the 2.x versions to be discontinued soon.