Hidden Variables and Non-locality

Review of hidden-variable theories
In $1.3 we saw that it is possible to repeat an experiment several
times, under apparently exactly the same conditions, and yet obtain
different results. In particular, for example, we could direct identical
particles, all with the same velocities, at identical potential
barriers, and some would be reflected and some transmitted. The
initial conditions would not uniquely determine the outcome.
Quantum theory, as explained in Chapter Two, accepts this lack
of determinism; knowledge of the initial wavefunction only
permits probabilistic statements regarding the outcome of future
measurements.
Hidden-variable theories have as their primary motivation the
removal of this randomness. To this end they regard the
‘apparently’ identical initial states as being, in reality, different;
distinguished by having different values of certain new variables,
not normally specified (and therefore referred to as ‘hidden’). The
states defined in quantum theory would not correspond to precise
values of these variables, but rather to certain specific averages over
them. In principle, however, other states, which do have precise
values for these variables, could be defined and with such initial
states the outcome of any experiment would be uniquely
determined. Thus determinism, as understood in classical physics,
would apply to all physics. Particles would then have, at all times,
precise positions and momenta, etc. The wavefunction would not
be the complete description of the system and there would be the
possibility of solving the problems with wavefunction reduction
which we met in Chapter Three. This latter fact is, to me at least,
a more powerful motivation than the desire for restoration of
determinism.
Any satisfactory hidden-variable theory must, of course, agree
with experimental observations and therefore, in particular, with
all the verified predictions of quantum theory. Whether it should
agree exactly with quantum theory, or whether it might deviate
from it to a small degree, while still remaining consistent with
experiment, is an open question, The normal practice seems to have
been to seek hidden-variable theories for which the agreement is
exact. A hidden-variable theory will, of course, tell us more than
quantum theory tells us-for example, it tells us which particles will
pass through a given barrier. What we require is that it gives the
same, or very nearly the same, results for those quantities that
quantum theory can predict.
There have been, and still are, many physicists who would regard
the question of the possiblity of such a hidden-variable theory,
agreeing in all measurable respects with quantum theory, as being
an unimportant issue. Readers who are still with us, however, are
presumably convinced that the quest for reality is meaningful, so
they will take a different view. The question is interesting and
worthy of our attention. Indeed, there are even pragmatic grounds
for pursuing it: different explanations of a set of phenomena, even
though they agree for all presently conceivable experiments, may
ultimately themselves suggest experiments by which they could be
distinguished. There is also the hope that better understanding of
quantum theory might help in suggesting solutions to some of the
other unsolved problems of fundamental physics.
The subject of hidden-variable theories was for many years
dominated by an alleged ‘proof’, given by von Neumann in 1932
(in his book Mathematische Grundlagen der Quantenmechanik
[Berlin : Springer] , English translation published by Princeton
University Press, 1955), that such theories were impossible, i.e. that
no hidden-variable, deterministic, theory could agree with all the
predictions of quantum theory. The proof was simple and elegant;
its mathematics, though subject to much scrutiny, could not be
challenged. However, the mathematical theorem did not really
have any relevance to the physical point at issue. The reason for
this lay in one of the assumptions used to prove the theorem. We
shall give a brief account of this assumption in the following paragraph. Since this account is rather technical and not used in the
subequent discussion, some readers may prefer to omit it.
Let us suppose that two quantities, call them X and Y, can be
separately measured on a particular system, and that it is also
possible to measure the sum of the two quantities, X + Y, directly.
Then the assumption was that the average value of X + Y, over any
collection of identical systems, i.e. any ensemble, was equal to the
average value of X plus the average value of Y. Since, in general,
the variable X+ Y is of a different kind, measured by a different
apparatus, from either X or Y, there is no reason why such an
equality should hold. Von Neumann was led to assume it because
it happens to be true in quantum theory, i.e. for those ensembles
specified by a given wavefunction. In a hidden-variable theory,
however, other states, defined by particular values of the hidden
variables, can, at least in principle, exist, and for such states the
assumption does not have to be true. Although several people
seemed vaguely to have realised this problem with von Neumann’s
theorem, it was not until 1964 that John Bell finally clarified the
issue, and removed this theoretical obstacle to hidden-variable
theories. The article was published in Reviews of Modern Physics
38 447 (1966).
At this stage we should emphasise that, although hidden variable
theories are possible, they are, in comparison to quantum theory,
extremely complicated and messy. We know the answers from
quantum theory and then we construct a hidden-variable, deterministic,
theory specifically to give these answers. The resulting
theory appears contrived and unnatural. It must, for example, tell
us whether a given particle will pass through a potential barrier for
all velocities and all shapes and sizes of the barrier. It must also tell
us ihe results for any type of experiment; not only for the
reflection/transmission barrier experiment of 0 1.3, but also for the
experiment with the mirrors. In the latter case, there can now be
no question of interference being the real explanation of what is
happening, because a given particle is certainly either reflected or
transmitted by the barrier and hence can only follow one path to
the detectors. Nevertheless, although it reaches only one of the
mirrors, which reflects it to the detectors, the path it follows must
be influenced by the other mirror. This is brought about by the
introduction of a new ‘quantum force’ which can act over
arbitrarily large distances. This quantum force is constructed in
order to give the required results. For details of all the various hidden-variable theories that are
available we refer to the excellent book by Belinfante, A survey of
hidden-variable theories [Oxford: Pergamon 19731. Here, we shall
only discuss a particular class of such theories; they appear to be
the most plausible and are the topic of our next section.