Insights into the nature of problem-solving

Problem-solving can be likened to exploring on foot an uncharted wilderness - it is full of obstacles to be overcome; trails which lead to dead ends; unexpected twists and turns; and a lot of thrashing about in dense undergrowth! To get a sense of this reality, the table below charts Einstein's struggle to develop his General Theory of Relativity.1 Don't worry that you probably won't be able to understand everything about the science that is said below, just getting the general gist will be enough. 

Date Einstein's (E) endeavours Comment
Nov 1907 E considers how Newton's law of gravitation has to be modified in light of his recently developed Special Theory of Relativity (STR).2 Comes up with his Equivalence Principle: locally, a uniform gravitational field can't be distinguished from uniform acceleration away from gravity.
From this (correctly) predicts gravitational redshift and gravitational time dilation over short vertical distances.
Predicts that the path of light will be bent by a gravitational field, but comes up with ½ the correct answer.
Demonstrates how in problem-solving, early thoughts might be on the right track but may not be completely correct. I.e. there is often quite a bit of trial-and-error work in problem-solving.
Jun 1911 Refines 1907 thinking. (In intervening period worked on problems in quantum mechanics.)  
Early 1912 Equivalence Principle leads E to the (only half correct) view that time but not space is warped, and writes several papers on that basis, but suspects this view might be wrong.
Reaches limits of kinematical analysis due to the fact that the Equivalence Principle is only valid locally, not globally.
One has to be prepared to face in problem-solving that one's initial approach may not work out and so may have to be dropped for other approaches. Nevertheless, often insight into the nature of the problem can be gained and so is a useful first step.
Later in 1912 Realises:
  • Lorentz transformations not enough.
  • variational principle gives equation of motion of a test particle, but doesn't know how to calculate some of the expressions in this equation.
  • mass and energy are sources of the field, not just mass as in Newtonian theory.
To get beyond the local nature of the Equivalence Principle, views gravity as being describable by the local speed of light which varies from place to place in the gravitational field (a "scalar c-field").
Gaining insights, but still hasn't found the fully correct route to a solution yet.
In a letter, E wrote: "Every step is devilishly difficult."
Aug 1912 Realises his scalar c-field idea is not enough to resolve all problems. Concludes on the basis of an analysis of a rotating turntable that Euclidean geometry won't hold (i.e. space as well as time is warped by a gravitational field) and that the theory of differential geometry might be the way forward.
This led E to the idea that gravity would be described by a field of 10 dynamical quantities equation.gif(a tensor) rather than just one (a scalar).
Sometimes working out specific cases, which may be simpler to solve, rather than the general problem, may lead to insights which can lead to a general solution.
Early 1913 Develops (correctly), the broad program of the General Theory of Relativity (GTR) as being described by Riemmanian geometry.
Incorrectly believes the fields are not generally covariant, but only covariant with respect to linear transformation of coordinates. Realises this is wrong in August 1913.
 
Late 1913 (Incorrectly) posits the gravitational field equations to be of the form R = constant x T. Science often progresses through the making of conjectures which are then tested. If they don't work out, they get trashed and another idea is sought. Such is life.
1914 In a paper on GTR this year, E still posits (incorrectly) that the equation.gif are uniquely determined by the source of the gravitational field. It took E three years to overcome his "prejudice" that the equation.gif should be determined uniquely by the gravitational source. This shows how preconceived ideas can get in the way of the problem-solving process, even for geniuses like E.
Jul-Oct 1915 E found objections to his "old" theory of gravitation.  
Nov 1915 Publishes a series of 4 papers on GTR; first 2 are steps forward, the third a step back; while the fourth is the final complete and correct theory.
With complete theory, finds (correctly) bending of light by sun to be twice what he had previously predicted.
The torturous route E took in developing his GTR supports the claim made above that research and problem-solving is often like exploring an unchartered wilderness on foot.

1. Source: A. Pais, "Subtle is the Lord ...": the science and life of Albert Einstein, Oxford: Oxford University Press, 1982.
2. STR says things can't travel faster than the speed of light, whereas Newton's law of gravitation posits that gravity effectively travels with an infinite speed, i.e. instantaneously.