In the broadest part of Whitehall, just south of the Horse Guards, is the Treasury which was originally built by Sir John Soane, but has been altered and provided with a new façade by Sir Charles Barry, the designer of the Houses of Parliament. It is a handsome, substantial pile, and is a hundred yards in length. Within are the offices of several Government departments, including the Privy Council and the Treasury (of which the Prime Minister for the time being is usually, though not invariably, First Lord). The street seen to the left in our view is Downing Street, which leads to the house that Mr. Gladstone and other Premiers have successively occupied.
source: The Queen's London : a Pictorial and Descriptive Record of the Streets, Buildings, Parks and Scenery of the Great Metropolis, 1896