The term "smoking gun" was originally, and is still primarily, a reference to an object or fact that serves as conclusive evidence of a crime or similar act. In addition to this, its meaning has evolved in uses completely unrelated to criminal activity: for example, scientific evidence that is highly suggestive in favor of a particular hypothesis is sometimes called smoking gun evidence. Its name originally came from the idea of finding a smoking (i.e., very recently fired) gun on the person of a suspect wanted for shooting someone, which in that situation would be nearly unshakable proof of having committed the crime. A piece of evidence that falls just short of being conclusive is sometimes referred to as a "smoldering gun."

The phrase originated in the Sherlock Holmes story, The Gloria Scott (1893). The origin and development of "smoking gun" was described by William Safire in his column, "On Language," as follows:

When did that phrase first become the favorite figure of speech meaning incontrovertible incrimination? The answer is elementary, Watson. In an 1893 Sherlock Holmes story, The Gloria Scott, Arthur Conan Doyle wrote of a grisly murder by a sham chaplain aboard a prison ship: We rushed into the captain's cabin . . . there he lay with his brains smeared over the chart of the Atlantic . . . while the chaplain stood with a smoking pistol in his hand at his elbow. A good copy editor would have fixed Doyle's awkward in his hand at his elbow, and Sir Arthur chose pistol rather than gun, but that Holmes citation seems to be the start of the cliché that grips us today.