In many ways, the story of dog evolution follows the same plot line as the evolution of horses and elephants: a small, inoffensive, ancestral species gives rise, over the course of tens of millions of years, to the respectably sized descendants we know and love today. But there are two big differences in this case: first, dogs are carnivores, and the evolution of carnivores is a twisty, serpentine affair involving not only dogs, but prehistoric hyenas, bears, cats, and now-extinct mammals like creodonts and mesonychids. And second, of course, dog evolution took a sharp right turn about 15,000 years ago, when the first wolves were domesticated by early humans.
As far as paleontologists can tell, the very first carnivorous mammals evolved during the late Cretaceous period, about 75 million years ago (the half-pound Cimolestes, which lived high up in trees, being the most likely candidate). However, it's more likely that every carnivorous animal alive today can trace its ancestry back to Miacis, a slightly bigger, weasel-like creature that lived about 55 million years ago, or 10 million years after the dinosaurs went extinct. Miacis was far from a fearsome killer, though: this tiny furball was also arboreal, and feasted on insects and eggs as well as small animals.
Before the Canids - Creodonts, Mesonychids & Friends
Modern dogs evolved from a line of carnivorous mammals called "canids," after the characteristic shape of their teeth. Before (and alongside) the canids, though, there were such diverse families of predators as amphicyonids (the "bear dogs," typified by Amphicyon, which seem to have been more closely related to bears than dogs), prehistoric hyenas (Ictitherium was the first of this group to live on the ground rather than in trees), and the "marsupial dogs" of South America and Australia. Although vaguely dog-like in appearance and behavior, these predators weren't directly ancestral to modern canines.
Even more fearsome than bear dogs and marsupial dogs were mesonychids and creodonts. The most famous mesonychids were the one-ton Andrewsarchus, the largest ground-dwelling carnivorous mammal that ever lived, and the smaller and more wolflike Mesonyx; oddly enough, mesonychids were ancestral not to modern dogs or cats, but to prehistoric whales. The creodonts, on the other hand, left no living descendants; the most noteworthy members of this breed were Hyaenodon and the strikingly named Sarkastodon, the former of which looked (and behaved) like a wolf and the latter of which looked (and behaved) like a grizzly bear.
The First Canids - Hesperocyon and the "Bone-Crushing Dogs"
Paleontologists agree that the late Eocene (about 40 to 35 million years ago) Hesperocyon was directly ancestral to all later canids--and thus to the genus Canis, which branched off from a subfamily of canids about six million years ago. This "western dog" was only about the size of a small fox, but its inner-ear structure was characteristic of later dogs, and there's some evidence that it may have lived in communities, either high up in trees or in underground burrows. Hesperocyon is very well-represented in the fossil record; in fact, this was one of the most common mammals of prehistoric North America.
Another group of early canids were the borophagines, or "bone-crushing dogs," equipped with powerful jaws and teeth suitable for scavenging the carcasses of mammalian megafauna. The largest, most dangerous borophagines were the 100-pound Borophagus and the even bigger Epicyon; other genera included the earlier Tomarctus and Aelurodon, which were more reasonably sized. We can't say for sure, but there's some evidence that these bone-crushing dogs (which were also restricted to North America) hunted or scavenged in packs, like modern hyenas.
The First True Dogs - Leptocyon, Eucyon and the Dire Wolf
Here's where things get a bit confusing. Shortly after the appearance of Hesperocyon 40 million years ago, Leptocyon arrived on the scene--not a brother, but more like a second cousin once removed. Leptocyon was the first true canine (that is, it belonged to the caninae subfamily of the canidae family), but a small and unobtrusive one, not much bigger than Hesperocyon itself. The immediate descendant of Leptocyon, Eucyon, had the good fortune to live at a time when both Eurasia and South America were accessible from North America--the first via the Bering land bridge, and the second thanks to the uncovering of central America. In North America, about six million years ago, populations of Eucyon evolved into the first members of the modern dog genus Canis, which spread to these other continents.
But the tale doesn't end there. Although canines (including the first coyotes) continued to live in North America during the Pliocene epoch, the first plus-sized wolves evolved elsewhere, and "re-invaded" North America shortly before the ensuing Pleistocene (via that same Bering land bridge). The most famous of these canines was the Dire Wolf, Canis diris, which evolved from an "old world" wolf that colonized both North and South America (by the way, the Dire Wolf competed directly for prey with Smilodon, the "saber-toothed tiger.")
The end of the Pleistocene epoch witnessed the rise of human civilization around the world. As far as we can tell, the first domestication of the Gray Wolf occurred somewhere in Europe or Asia anywhere from 30,000 to 15,000 years ago. After 40 million years of evolution, the modern dog had finally made its debut!
Here's a list of the most notable prehistoric dogs and dog ancestors; just click on the links for more information.
Aelurodon This "cat-toothed" dog behaved more like a hyena.
Amphicyon Otherwise known as the "bear dog."
Borophagus One of the biggest of the bone-crushing canids.
Cynodictis This was once thought to be the first true dog.
Dire Wolf A giant wolf of the Pleistocene epoch.
Dusicyon This "foolish dog" went extinct in the 19th century.
Epicyon This prehistoric dog was built more like a big cat.
Eucyon The immediate predecessor of modern canines.
Hesperocyon An early ancestor of the modern dog.
Ictitherium One of the first ground-dwelling hyenas.
Leptocyon A tiny, foxlike ancestor of the modern dog.
Tomarctus A bone-crushing dog of the Miocene epoch.