The name of the town comes from Latin castrum, and means "fortified place". Castres grew up round the Benedictine abbey of Saint Benoît, which is believed to have been founded in AD 647, possibly on the site of an old Roman fort (castrum). Castres became an important stop on the international pilgrimage routes to Santiago de Compostela in Spain because its abbey-church, built in the 9th century, was keeping the relics of Saint Vincent, the renowned martyr of Spain. It was a place of some importance as early as the 12th century, and ranked as the second town of the Albigeois behind Albi. Despite the decline of its abbey, which in 1074 came under the authority of Saint Victor abbey in Marseille, Castres was granted a liberal charter in the 12th century by the famous Trencavel family, viscounts of Albi. Resulting from the charter, Castres was ruled by a college of consuls. During the Albigensian Crusade it surrendered of its own accord to Simon de Montfort, and thus entered into the kingdom of France in 1229. In 1317, Pope John XXII established the bishopric of Castres. In 1356, the town of Castres was raised to a countship by King John II of France. However, the town greatly suffered from the Black Plague in 1347-1348, then from the Black Prince of England and the Free Companies (bands of lawless mercenaries) who laid waste the country during the Hundred Years' War. Consequently, by the late 14th century Castres entered a period of sharp decline. In 1375, there were only 4,000 inhabitants left in town, only half the figure from a century before. Following the confiscation of the possessions of Jacques d'Armagnac, duke of Nemours, to which the countship of Castres had passed, it was bestowed in 1476 by King Louis XI on Boffille de Juge (Boffillo del Giudice), an Italian nobleman and adventurer serving as a diplomat for Louis XI, but the appointment led to so much disagreement (family feud between Boffille de Juge, his only daughter, and his brother-in-law) that the countship was united to the crown by King Francis I in 1519. Around 1560, the majority of the population of Castres converted to Protestantism. In the wars of the latter part of the 16th century the inhabitants sided with the Protestant party, fortified the town, and established an independent republic. Castres was one of the largest Protestant strongholds in southern France, along with Montauban and La Rochelle. Henry of Navarre, leader of the Protestant party, who later became King Henry IV of France, stayed in Castres in 1585. The Protestants of Castres were brought to terms, however, by King Louis XIII in 1629, and Richelieu came himself to Castres to have its fortifications dismantled. Nonetheless, after these religious wars, the town, now in peace, enjoyed a period of rapid expansion. Business and traditional commercial activities revived, in particular fur and leather-dressing, tanning, and above all wool trade. Culture flourished anew, with the founding of the Academy of Castres in 1648. Castres was turned by the Catholic Church into an active center of Counter-Reformation, with the establishments of several convents in town, and the building of a renowned bishop's palace by Mgr. Tubœuf, still the most famous monument in town today. A new cathedral was also built, after the destructions of the religious wars. Perhaps even more important, Castres was made the seat of the Chambre de l'Édit of the Parliament of Toulouse, a court of justice detached from the Parliament of Toulouse and in charge of dealing with the cases involving the Protestants of Languedoc, a measure of protection granted to them by the Edict of Nantes. This court attracted lots of business to Castres. In 1665, there were 7,000 inhabitants in Castres, 4,000 of whom Catholic, and 3,000 Protestant.
In 1670 however, the Chambre de l'Édit was transferred to Castelnaudary, much to the discontent of even the catholic citizens of Castres, who lost a major source of business and revenue with the departure of the lawyers and the plaintiffs. The Revocation of the Edict of Nantes soon followed, and Castres suffered a lot when a great number of Protestants chose to go into exile. Then came the plague of 1720-1721 and the fire of 1724. Last but not least, Castres lost its liberal charter in 1758. In the 1760s, a few years after the famous Calas Affair in Toulouse, Castres made the headlines nationwide: Pierre-Paul Sirven and his wife, both Protestants, were wrongly accused of having murdered their daughter in order to prevent her from converting to Catholicism. Tried and sentenced to death in absentia on March 29, 1764, they were defended by Voltaire, and eventually exonerated in 1771. The outbreak of the French Revolution was generally welcomed in Castres, particularly among the local Protestant merchants and entrepreneurs, but the majority of the population remained moderate during the whole period. In 1793 for instance, Protestant pastor Alba La Source, Castres' representative at the Convention in Paris, opposed the deportation of "non-juror" Catholic priests to French Guiana, where death in the horrid jungle was certain (see Civil Constitution of the Clergy). "Non-juror" priests were by far the majority in the region of Castres. Accused of being a moderate, Alba La Source was guillotined in October 1793. Suspected of being lukewarm toward the revolution, Castres was duly chastised. The bishopric which had been established by Pope John XXII in 1317 was abolished, Castres later becoming part of the bishopric of Albi. Capital of the département of Tarn in 1790, the town was downgraded to capital of an arrondissement in 1797, Albi being made the capital of the département. Despite these setbacks, in the 19th century the economy of Castres developed greatly, and the town grew outside of its old medieval center. As early as 1815, the first mechanized wool mill was set up in town. Originally specialized in luxury cloth, the Castres textile industry then turned toward more ordinary types of cloth, whose markets were considerably larger. Around 1860, there were 50 wool mills in town, employing 3,000 people. In the end of the 19th century, mechanical engineering industries appeared beside the textile industry, which led to Castres becoming a major arsenal for the French army during the First World War. Castres was linked to the French railway network in 1865. At the end of the 19th century, Castres was the largest town in the département of Tarn, with 5,000 more inhabitants than Albi. However, in the 20th century the town entered a new period of decline. Although Castres emerged from the two world wars unscathed, no military operations or combats taking place in southwest France, the local economy has been hard hit by change. Like so many towns and cities of Europe which had benefited most from the Industrial Revolution, Castres is experiencing a difficult restructuring of its industrial base. Textile has particularly suffered. Castres is also crippled by its geographical location, isolated in a dead end at the foot of the Massif Central mountains, away from the main exchange and transport routes. Castres is still not connected to the motorway (freeway) network of France, the only town of that size in France not yet connected. The creation of the Greater Castres-Mazamet Council in 2000 was expected to deal with the transport problem, and to work on attracting new industries. The good fortune of Castres is to be located only 79 kilometres (49 mi) away from the very dynamic Toulouse. The long promised motorway link with Toulouse is due to be completed soon, and Castres is hoping to benefit from its proximity with the big Occitan city.