People say of something subject to change that it's "not carved in stone." Conversely, carving things in stone has been used throughout history as a sign of permanence, along with dignity and importance. This goes back to, well, the Stone Age. Stone has been used for statues, buildings, monuments, and shrines, from Stonehenge to the Easter Island heads to the Parthenon. Also, it has been used to carve written language, especially decrees of great importance. It wasn't a bunch of sheets of paper that Moses brought down from Mount Sinai, after all. The famous Rosetta Stone had one such decree engraved in several languages of its time, providing an invaluable clue to the decyphering of heiroglyphs. Tombstones have information about the deceased on them for future generations to see. In the era when banks weren't constantly merging and changing their names, they often engraved their name in stone at the front of their building to give themselves the weightiness they thought a place entrusted with the public's money deserved.
Stone carving, very popular in the Roman era, influenced the shapes of the capital letters of the Latin alphabet, which are full of straight lines which are relatively easy to carve. In contrast, the lower case letters, developed by medieval scribes for use with pen and ink on parchment, are full of curves. Traditionally, stone-carved writing often uses the letter V in place of U, in imitation of the Romans for whom they both were the same letter, and the straight-lined version was the one used in carving stone.