Example in architecture: Aqueducts. In fact, in some places in Europe, the original roman Aqueducts are still used for transporting water! Another are the Roman roads: some of the Roman roads, some of which are still (with modification) used today: this is possible because the Romans used a specially efficient system for building their roads.
Example in language: The language of the Roman empire is still used today! Well, it evolved and changed, but it formed the basis for many modern languages, such as Italian and Spanish.
But as far as "ideas" are concerned, a good example would be the Roman script, which is still used today (it's the script I am writing in now, too). It wasn't invented by the Romans, but it was adapted by them from the Greek. Still, the version we use today is using the changes that the Romans made.
