First time the name of the town (Gelvin) appears in Crusaders’ chronicles in 1385.   By the first version the name of the town derives from vernacularism “gelvonis” which means “bee’s sting”; the other version is that the name of the town derives out of the lake Gelva (Gelves) name that is situated 5 kilometres away form the town. Near that lake the initial town place might be.

The history of the town is related to the history of polish and lithuanian Pliateris grandees family, that sprung from 13^th century Westphalia knight Herebold Brejelietis, dubbed Pliateris, loins. One of this family heirs, who was the owner of the manor in 1895 – 1897, built modern church of St. Virgin Maria Visit. The church is built in Neoclassicism style with Neobaroque elements, and no tower. The church is built out of masonry that is harmonized with elements of red blocks.

In the graveyard of the church there is a 19^th century chapel. A big stone about 90x36 cm in size with a man’s foot-like print is set to the wall of the chapel. There is a story that long time ago a woman was passing by the thick forests near Gelvonai place. And Mother of God had appeared to the woman on a stone. When the vision was over the woman noticed a footprint on the stone. After that the forest was cut down and near that stone the chapel was built. In the graveyard there is a grandees’ Pliateriai crypt.  

In 1848 – 1849 a poet and bishop Antanas Baranauskas, who wrote a poem “Anyksciu silelis”, was living in Gelvonai. When he was 14 years old he was working as a menial at parson’s house. In 1998 a monument in remembrance to the Great War partisans was built in Gelvonai (the sculptor is Jonas Jagela).  

The most interesting place in Gelvonai district is Mikalajunu street-village that was founded in 16^th century during Wallachian reform. There are only couple villages of this type in Lithuania, because during the later reforms they were reorganized to steadings.