Elegante é mais ou menos o mesmo que simples, limpo, econômico e bonito.  Um programa elegante não tem código supérfluo nem variáveis desnecessárias.

A experiência mostra que programas elegantes são, em geral, rápidos e programas rápidos são, em geral, elegantes.

"First, beautiful software ought to be visually beautiful -- the written code I mean -- and almost a pleasure to read. I don't equate cleverness with beauty necessarily. Too often very clever code is actually quite ugly to read and understand.  Secondly, the code ought to be lean, spare. However large and powerful the programming language used is, I am not impressed by someone's use of every possible construct in the language, but rather their mastery of key constructs. Again, cleverness and a showy knowledge don't impress me so much as a kind of modest strength and clarity."
— Don Sherwood Olson, writer and former software engineer