Wilhelm von Humboldt and Noam Chomsky
Wilhelm von Humboldt's phrase that language is 'the infinite use of finite means' has been cited by Noam Chomsky, who in turn has frequently referred to the concept of infinite language, a speaker's competence, which in Chomsky's terminology is their ability to produce a potentially infinite number of correct sentences. It has also been argued that it is possible to generate an infinitely long sentence along the lines of 'this is the rat that ate the corn stored in the barn by the house that Jack built'.
Language can be theoretically infinite both textually and diachronically, over time. It would probably be more accurate to use a term such as Alan Turing's 'infinitive state', which he applied to computer memory which could always be added to, but would, at any point in time, be limited in actual use. At any given moment, however brief, there are a finite number of texts in existence, even though we couldn't calculate them and the number is changing all the time.
Is Language Infinite Synchronically?
What I would like to look at is whether language is infinite synchronically, at a point in time. To do this, I want to look at an abstracted and theoretical version of language, one where the entire vocabulary has been recorded, together with every single rule for the possible combinations of words that will allow meaningful sentences to be formed. This is our snapshot of language and it holds true until a new word or meaning is formed. When that happens, the system will simply be updated. Every name from every language, every place, etc., are all included; we a have a machine that contains the entire language in terms of words and rules. All dialects, all 'questionable' uses are included; everything that would be accepted and understood by members of the speech community has been codified and included. We have inputted all of this into a mega-computer capable of spewing out trillions of sentences in nanoseconds.
If we take a word like 'book' and consider the list of adjectives that could come before it, we run through our dictionary and produce a list that eventually includes every possible adjective. No use, no matter how implausible or eccentric has been excluded; we keep building the list until no-one can find any more appropriate adjectives. The list runs to tens or hundreds of thousands, but size is not important; what matters is that it has an end. These adjectives can in turn be modified by one or more adverbs, of which there may well be thousands. Nouns can be used in the same way as adjectives, as can other words. We list them all and work out every possible permutation.
Our machine, like Bob Newhart's infinite number of monkeys, pours out trillions of combinations and entire forests are felled to keep the printers stocked with paper. But eventually the printer stops and someone says that it's time to make 'book' plural. Then it will be time to move onto the verbs it goes with, etc. In this sense, it is theoretically possible to generate the entire output of a language at any given time. The numbers involved are so astronomical that they may seem infinite, but they are not; immeasurable and incalculable are not synonymous with infinite. A single neologism could offer trillions and trillions of new combinations, but the synchronic view is always finite; a closed system with a limited number of words that can combine in a limited number of ways cannot produce infinity. The fact that we couldn't get remotely close to exhausting a language's potential supply of sentences does not mean that it is not theoretically possible. Wilhelm von Humboldt seems to me wrong to believe that the finite can generate infinite variety; it can only be used infinitely through repetition.