K R Parthasarathy Thesis

K R Parthasarathy was awarded a Ph.D. for his doctoral thesis Some Problems of Ergodic Theory and Information Theory. The thesis was typed and is now available on the web. The quality of the reproduction is poor so we give below the Preface and Introduction to the thesis.

Preface.

This thesis is being submitted to the Indian Statistical Institute in support of the author's application for the degree of Doctor of Philosophy. The thesis embodies research carried out by the author during the period 1959-1961 under the supervision of Prof C R Rao at the Indian Statistical Institute, Calcutta.

The thesis is concerned with the development of a duality between the theory of dynamical systems and information theory and its application to problems of information theory. Some of the results have already appeared in an article by the author.

The author is indebted to Messrs R Ranga Rao and S R S Varadhan for their criticisms and suggestions. The author also records his gratefulness to the Research and Training School of the Indian Statistical Institute for providing facilities for research. Thanks to Messrs G M Das and J N Seth for their efficient typing of the thesis.

Introduction.

The subject of ergodic theory which is the mathematical outgrowth of the gas problem in statistical mechanics and the subject of information theory which is the mathematical outgrowth of the problem of information transmission through communication channels have many interesting dual features. In ergodic theory we study collectively and individually the measure preserving transformations of a fixed measure space, whereas in information theory we do the same with the set of all measures in variant under a fixed measurable transformation of a measurable space. The object of this thesis is to study some of the dual relations between the two theories with emphasis on the information-theoretic aspect.

In the first chapter we take up the global problem. Here the problem of ergodic theory is to ensure that there is a large class of ergodic transformations in any non-atomic measure space. If

(X, S, \mu)

is a finite measure space and

G

the group of all one to one measure preserving transformations, then two interesting topologies on be assigned to

G

which make it a topological group. In dynamical problems it is of interest to know whether a particular transformation is ergodic or not. Even though this problem has defied solution till now, the existence of a large class of ergodic transformations has been shown by the determination of their category in

G

. In particular, Halmos proved that the set of weakly mixing transformations is a dense

G_{0}

G

under the weak topology. Similar results were proved earlier by Oxtoby and Ulam. Rokhlin proved that under the same weak topology in

G

, the set of strongly mixing transformations is a set of first category. Here the main tool is to show the density of what are called periodic transformations.

In problems of information-theoretic interest, we have a fixed measurable space

(X, S)

and a one-to-one both ways measurable map

T

X

onto itself. Here, it is of interest to know whether there are a lot of ergodic measures in the space of invariant probability measures. In order to study this problem, we take

X

to be a topological space,

S

the Borel

\sigma

-field and

T

a homeomorphism of

X

onto itself. Taking

X

to be a complete and separable metric space and assigning the weak topology to the space of invariant probability measures, we show that the set of ergodic measures is a

G_{0}

. Now the question arises as to what are the complete separable metric spaces and what are the homeomorphisms under which the ergodic measures are dense. This classification problem has not been solved even in the case of a compact metric space. But, however, in spaces of information-theoretic importance we have solved this problem. When

X

is a countable product of complete and separable metric spaces and

T

is the shift transformation, we show that the ergodic measures form a dense

G_{0}

under the weak topology. In this context it is not without interest to note that the ergodic measures constitute the set of extreme points of the convex set of all invariant probability measures. Examples are given to show that ergodic measures need not be dense in the general case.

We introduce the concept of periodic invariant measures and study their structure. We show that, in the case of shift transformation, the periodic measures are dense in the weak topology and thereby deduce the first category nature of the set of strongly mixing measures. Further, whenever the periodic measures are dense in the weak topology the closure of the set of periodic points is of invariant measure one. The converse problem remains open.

In the second chapter we introduce the notions of entropy of a stationary source and rate of transmission of a stationary channel and study some of their properties. In the problem of classification of measure-preserving transformations the fundamental role of the notion of entropy as a metric invariant has been demonstrated in the recent works of A N Kolmogorov. In this connection V A Rokhlin points out that there is a large class of measure-preserving transformations with zero entropy and hence in such cases entropy happens to be a trivial invariant. This is done by examining the category of the set of transformations with zero entropy in the space of all measure-preserving transformations under a suitable topology. We have shown that in the case of shift transformation the set of all ergodic probability measures with zero entropy is a dense

G_{0}

in the space of all invariant probability measures.

Then we study the problem of representing the entropy of a stationary information source as an integral of the entropies of its ergodic components. We do the same for the rate of transmission of a stationary input distribution through a stationary communication channel. The continuity properties of different functionals associated with a channel are discussed.

In the last chapter we study the applications of the results obtained in the first and second chapters. We prove that the stationary and ergodic capacities are equal for an arbitrary stationary channel. This has been proved for channels with finite memory by I P Tsaragradsky, A Feinstein and L Breiman. We then study the problem of achievement of capacity for channels of finite memory in the sense of Feinstein. Finally we give a limiting form of the famous Feinstein's fundamental lemma which throws some light on the assumptions under which the lemma is proved.

Last Updated October 2026