capacity of channel with given restriction on power and distortion
XOR distortion(x, y, xhat) = x (xor) y (xor) xhat
Determine capacity for various values of distortion
Power allocation P=P_1+P_2 for two Gaussian sources (variance sigma_i) conveyed over channels with AWGN (variance n_i).
Appropriate feature selection and weighting is crucial for clustering algorithms to successfully handle multi-dimensional data. A feature is relevant when it is correlated with the classification, mutually independent of other features, but possibly correlated with other features. For any feature, these characteristics, and hence the weighting, can be determined using information theoretic quantities, e.g., mutual information with other features and the veridical cluster assignment available from training data. An application of the technique to feature weighting in a speech separation task is presented.
