Statistics plays a fundamental role in daily life, allowing costly medical screening, drug development, marketing campaigns or government regulation to be better targeted through improved understanding of the scientific or societal truths underpinning the data we observe. More and more frequently, the scientific truths we wish to learn correspond to a high dimensional parameter. This project considers covariance matrices and related quantities such as inverse covariance matrices, which are particularly important types of high dimensional parameter, arising in numerous statistical applications. When the dimensionality of the covariance matrix is larger than the number of available data points, structure (sparsity in some domain) must be assumed in order to obtain estimates that are well behaved statistically. This project explores new types of structure for covariance and inverse covariance matrix estimation. Some of these structures facilitate uncertainty statements about the true high dimensional parameter rather than simply providing a point estimate. They also allow different estimates to be aggregated without losing statistical accuracy.