The phenomenon of changing variance and covariance is often encountered in financial time series. As a result, during the last years researchers focused on the time-varying volatility models. These models are able to describe the main characteristics of the financial data such as the volatility clustering. In addition, the development of the Markov Chain Monte Carlo Techniques (MCMC) provides a powerful tool for the estimation of the parameters of the time-varying volatility models, in the context of Bayesian analysis. In this thesis, we adopt the Bayesian inference and we propose easy-to-apply MCMC algorithms for a variety of time-varying volatility models. We use a recent development in the context of the MCMC techniques, the Auxiliary variable sampler. This technique enables us to construct MCMC algorithms, which only consist of Gibbs steps. We propose new MCMC algorithms for many univariate and multivariate models. Furthermore, we apply the proposed MCMC algorithms to real data and compare the above models based on their predictive distribution