In this paper, we present some models that explain how one estimates time-varying volatilities and correlations of multivariate time series. For demonstration, we consider monthly log-returns of IBM stock, S&P 500 index, and Coca Cola stock from January 1963 to December 2019. The objective of multivariate time series analysis is to study the past observations of a multiple time series to an adequate model that describes how the time series are interrelated. We give a brief description of the univariate ARCH and GARCH models. GARCH models are used in financial time series because they have time-varying variance. An additional reason for using GARCH models is a feature of time series called volatility clustering. Financial time series such as stock returns often exhibit volatility clustering, where time series show periods of high volatility and periods of low volatility. Models such as the Exponentially Weighted Moving Average (EWMA) model and Dynamic conditional correlation (DCC) model are extremely parsimonious because they estimate only one or two parameters. For that reason, they tend to be rejected by diagnostic checks. The simple Baba-Engle-Kraft-Krone (BEKK) model provides a general framework for estimating time-varying volatilities but encounters some difficulties in practice when the number of time series is large. We also give a procedure for multivariate volatility modeling using Cholesky decomposition. Finally, we employ the Go-GARCH model which is suitable for a large number of time series.