This chapter presents an overview of the basic concepts, terminology, and related work regarding the problems that we address in this dissertation. Since our focus is on matrix analytic methods, whose fundamentals are on Markov chains, we present basic definitions, concepts, and solution methods for Markov chains. We identify the infinite Markov chains with repetitive structures that we focus on. We also describe PH-type distributions and Markovian Arrival Processes as the stochastic processes most commonly associated with matrix-analytic methods.
This chapter is organized as follows. In Section 2.1, we introduce basic notation. In Section 2.2, we define stochastic processes in general and focus on the exponential distribution and its variations that we use throughout this dissertation. We give an overview of the definition and classifications of Markov chains in Section 2.4. In Section 2.5, we present examples and define the infinite Markov chains with repetitive structures. We describe the related work on aggregation/decomposition techniques for solution of Markov chains in Section 2.8.