(An) empirical study for large log-linear modelling

Abstract

Consider building a log-linear model of 20 categorical variables, each of which is of three categories. Dealing with these variables all together is not feasible due to lack of memory capacity and weeks-long computing time. We have to deal with about 3.5 billion cells of contingency table. This thesis aims to propose a method and apply it to build a log-linear model for those variables. The method consists of firstly splitting the whole variable set into some subsets of manageable sizes, secondly building hierarchical log-linear models for those subsets of variables, and thirdly combining these marginal models into a model of the whole data set. A main theme in combining marginal models is that decomposability of probability distribution is preserved between a model and its submodel and that a particular type of separators in the graph of model is found in a decomposable graphical model and in a collection of its submodels. These separators, which are called minimal connectors in the thesis, are a guideline for model combination. A theory for the guideline is laid out to the effect that we may use the minimal connectors for drawing a blueprint based on which a combined model is formed. This theoretic result is then carried over to a more larger set of hierarchical log-linear models by applying the concept of interaction graph. The theoretic result of the thesis is applied to the data set of 20 variables, and the model-searching process is described in detail until a final model is reached.