Access control is a critical component of most approaches to providing system security. So many different access control policies and models have been developed to suit a variety of goals. These include Role-based access control, History-based access control, MAC, DAC and so on. Since each of these policies applies to a particular situation, we have to check whether conflict exists when we combine them. So formalization and model which are general enough to encompass a range of these policies and models in using more than one access control model within a given system are important. Then access based on history is so complex in many cases. So administrator must check many cases when he determines the permission. To simplify this, we design the mathematical model called non-access structure. Non-access structure contains to historical access list. If it is given subjects and objects, we look into subject grouping and object relation, and then we design Non-access structure. Then we can determine the permission based on history without conflict.