For many years, the design of optical systems mainly comprised a linear arrangement of plane or spherical components, such as lenses, mirrors or prisms, and a geometric-optical description by ray tracing lead to an accurate and satisfactory result. Today, many modern optical systems found in a variety of different industrial and scientific applications, deviate from this structure. Polarization, diffraction and coherence, or material interactions, such as volume or surface scattering, need to be included when reasonable performance predictions are required. Furthermore, manufacturing and alignment aspects must be considered in the design and simulation of optical systems to ensure that their impact is not damaging to the overall purpose of the corresponding setup. Another important part is the growing field of digital optics. Signal processing algorithms have become an indispensable part of many systems, whereby an almost unlimited number of current and potential applications exists. Since these algorithms are an essential part of the system, their compatibility and impact on the completed system is an important aspect to con- sider. In principle, this list of relevant topics and examples can be further expanded to an almost unlimited extend. However, the simulation and optimization of the single sub-aspects do often not lead to a satisfactory result. The goal of this thesis is to demonstrate that the performance prediction of modern optical systems benefits significantly from an aggregation of the individual models and technological aspects. Present concepts are further enhanced by the development and analysis of new approaches and algorithms, leading to a more holistic description and simulation of complex setups as a whole. The long-term objective of this work is a comprehensive virtual and rapid prototyping. From an industrial perspective, this would reduce the risk, time and costs associated with the development of an optical system.