To evaluate the performance of commercial as well as custom-made scanners, dedicated phantoms with defined magnetic nanoparticle (MNP) distributions are required. Prerequisite for the development of such phantoms is the establishment of suitable MNP-matrix combinations. In this study, two different gel types were investigated as potential matrix materials: water-based biopolymers and synthetic polymers. These materials exhibit similar imaging behaviour to body tissue in MRI and MPI. Aqueous suspensions of MNP coated with different types of functionalized dextranes were used for embedding particles into the biopolymers, and organic fluids with oleic acid coated MNP for synthetic polymers, respectively. The obtained MNP-matrix combinations were tested for their shape stability. The homogeneity of MNP distribution and immobilization within the matrix was determined by optical investigation of the samples with a microscope, and the magnetic properties of the composite materials measured by vibrating sample magnetometry. From the tested combinations of MNP and matrix material, oleic acid coated MNP embedded in Permagel was found to be the most suitable for the construction of MPI phantoms. This was based on the reliable and homogeneous fixation of the MNP within the matrix without agglomeration of the particles.