Porous materials are ubiquitous and exhibit properties suitable for depositing therapeutic compounds. Drug loading in porous materials can protect the drug, control its release rate, and improve its solubility. However, to achieve such outcomes from porous delivery systems, effective incorporation of the drug in the internal porosity of the carrier must be guaranteed. Mechanistic knowledge of the factors influencing drug loading and release from porous carriers allows rational design of formulations by selecting a suitable carrier for each application. Much of this knowledge exists in research areas other than drug delivery. Thus, a comprehensive overview of this topic from the drug delivery aspect is warranted. This review aims to identify the loading processes and carrier characteristics influencing the drug delivery outcome with porous materials. Additionally, the kinetics of drug release from porous materials are elucidated, and the common approaches to mathematical modeling of these processes are outlined.