Recent advances in the understanding of molecular recognition and protein–ligand interactions have facilitated rapid development of potent and selective ligands for therapeutically relevant targets. Over the past two decades, a variety of useful approaches and emerging techniques have been developed to promote the identification and optimization of leads that have high potential for generating new therapeutic agents. Intriguingly, the innovation of a fragment-based drug design (FBDD) approach has enabled rapid and efficient progress in drug discovery. In this critical review, we focus on the construction of fragment libraries and the advantages and disadvantages of various fragment-based screening (FBS) for constructing such libraries. We also highlight the deconstruction–reconstruction strategy by utilizing privileged fragments of reported ligands.