Asphalt concrete (AC) is a particulate-filled material composed of asphalt binder, aggregates, and air voids, widely used in pavement construction. Due to its high heterogeneity, AC exhibits complex mechanical behavior and further complicates pavement performance under traffic loads and climatic conditions. To understand the underlying mechanisms of AC’s and asphalt pavement’s performance, our recent research mainly focuses on the multiscale characterization and modeling of asphalt pavement, especially the effect of AC’s tension-compression (TC) asymmetry on asphalt pavement response. At the microscale, the contact region (CR), referring to the narrow area between aggregates in contact, was characterized, and its mechanical behavior was predicted by the developed micromechanical model. At the mesoscale, the CT-based AC model was developed to predict AC’s dynamic response. At the macroscale, a dual viscoelastic constitutive model was developed to introduce AC’s TC asymmetry into pavement modeling. The outcomes from the modeling results are expected to help enhance the design and maintenance of durable asphalt pavements.