Prior studies on longitudinal continuity of adversity have mostly examined persistence of individual adversity, rather than of families and have focussed mainly on material disadvantage. However, adversity is multi-dimensional, and in the case of families with children, it includes psychosocial as well as material elements. While both material and psychosocial elements are recognized as critical to child development, these aspects of family adversity are often studied in isolation and there is a dearth of longitudinal evidence on the extent to which such factors are transient or persistent. Using the first three waves (2004–2005, 2006–2007, 2008–2009) of the longitudinal study of Australian children this paper investigated the consistency and continuity of multiple adversity in families with children using material and psychosocial indicators. First, longitudinal factor analysis determined that a consistent factor structure of multiple adversity adequately fit the data longitudinally. Second, cross-tabular analysis showed significant changes in the prevalence of specific adversities over waves. In particular adversity related to changes in family composition and social support followed an increasing trend. Adversity in two material elements—economic status and hardship—decreased over time. Third, variance-components models revealed that though aggregate scores of material and psychosocial adversity indicated a high degree of continuity over time, continuity in disaggregated measures showed that within families a great deal of variability in adversity occurs, with intra-class correlations ranging from 0.27 to 0.84. The more persistent forms of adversity—family, substance use, economic adversity—are areas where targeted interventions are feasible while employment conditions and time pressure are more transient and may require intervention at the population level to reduce overall prevalence of the adversities. Our study has reinforced the need to assess psychosocial adversity in studies of families with children as a balance to the more commonly utilized material adversity.