Naughty Numbers: Ways Bad Data Tricks us into Making Bad Decisions

with Jennifer Nailos, Cynthia Cogswell, Elijah Howe

About This Course

Data informs the decision making of scholars, administrators, and student affairs professionals. In this session we will explore the life-cycle of “bad data.” We will discuss how it comes to be, the ways in which it is used and abused, and the potential consequences of bad data.

Data-driven decision-making is the mode of operation in higher education, and for many colleges and universities. Reporting, compliance, funding, etc. are reliant on the processing and reporting of data. We rank institutions against one another, we report success measures, we analyze trends, and we create compelling cases for change with various points of data and information. Using the numbers to inform our choices and next steps means that power lies in the data and its interpretation. With all the trust placed in our sources of information, have we stopped to ask: what if this is bad data, or, what are the implications of using this information in certain contexts? 

Information integrity is a growing concern in higher education, among numerous other professional fields. As we rely on data for decision-making, good and trustworthy practice continues to be important to our work in assessment. The policy, practice, and ethical implications of appropriate and accurate data use are also acknowledged in fields outside higher education, such as law and medicine. Data misuse can range from bad analysis, context of application, over-generalization, or even simply bad data. Surveying education professionals, Bichsel (2012) found that “between one-quarter and one-third of survey participants reported misuse of data or inaccurate data as a large or major concern” (p. 15). 

Addressing the importance of data integrity and good practice by illuminating common missteps, examples of poor analyses, and pitfalls in practice. Practitioners interested in learning more about applying data and information to inform decision-making will benefit from this session. This live briefing will further participants' understanding of the issues and awareness of strategies to mitigate data problems on campus.

Learning Outcomes

Participants in this live briefing will: 

  • gain an understanding of the stages and basic principles of data collection, analysis, and use; 
  • evaluate categories of bad data and bad data use that influence decision-making; and 
  • identify best practices for data use related to analysis, interpretation, reporting, and application.

You might also be interested in these courses: