The Pitfalls of Paper Testing: Threats to Testing Integrity

Envisage analysis published in 2021 highlighted a shift in attitudes toward dishonest test taking. The findings suggest that when cadets come from a “high school or college environment in which cheating is seen as a necessary convenience or tool for continued success,” fire department trainers and other public safety instructors must grapple with a “new normal” for testing: one that includes fewer opportunities to act on temptation, given the laxer attitude that students might harbor about academic dishonesty.

The increased need to combat cheating has forced many agencies to move away from traditional paper-based testing and toward made-to-order technologies designed to minimize dishonesty. While technology has played its own role in aiding those who would circumvent test integrity—smartphones, with their quick access to information and worldwide messaging capabilities, being one obvious culprit here—it can also play a headline role in keeping various forms of academic dishonesty to a minimum.

While no system is totally safe from the effects of integrity problems, sticking with paper means inviting the following problems that would be easy to combat in a modern testing environment.

1. Access to outside information

If you’ve had some experience in the public safety education field, you may judge online education systems based upon their early days, when getting out of the digital testing environment (and into a web browser, where all manner of information could be accessed) was often as easy as hitting ALT-TAB. This, in turn, may make you question the security of modern, PC-based testing tools vs. paper testing procedures.

Today, however, catching tabbing and other forms of dishonesty is even easier than spying someone glancing down at a phone. Systems can automatically log any time users navigate away from a test, resulting (per your policy) in immediate failure, a retake, or other consequences; likewise, many systems allow proctors to look in on their users’ screens in real time, adding another layer of security to higher-stakes testing systems or academy training.

2. More opportunities for information to be leaked

Of course, information getting into the testing environment isn’t the only concern. In the wrong circumstances, information leaving the environment can be catastrophic: In paper-based systems, for instance, copies of tests and teacher editions of textbooks can both introduce serious integrity concerns when they’re leaked from the classroom.

As above, digital systems introduce a few added layers of test security here. First, information such as test keys are centralized, digitized, and secure, meaning industrious students can’t get answers by ordering a teacher book from Amazon. Second, modern tools prevent users from print-screening, snipping, or otherwise capturing screens, preventing test leaks before future takers should see them. Both these can be hugely beneficial in keeping information where it belongs—inside.

3. Poor or difficult randomization

Moving away from security flaws and into inherent weaknesses, digital systems have a massive advantage in how tests and assignments are administered. Consider a paper-based course that uses a mix of five tests to combat dishonesty. Undoubtedly better than one test with one key, this system nevertheless invites all the issues mentioned above: Internal and external leaks can render them ineffective.

Not so with digital training tools. Their ability to randomize the type and order of content that test-takers see means every test is individualized—and thus, much harder to cheat on. Instead of one, five, or 10 versions of a paper test (and all the extra administration that requires), stakeholders need do little more than enter their questions and let the platform take care of the rest.

4. Reduced monitoring and recording

At some level, monitoring a large number of people taking a test will always be difficult. Here, digital systems have two overt advantages:

  1. Real-time access: Proctors can view students through webcams, view live feeds from their computers, and undertake other anti-dishonesty measures in real time.
  1. Recording: The ability to record the test-taking environment gives proctors even greater control, allowing them to double-check their live test for signs of dishonesty.

Somewhat replicable via paper systems, paper nevertheless falters here because it lacks the full depth of visibility provided by more sophisticated testing and education systems.

Improve testing integrity by eliminating paper tests

Modern digital testing tools now allow you to evolve faster than the threats to your testing integrity. Meanwhile, the detriments of paper-based testing systems have been apparent for some time. There are now fewer reasons than ever to rely on the fallible processes of paper tests. Don’t let would-be cheaters bypass testing integrity; establish systems that will be able to adapt to the ever-changing testing environment.

The Acadis® Readiness Suite is the most comprehensive public safety training and compliance software platform for officer performance management, use-of-force reports and cases, decertification, and analytics, and is trusted by more than 2 million public safety professionals and 10,000 agencies. Learn more about the testing capabilities of Acadis.

Posted on Mar 9, 2021