Fraud-proof your exams with the futuristic SmartItem

What is a SmartItem?

A SmartItem uses special technology during development and delivery so that the item changes each time it is administered and completely covers the skill being measured.

Why SmartItems?

Because of the unique way they are developed and delivered, SmartItems solve many major problems facing testing today. Benefits of using SmartItems include:

What makes SmartItems different from other technology-enhanced items, such as items created with Automatic Item Generation (AIG)?

The goal of AIG is to expand an item bank by creating hundreds or thousands of items automatically. The goal of SmartItems is to reduce the size of an item bank to, ideally, one item per objective (e.g., competency, ability, skills). This one "smart" test question covers the skill completely, prompting examinees to study the material more thoroughly and deeply.

For more information on the difference between AIG and SmartItems, click here.

"It is time for us to embrace innovation and technology and bring the multiple choice question into the 21st century where it can evolve to address our current needs and proactively tackle the problems of the future. "
Dave Foster
Ph.D., Caveon Chairman & CEO

Objective: Solve two-step word problems using addition and subtraction. Represent these problems using equations with a letter standing for the unknown quantity. Assess the reasonableness of answers using mental computation and estimation strategies including rounding.

Audience: Grade 3 students

SmartItem Design:

Stem:

name1This variable is selected from a list of six names. Bill, Jenny, Jijay, Kimberly, Cameron, or Ashley
and
name2This variable is selected from a list of six names. Bill, Jenny, Jijay, Kimberly, Cameron, or Ashley
were picking
fruitThis variable is selected from a list of six fruit. apples, oranges, lemons, limes, plums, nectarines
from a tree.
name1This variable is selected from a list of six names. Bill, Jenny, Jijay, Kimberly, Cameron, or Ashley
picked
num1This variable is a random number from 1 to 30.
fruitThis variable is selected from a list of six fruit. apples, oranges, lemons, limes, plums, nectarines
,
name2This variable is selected from a list of six names. Bill, Jenny, Jijay, Kimberly, Cameron, or Ashley
picked
num2This variable is a random number from 1 to 30.
fruitThis variable is selected from a list of six fruit. apples, oranges, lemons, limes, plums, nectarines
. If
num3This number is a random number from 1 to the sum of num1 and num2.
of the
fruitThis variable is selected from a list of six fruit. apples, oranges, lemons, limes, plums, nectarines
were rotten, how many good
fruitThis variable is selected from a list of six fruit. apples, oranges, lemons, limes, plums, nectarines
did they have?

Options:

  • Option 1:
    correctnum1 + num2 - num3
  • Option 2:
    correctnum1 + num2 - num3
    + 1
  • Option 3:
    correctnum1 + num2 - num3
    - 1
  • Option 4:
    correctnum1 + num2 - num3
    + 2

Try this Item:

When reloading this item ask yourself:

Could I steal this question to share it with someone else?

Could I cheat on this item using pre-knowledge?

Could I use testwiseness to get this question correct?

Objective: Know the order of the planets from the sun

Audience: General Audience / Trivia

This introductory example of a SmartItem will walk you slowly through the concept.

Likely, you are familiar with traditional exam development, so let's start there and see how SmartItems differ.

In traditional exam development, the writer creates a select number of items per the learning objective as stipulated by the blueprint. The writer chooses which pieces of the skill to measure ‐ in our case here, which planets to test on. The final product might look something like this; 2 items, as follows:

1. Which planet is second from the sun?

  1. Earth
  2. Saturn
  3. Mercury
  4. Venus

2. Which planet is Third from the sun?

  1. Earth
  2. Saturn
  3. Mercury
  4. Venus

In contrast, a SmartItem is created within a computerized tool to encompass the entire objective, not just a piece of it. Here's what the scaffolding of our SmartItem for this objective will look:

SmartItem Design:

Stem:

Which planet is
First, second, third, fourth, fifth, sixth, seventh, eighth The computer will select from this list, at random, upon delivery.
from the sun?

The computer will select from this list, at random, upon delivery.

One candidate may see "Which planet is the first from the sun?"; another may see "Which planet is the third from the sun?" and so on.

The correct option is dependent upon which list item is presented to the candidate. If "third" appears in the stem, "Earth" is the correct option, for example. The incorrect options will be any planet in the solar system that is not the correct option and are also presented at random.

Try this Item:

When reloading this item ask yourself:

Could I steal this question to share it with someone else?

Could I cheat on this item using pre-knowledge?

Could I use testwiseness to get this question correct?

The objective:

Identify the highlighted bone displayed in an image.

SmartItem Design:

Stem:

Which bone is highlighted below?

ImageThis displays an image with a randomly selected bone

Options:

  • Option 1:
    correctThe name of the highlighted bone
  • Option 2:
    incorrectRandomly selected from the list of unused bones
  • Option 3:
    incorrectRandomly selected from the list of unused bones
  • Option 4:
    incorrectRandomly selected from the list of unused bones

Try this Item:

When reloading this item ask yourself:

Could I steal this question to share it with someone else?

Could I cheat on this item using pre-knowledge?

Could I use testwiseness to get this question correct?

Benefits of using SmartItems

Prevent Theft and Cheating

SmartItems make item harvesting a useless exercise. While parts of the SmartItem can be captured with cameras or in other ways, they provide no useful information to other test takers, who will see different parts of the SmartItem. Most of the cheating seen today depends on someone else successfully stealing the question content beforehand. If harvesting is rendered useless by SmartItems, then cheating by having pre-knowledge of the test content will not be effective either.

Cost Reduction

By neutralizing the benefits of stealing questions, you save all the resources that would have been used to replace items or to deal with legal issues associated with the theft.

A SmartItem typically only has to be created once and is nearly immortal. This makes them financially very attractive, compared to the amount of re-writing of items that occurs today in all testing programs worldwide. They save substantial costs when it comes to re-writing and re-developing tests.

Detect Theft (Watermarking)

If an individual attempts to share the part of the SmartItem they see on an exam, and post it online, a testing program can trace the stolen content back to the administration instance and the thief.

Fairness

Because SmartItems will reduce the impact of test-taking skills, the test will be more fair to all test takers, not just to those who can afford training in test taking skills, or who have honed them over time.

Get Started:

Create SmartItems that can be used in your current delivery system with The Caveon SmartItem API.

smart.caveon.com

SEI (Secure Exam Interface) has support for delivering exams with SmartItems.

sei.caveon.com