Background

In 1990, Texas Instruments (TI) released their first graphing calculator: the TI-81. Due to TI’s involvement with high school textbook manufacturers such as Pearson, high schools were quick to adopt TI’s new graphing calculators. Even high-stake tests such as the Scholastic Aptitude Test (SAT) or American College Test (ACT) started to allow use of these new tools.

There was a problem though. As technology advanced, calculators became smarter. It became possible to store notes, write programs and even solve equations directly from a calculator. This created new opportunities for cheating. To solve this, newer calculators were equipped with a test mode. Once activated, students were no longer able to access their notes or their custom programs. Other companies, such as Casio and HP, also quickly adopted these measures.

Numworks released their first graphing calculator in 2017. This calculator is unique in the sense that it is fully open source. Anyone can download its source code or its PCB schematics. There is documentation on every single piece of the plastic housing and there are blog posts that describe the difficulties experienced during mass production.

As the Numworks graphing calculator is open source, anyone can contribute to its source code to improve it, or branch off Numworks’ work entirely, but this also creates many new opportunities for cheating on tests.

The Numworks graphing calculator

Epsilon

The calculator’s operating system is called Epsilon. It includes numerous apps that allow calculation of expressions, graph functions and Python code. Numworks even provides a simulator that can be trialed straight from your browser. Epsilon comprises of five main components that work together. Ion, for example, directly communicates with the calculator’s hardware and provides an abstraction layer for it, while Poincaré is a mathematical engine that can simplify and evaluate mathematical expressions.

A graph of Epsilon's architecture

Examination mode

On selecting examination mode, a few different things should happen with a calculator. In the Netherlands, calculators in this mode should

  • Display a visual indicator that examination mode is enabled (e.g. blinking LED)
  • Be impossible to disable during the examination
  • Disable the Computer Algebra System (CAS)
  • Clear the memory (stored files and user settings)
  • Block the creation of new files and programs

The Numworks calculator satisfies all of these conditions and is currently allowed for use in the national high-stake exams of 2022, under the condition that the calculator’s software is updated to use the latest Dutch examination mode. There is a slight difference between this mode and the normal examination mode on Numworks (for example, the Dutch examination mode flashes a yellow LED and the normal exam mode flashes a red LED). However, I did not find any resources on how to download or enable the Dutch examination mode. The focus in this article is the normal examination mode.

The source code

To download Epsilon’s source code, Numworks’ repository can be cloned on Github

➜  ~ git clone https://github.com/numworks/epsilon.git

Compiling and flashing the OS to the calculator has been made very easy by Numworks. Ensure that the correct compiler is used (arm-none-eabi-gcc version 9.2.0 as of 8 April 2020, not version 9.3.0). To compile this source code, make can be used, and to finally flash the OS to the calculator epsilon_flash must be used:

➜  epsilon git:(master) sudo make epsilon_flash
DFU     epsilon_flash
INFO    About to flash your device. Please plug your device to your computer
        using an USB cable and press at the same time the 6 key and the RESET
        button on the back of your device.
File: output/release/device/n0110/epsilon.dfu
    b'DfuSe' v1, image size: 877141, targets: 1
    b'Target' 0, alt setting: 0, name: "ST...", size: 876856, elements: 2
      0, address: 0x90000000, size: 867472
      1, address: 0x08000000, size: 9368
    usb: 0483:a291, device: 0x0000, dfu: 0x011a, b'UFD', 16, 0x8ea9437e
Writing memory...
0x90000000  867472 [=========================] 100% 
0x08000000    9368 [=========================] 100% 
Exiting DFU...

Flashing only takes a couple of seconds and works really well. With the ability to compile and flash source code, it is possible to start making modifications.

After flashing the source code, a warning has been added in Settings > About.

Calculator shows a warning that the OS is an unofficial build

To remove it, the following can be found in the source code:

// ./apps/settings/sub_menu/about_controller_non_official.cpp
... 

void AboutController::viewWillAppear() {
  GenericSubController::viewWillAppear();
  // --------------------- Please don't edit these lines ----------------------
  I18n::Message cautionMessages[] = {I18n::Message::AboutWarning1, I18n::Message::AboutWarning2, I18n::Message::AboutWarning3, I18n::Message::AboutWarning4};
  m_view.setMessages(cautionMessages, sizeof(cautionMessages)/sizeof(I18n::Message));
  // --------------------------------------------------------------------------
}

...

After these lines are removed, and the code recompiled and reflashed, the calculator looks original again.

Calculator warning is gone

Focusing on examination mode next, firstly, it is desirable to keep files once entering examination mode.

// ./apps/apps_container.cpp
...

void AppsContainer::reset() {
  // Empty storage (delete functions, variables, python scripts)
  Ion::Storage::sharedStorage()->destroyAllRecords();
  // Empty clipboard
  Clipboard::sharedClipboard()->reset();
  for (int i = 0; i < numberOfApps(); i++) {
    appSnapshotAtIndex(i)->reset();
  }
}

...

This is the code responsible for removing all files. If the lines within this function are removed, the calculator doesn’t remove its files once in examination mode. Here is a video to demonstrate:


New files can be created, but that seems to be normal behaviour for this calculator in examination mode. Maybe this is different in Dutch examination mode? I could not find any information about this.

After all of these modifications, the top LED still flashes, as if the exam mode hasn’t been modified at all. It is not possible for examiners to detect that anything from this calculator has been altered. The possibilities with access to the source code are endless.

Conclusion

The fact that this calculator’s source code is open source made it a lot easier to modify the calculator for cheating on tests. Other manufacturers don’t release their software, and even if you do manage to make a modified operating system, most calculators don’t allow firmware to be flashed without some sort of digital signature from the manufacturer. The fact that the Numworks calculator doesn’t do this is an inherent problem with Numworks’ ideology and the problems this opens up can’t be fixed with a simple software update. Open source calculators lower the barrier of cheating on tests, and allowing the Numworks graphing calculator to be used during 2022’s national exams would be highly irresponsible.