Jackboy

Modular Smartphone Sensing Platform

Jackboy modular smartphone sensing platform
Conceptual analogy illustrating the Jackboy design paradigm: a modular screwdriver with interchangeable bit heads maps to a smartphone with swappable sensor PCB modules interfaced via the 3.5 mm audio jack.

Collaborators: MIT Media Lab (Camera Culture), AHLab Singapore, Prof. Suranga Nanayakkara, Nitesh Kadyan. Supported by Sharat Satyanarayana, Nikhil Thard, 80Edifice

Jackboy concept poster
Jackboy prototype in operation: a custom sensor PCB module is connected to an Android smartphone via the audio jack, with the companion application rendering a real-time thermal heatmap overlay of the captured scene.

Apart from the access to the built-in set of sensors (GPS, camera, microphone, etc.), Jackboy gives an opportunity to add more sensors to Smartphones as and when required and utilized by research labs, expert users, or the maker community. Our low-cost, plug-and-play, by encoding sensor data into audio jack or microphone readable signals on phones, computers, cameras, or anything supporting 3.5 mm Audio Jacks (later evolved to generic microphones).

Jackboy schematic design
Voltage-to-frequency converter circuit based on the LM331 precision V/F IC, used to encode analog sensor output as frequency-modulated audio signals compatible with the smartphone microphone input.
Jackboy sensor modules detail
Complementary frequency-to-voltage converter circuit enabling bidirectional signal conditioning between the sensor module and the host device's audio codec.

Jackboy will turn a smartphone into a powerful ‘magnifying glass’ enabling the users to probe the invisible real-world characteristics.

via a) Smartphone Spectrometry b) Consumer sensors c) Actuators/Lights (UV/IR)

Jackboy use cases
End-to-end machine learning pipeline for sensor data classification: raw accelerometer, gyroscope, and external sensor data are captured, filtered, and labeled before constructing a decision tree that is embedded on-device for real-time inference.
  • “Connecting Geiger counter to the Jackboy will give a geotagged map of potential nuclear radiation hazard while you drive”
  • “You can walk out with UV-Jackboy and be alerted for potential skin-cancer hazard”
  • “Interfacing Jackboy with a NIR spectrometer will convert your phone into an on-the-go spectrometric fingerprinting device for food items letting the user find out whether the food item is edible or not”
Jackboy system architecture diagram
System architecture diagram titled "Phone as a probe to the real world": built-in smartphone sensors (GPS, camera, accelerometer, proximity, NFC, ambient light) are fused with external Jackboy sensor modules to enable environmental sensing (temperature, humidity, pressure, soil health, air quality, UV, blood alcohol, water safety) and quantified-self applications (ECG, heart-rate, food organic sensing, spirometry), with auditory/haptic feedback and internet connectivity.
Jackboy hardware evolution timeline
Hardware evolution timeline of the Jackboy platform: from early breadboard prototypes connected to an Android phone (April-May 2015), through intermediate circular PCB revisions (July 2015, July-September 2015), to the current miniaturized shipping PCB (Rev 3) shown at coin-scale, representing a 30% size reduction from Rev 1.

We built the Jackboy on an analogy similar to a multipurpose screwdriver, the affordance of a plug-and-play device, with a modular framework to interface with different sensors through 3.5 mm Audio Jack route.

Jackboy sensor modules attached to smartphones
Three smartphones (iPhones and an Android device) with different Jackboy sensor modules physically attached: a cylindrical optics module clipped to the rear camera, a sensor PCB connected via the Lightning port, and a circular sensor board with exposed components wired to the phone's back.
Jackboy sensor and camera chip components
Disassembled sensor and camera modules used in the Jackboy platform: five bare image sensor and camera chip packages of varying sizes laid out on a surface, showing flex cable connectors and exposed silicon dies.
Jackboy PCB module close-up on smartphone
Close-up of the Jackboy sensor PCB mounted on the back of a white smartphone: a circular green PCB with a piezo element and microcontroller, connected to an elongated blue PCB carrying additional sensor components and a metal-can transistor package.
Jackboy companion app output options
Jackboy companion Android application showing the "Suggested Outputs" screen after selecting a temperature and GPS sensor module: output options include 2D Heatmap, 3D Heatmap, CSV Raw Data, and See Realtime, with the sensor PCB visible in the background.
Jackboy Android application interface
Three screens of the Jackboy Android application: the connection screen prompting the user to plug in the Jackboy module, a confirmation screen showing "Sensor Connected" with a checkmark and Start button, and a UV selfie gallery screen displaying recent facial photographs captured under ultraviolet illumination revealing subsurface skin features.
Jackcam - camera module for Jackboy
JackCam variant demonstrating cross-device compatibility: the Jackboy sensor PCB is interfaced with a Canon DSLR via the 3.5 mm remote trigger port, extending the modular sensing paradigm beyond smartphones to professional imaging equipment.
Jackboy modular components
Detailed circuit schematics for the Jackboy signal conditioning stage, showing the LM331-based voltage-to-frequency and frequency-to-voltage converter topologies with component values for multiple operating frequency ranges.
Jackboy smartphone spectrometry
Conceptual illustration of remote optical sensing, evoking the Jackboy spectrometry module's capacity to extend smartphone perception into spectral domains invisible to the unaided eye.
Jackboy freckle detection application
Comparative dermatological imaging under UV-A illumination (left) versus visible light (right), demonstrating the Jackboy UV module's ability to reveal subcutaneous melanin deposits and photodamage patterns imperceptible under standard lighting conditions.
Jackboy usage scenarios
Multi-view demonstration of the Jackboy companion application: sensor module selection interface, real-time numerical readout of environmental parameters, and false-color thermal heatmap visualization composited over the smartphone camera feed.