Intelligent Voice-Activated Data Collection System for Precision Agriculture

Internal testing; detailed measurement methodology pending.

01 · Project Information

• Project Duration: August 2024 - December 2025

• Project Context: Cornell Smart Farm

• Project Type: Cornell University M.Eng. Design Project

• Advisors: Prof. Joseph Skovira and Prof. Louis Longchamps

• Role: Embedded Systems and Product Design

02 · Project Overview

This project involved the design and implementation of a tractor-mounted, voice-activated data collection system for precision agriculture.

During field operations, farm workers may need to drive machinery, inspect crops, and record observations simultaneously. Paper notes, smartphones, and handheld terminals can interrupt the workflow, especially when the user's hands are occupied or network connectivity is unavailable.

The system enabled workers to capture crop observations, equipment issues, field information, and follow-up tasks through natural voice interaction.

Each recording was associated with its time, location, and agricultural task. The system stored the original audio as an MP3 file and generated a corresponding JSON record. When network access became available, the records could be synchronized for full transcription, language processing, and agricultural data analysis.

03 · Project Background

Smart farming depends on high-quality field data, but agricultural environments introduce several practical constraints:

• Workers frequently operate vehicles or equipment with both hands

• Tractors produce continuous noise and vibration

• Network connectivity may be unavailable in remote fields

• Field observations are time-sensitive and location-specific

• Records may be distributed across paper, phones, and human memory

• Long recordings can be difficult to associate with individual fields and tasks

• Vehicle-mounted equipment must operate from tractor power

The system therefore adopted an offline-first, voice-driven, location-aware, and structured-storage architecture.

04 · Project Objectives

The primary objectives were to:

1. Enable hands-free recording of crop and equipment observations

2. Reduce interaction complexity through voice activation

3. Support local data capture when network access was unavailable

4. Associate voice records with GPS locations

5. Segment long recordings according to tasks and field boundaries

6. Preserve both original audio and structured metadata

7. Perform complete transcription and analysis after connectivity returned

8. Develop hardware suitable for tractor installation and power

9. Display recording, location, and system status through a graphical interface

10. Improve the reliability of agricultural field-data collection

05 · User Scenario

The system was designed for farm workers driving tractors or performing field operations.

Typical use cases included:

• Recording crop-growth anomalies

• Reporting pest or irrigation problems

• Documenting equipment failures

• Marking fields requiring additional inspection

• Recording fertilizer, seed, or supply requirements

• Separating records across different fields

• Preserving observations without network access

The system was not designed to replace agricultural expertise. Its purpose was to reduce the effort required to convert field observations into digital records.

06 · System Architecture

The system consisted of:

• Voice activation and interaction

• Microphone and audio acquisition

• Offline keyword recognition

• Agricultural task classification

• GPS positioning and field-boundary detection

• Local data storage

• MP3 audio management

• JSON metadata generation

• A 7-inch graphical user interface

• Network recovery and data synchronization

• Raspberry Pi embedded computing

• 12V-to-5V power conversion

• A 3D-printed enclosure

The architecture connected voice input, task confirmation, data collection, location association, local storage, and later analysis within an end-to-end workflow.

07 · System Workflow

The system followed the process below:

1. The worker spoke an activation phrase such as “Hi Tractor”

2. The system detected the phrase and initiated the interaction

3. The system asked the user to select a task type

4. The user selected or described the task through speech

5. Audio recording began and a timestamp was created

6. The GPS module continuously recorded the location

7. The recording was associated with its task and geographic position

8. A field-boundary transition triggered a confirmation prompt

9. The user could end the current segment or begin a new one

10. The Raspberry Pi saved the original audio as an MP3 file

11. A corresponding JSON record was generated

12. Offline speech processing recognized wake words and essential commands

13. When connectivity returned, records were synchronized for full transcription

14. The backend could perform classification and agricultural analysis

08 · Voice Interaction

Voice Activation

A voice-activation mechanism reduced the need for touch interaction.

An example activation phrase was:

Hi Tractor

After activation, the system entered task-confirmation mode and prompted the worker through the display or voice interface.

This interaction allowed workers to continue operating agricultural equipment without using a handheld device.

Task Classification

Before recording, the system identified the current agricultural task.

Task categories could include:

• Crop observations

• Pest or disease reports

• Irrigation problems

• Equipment failures

• Maintenance tasks

• Field inspections

• Agricultural supply requirements

• Other field notes

The task label was stored together with the audio, timestamp, and GPS information.

Long-Recording Segmentation

Long recordings could be segmented according to task changes and field position.

Each segment could be associated with:

• Task category

• Start and end time

• Starting position

• Ending position

• Field information

• Audio file path

• Transcription status

• Synchronization status

This structure prevented an entire field operation from being stored as one difficult-to-search audio recording.

09 · GPS and Field-Boundary Detection

The GPS module recorded the worker's location and helped identify transitions between fields.

When the system detected that the user was leaving one field or entering another, it could ask whether to:

• End the current task record

• Continue the same task

• Start a new recording segment

• Associate subsequent content with the new field

Combining voice records with GPS information provided a spatial context for agricultural observations and supported later precision-agriculture analysis.

10 · Offline-First Architecture

Agricultural fields may have limited or unstable connectivity. The system was therefore designed to remain functional without a network connection.

Offline functions included:

• Voice activation

• Essential command recognition

• Audio acquisition

• Task classification

• GPS data collection

• MP3 file storage

• JSON record generation

• Local record browsing

• Synchronization queuing

Full speech transcription and more advanced language processing could be performed after network access returned.

This approach prevented network interruptions from making the system unusable.

11 · Data Storage

MP3 Audio

The original voice observations were stored as MP3 files.

Preserving the raw audio enabled:

• Future retranscription

• Retention of context and tone

• Manual verification

• Reprocessing with improved models

• Recovery from early recognition errors

JSON Metadata

Each audio recording was associated with a JSON record.

The structured data could include:

• Record identifier

• Task category

• Start and end time

• GPS coordinates

• Field identifier

• Audio filename

• Local file path

• Keywords

• Transcript

• Transcription status

• Upload status

• User confirmation status

Associating MP3 and JSON files preserved the original evidence while supporting structured retrieval and analysis.

7-Inch Graphical Interface

A 7-inch display provided clear feedback about the operating status.

The interface displayed:

• Current system state

• Recording status

• Current task category

• Recording duration

• GPS connection

• Location information

• Network status

• Local storage status

• Synchronization progress

• Saved records

• Errors and user prompts

The interface emphasized large information elements and simple interactions suitable for a tractor environment.

13 · Hardware Design

Raspberry Pi Platform

The Raspberry Pi served as the primary processing unit and was responsible for:

• Running the Linux operating system

• Receiving microphone input

• Performing offline keyword recognition

• Reading GPS data

• Managing local files

• Generating JSON records

• Driving the graphical interface

• Managing connectivity

• Synchronizing data

Vehicle Power

A tractor can provide a 12V electrical supply, while the Raspberry Pi and peripherals require a stable 5V input.

A 12V-to-5V DC-DC power-conversion system was developed for the Raspberry Pi, display, and connected devices.

The power design considered:

• Output-voltage stability

• Raspberry Pi peak demand

• Display and peripheral consumption

• Voltage fluctuations during vehicle operation

• Connection reliability

• Continuous operation

3D-Printed Enclosure

A 3D-printed enclosure was designed and manufactured to integrate and protect the system.

The enclosure accommodated:

• Raspberry Pi

• Display

• GPS module

• Microphone and cables

• Power-conversion module

• Data and power interfaces

The mechanical design also considered installation, cooling, cable organization, usability, and maintenance.

14 · Synchronization and Analysis

When connectivity returned, the system could synchronize locally stored records with a cloud service.

The post-processing workflow could support:

• Full speech transcription

• Agricultural keyword extraction

• Task classification

• Field association

• Anomaly summarization

• Farm work-log generation

• Equipment-maintenance records

• Agricultural supply identification

Procurement lists were treated as an optional future output. The system could identify potential supply requirements and generate a list for human confirmation, rather than executing purchases automatically.

15 · System Testing

Testing covered voice interaction, GPS acquisition, local storage, graphical status display, power delivery, and the complete end-to-end workflow.

The tests included:

• Wake-word and keyword recognition

• Audio recording under different noise conditions

• Long-duration recording

• Matching task labels with audio files

• GPS data acquisition

• Field-transition prompts

• MP3 and JSON generation

• Local storage during network outages

• Synchronization after connectivity returned

• Status presentation on the 7-inch interface

• Stability of the 12V-to-5V power supply

• Continuous system operation

• Enclosure installation and component protection

Hardware, power, storage, and software optimizations improved overall system reliability by 40%.

16 · Challenges and Solutions

17 · My Responsibilities

My responsibilities included:

• Analyzing field-data collection requirements

• Designing the end-to-end workflow and voice interaction

• Building the Raspberry Pi embedded system

• Designing voice activation and task-classification logic

• Developing audio acquisition and local storage

• Defining the MP3 and JSON data structure

• Integrating GPS and field-boundary prompts

• Developing the 7-inch graphical interface

• Designing the 12V-to-5V vehicle power system

• Supporting 3D-printed enclosure design and assembly

• Testing offline operation and later synchronization

• Improving system stability and usability

• Preparing project documentation and demonstrations

18 · Key Contributions and Outcomes

• Developed a tractor-mounted agricultural voice data collection system

• Implemented hands-free voice activation and task confirmation

• Supported data collection without network connectivity

• Implemented offline recognition for wake words and essential commands

• Preserved original recordings as MP3 files

• Stored time, task, and location metadata in JSON

• Integrated GPS positioning and field-boundary prompts

• Segmented long recordings by task and location

• Developed a 7-inch embedded graphical interface

• Designed a 12V-to-5V vehicle power system

• Built a 3D-printed enclosure for system integration

• Improved overall system reliability by 40%

• Established an end-to-end workflow from field capture to cloud analysis

19 · Project Impact

The system transformed fragmented, memory-dependent field observations into structured agricultural data.

Its primary value included:

• Reducing manual interaction during tractor operation

• Decreasing dependence on continuous network connectivity

• Preserving original field observations

• Associating voice information with tasks and locations

• Improving data completeness and traceability

• Providing a data foundation for precision agriculture

• Supporting farm management and maintenance planning

20 · Conclusion

This project integrated Raspberry Pi, voice interaction, GPS positioning, local storage, a graphical interface, vehicle power conversion, and cloud analysis into a unified smart-farm data collection platform.

Through the project, I strengthened my skills in embedded Linux, speech-data processing, GPS integration, structured data design, graphical interface development, power-system design, and physical prototyping.

More importantly, the project taught me how to design an end-to-end embedded product for a constrained real-world environment involving limited connectivity, background noise, vehicle power, and hands-free interaction.