Operator Challenge Response

The way we currently collect sensory data is very complex and disconnected. Each sensor or data source is handled separately by an operator at HQ who has to analyze, understand, and then disseminate the right information to the right person in the field. Since oftentimes this information is time sensitive, the current system can put operators in greater danger if the information becomes obsolete. Our project, Modular Overwatch Sensory System (MOSS), focuses on automating the accession and processing of sensory data on a modular platform and eliminating the need for a stand alone operator to analyze sensory data.

Operators face an increased level of danger when all available data is not readily accessible by them. The issue we face is not one of a lack of data or sensory collection methods, but an issue of timely data analysis and delivery. Devise a system that solves the problem of turning data from distributed sensors into comprehensible information, sending it to the right people, and doing so scalably as the number of sensors and the volume of raw data grows exponentially.

MOSS is a hardware and software solution with three primary goals:

• Scalability--ability to switch in and out different sensors, additional sensory modules, and separate data streams, pairing that with an equally scalable software application that can still provide useful information to the operators when some sensors are not available.
• Versatility--independent of time and environment, whether it's raining or dark, the system will continue to provide dependable and vital information from the areas of coverage. The hardware modules can be deployed statically for surveillance or on a moving platform, such as a UAV or RC vehicle through a remote connection.
• Accuracy and Usefulness--the system’s main effort is to provide accurate and useful information to the operators, that is also easily digestible. This can include personnel, affiliation, weapons, location, and direction of movement, and more depending on the types of data being analyzed.

On the hardware front, MOSS seeks to create a standardized sensory module that can be deployed in any type of environment to read and record the raw data from the operators surroundings with a minimum cost of ~$90 per module and a maximum cost of ~$200 per module, depending on intended use. As the need for larger areas of coverage increases, operators can easily deploy more sensory modules to cover larger or more distributed areas, and can even utilize encrypted and secure remote connections. Hardware modules will make use of small, distributed, low-power computers to do some of the data processing and formatting before sending the data to the central unit. Currently, we are looking at prototyping this system on a Raspberry Pi 4B as the distributed computing platform (accounts for $35 of the total cost).

All data MOSS collects will then consolidate into the central processing unit where MOSS’s software implementation can intelligently sort through the data. This central unit takes the form of a computer or mobile device that is in the hands of the operators and connected to all deployed sensory modules--both physical and network connections--to provide them the information most directly. The main component of the software implementation, and what allows for the software to be intelligent and flexible, will be machine learning, which we can use to discern and even identify faces, bodies, and objects (such as weapons) to create patterns of life for possible key suspects. MOSS will be built on open source software, with current plans to implement fully with Python.

At the most basic level, our system would include the following:

• A sensory module with at least a camera connected to it (can be any other single sensor, camera recommended).
• A central computer to run the application and connect the sensory module to (an MSD would be a great example that is already in military supply chains).
• The software loaded onto the sensory module and the central computer.

• Though machine learning has come a long way in the last few years, with many easily usable, accessible, and open source libraries available for use, the biggest challenge with using machine learning will be ensuring that results are accurate--this may require operator interference, or we could work towards building several software and hardware contingencies to reduce the likelihood of bad data.
• Though physical connections between modules and the central unit are mostly simple, encrypted remote connections will be much harder to deploy, especially when taking into consideration possible areas of operation where there may not be networks in the first place.
• The operational environments operators may find themselves in are not conducive to powering large data gathering and processing systems. Possible solutions may include solar power or other contingent power sources.