A Review of TPMI’s 21st – 30th Improvements to the Integrated Visual Augmentation System

Ten Patented Technologies to improve situational awareness, unit coordination, usability, training and military strategy.

ORLANDO, FL, USA, October 23, 2023 /EINPresswire.com/ — The Integrated Visual Augmentation System (IVAS) program Statement of Objectives states that the IVAS will “provide increased lethality, mobility, and situational awareness”. Tipping Point Military Innovation (TPMI) is dedicated to developing a platform of technology to help the US Army achieve these goals.

This purpose of this publication is to review TPMI’s #10 most recent articles discussing patented improvements to the IVAS. A review of TPMI’s first 20 patented improvements to the IVAS was published on August 14, 2023.

Section #1 Improving Situational Awareness: Situational awareness is important in training and critical in combat. Survivability and lethality depend on it.

Challenge 1: Determining the location of gunfire is a challenge. Vehicular noise and radio noise can have significantly higher volume than a gunshot a few hundred meters away. Wearing a helmet also can limit localization of a gunshot. Consider a scenario in urban warfare. A shot is heard coming from a building with eight windows. The Soldier looking at the building, but does not know where the sound is coming from.

Solution 1: TPMI developed critical technology to aid the Soldier in sound localization (US Patent 10,846,911). Using multiple microphones, the location of the origin of the sound is computed. If TPMI’s technology is integrated into a future version of the IVAS, the TPMI upgraded IVAS would display a virtual object at the computed location. By providing the Soldier a visual alert, the Soldier would have the capability to rapid localize a sound. If the virtual object is not within the Soldier’s field of view, TPMI’s technology provides arrows to indicate to the Soldier the direction of the sound’s computed location.

If technology from TPMI’s ‘911 patent is combined with TPMI’s ‘731 and ‘546 patents, the sound could be localized, assigned a timepoint and assigned a GPS coordinate. TPMI’s technology would also enable other Soldiers’ critical knowledge of where the sound is originating from. Moreover, TPMI’s novel GPS-coordinate based image enhancement illuminates imagery at the origin of the sound for further improved detection and characterization. This will enable multiple Soldiers at different locations to ‘see where the sound came from’. Soldiers can quickly take cover and return fire. When multiple shooters are present, Soldiers can coordinate return fire coverage. Overall, this technology will significantly enhance situational awareness by providing rapid localization of a non-visible enemy who has made a sound.

Section #2 Achieving improved unit effectiveness via coordination and mobility: A #5 October Army Times article quoted BG Schneider, PEO Soldier who said “IVAS is not just what you’re wearing,” Schneider said. “A lot of the tools reside on IVAS already. A soldier can fly a soldier-borne sensor right now, take imagery of a target, send that to the tactical cloud package, which turns it into a 3D rendering.” Real time data input into the One World Terrain (OWT) is valuable. Building on this, TPMI’s technology enables an enhanced ‘virtual sand table’, which offers the potential for a wide range of improvements. In this section, improvements to unit coordination will be discussed.

The OWT is comprised of vast and dynamic 3D data. It is critical that personnel of all ranks be on the same sheet of music. Units at all levels need effective coordination.

Challenge 2: A major challenge faced by the user of a Mixed Reality headset is the ability to precisely interact with holograms. The IVAS enables the Soldier to interact with holograms using hand gestures, such as finger tap maneuvers. Specifically, an important limitation of the hand gesture is the ability to precisely pinpoint a specific spot on a hologram. A limited ability to precisely interact with the virtual One World Terrain will hinder unit coordination.

Solution 2: TPMI has developed a revolutionary way to interact with holograms (US Patent 11,207,133), which is through using a set of tangible tools specifically designed to precisely mark up holograms. Using a hand-held tangible tool, a user can precisely mark up virtual objects far more accurately than via finger tap maneuvers. Using TPMI’s technology, a Colonel could use a tangible pointer and mark up a specific route on the OWT hologram, which would enable all personnel to be on the same sheet of music. The Colonel will be able to draw on the OWT with his tool to mark up routes to coordinate activities. The Colonel will grow to love his tool because he can precisely mark up the OWT. This would have a highly positive impact on unit coordination.

Challenge #3: For the OWT to be used in simulations, it must be modifiable on the fly. Consider a scenario where a Colonel wants to modify the OWT by taking out a portion of a bridge. He is wearing the IVAS and does not want to have to use a keyboard or mouse. Verbal voice commands are simply not sufficient to cause the portion of the bridge to be taken out in the way the Colonel envisions.

Solution 3: TPMI’s revolutionary technology platform allows a user the ability to structurally modify the OWT using a tangible tool (US Patent 11,417,071). This would have a highly positive impact on Unit Coordination because it would enable precision modification of portions of the virtual sand table. This would allow the Colonel to use the tangible tool to create the virtual damage to the OWT that he envisions. This would keep people on the same sheet of music as the OWT structurally changes. Thus, the ‘071 patent enables not only a markup of the battlefield, but a transformation of it. The Colonel will grow to use his tangible tool even more.

Challenge #4: When two people are looking at the same complex 3D scene, one person could be noticing and mentioning various aspects of the scene. For example, consider Lieutenant Kathy and Colonel Bob. The experienced Colonel is viewing a first target from a Northeast direction. Then, he rotates the virtual sand table and moves his position to view a second target from a South direction at a closer in distance. The Colonel is talking fast. The Colonel is not great at explaining things. The Lieutenant cannot keep up. This scene illustrates the difficulty for one user to understand where another user is looking in a complex virtual scene.

Solution 4: TPMI’s technological solution to this process is called “multi-user rendering” (US Patent 11,574,435). TPMI performs a novel rendering algorithm, which coordinates both viewing positions and viewing angles for multiple users enabling understanding of complex scenes. In some instances, a deeper understanding of the battlefield is achievable only from a particular viewpoint and viewing angle. TPMI’s technology will have a high impact on Unit Coordination because it enables users to coordinate viewing and improve understanding of complex virtual scenes.

Section 3 Usability & Soldier Acceptance: The US Army is scheduled to receive 100,000 IVAS units. In this section, TPMI will discuss its patented technology, which will enable key “use cases” for the US Army, which are outside of direct combat.

Challenge 5: Every Soldier’s career begins with “boot camp” where a Soldier faces a collection of challenges. One of the tasks in boot camp is shooting a rifle and taking apart a rifle. Challenges of determining where each part goes and how each part is supposed to be held are faced by every recruit. These questions re-enter the mind of Soldiers when they manually operate complex equipment throughout their career, such as a MK-47 Striker 40 mm automatic grenade launcher equipped with a fire-control system.

Solution 5: If TPMI’s technology (US Patent 11,285,674) is integrated into the IVAS system, it would enable a 3D virtual hand to be displayed on the IVAS. The virtual hand and virtual fingers would be registered to the military equipment so that the Soldier sees the virtual hand through the TPMI upgraded IVAS correctly positioned on the tangible equipment. This would help the Soldier learn hands-on tasks, which would yield safer and more effective operations by guiding the Soldier’s hands and fingers during hands-on tasks.

Challenge #6: LTG (ret.) Aundre Piggie, former Army’s Deputy Chief of Staff, explained the critical importance of 3D printing, which enables filling gaps during delays in the supply chain and creating customized parts. A Soldier in the field tasked to operate complex machinery with customized parts faces several key challenges. First, the Soldier must appropriately position Part 1 with respect to Part 2. Second, the Soldier must appropriately align Part 1 with Part 2 with the correct orientation. When facing these challenges, a Soldier could experience both confusion and frustration.

Solution 6: TPMI’s patented technology (US Patent 11,090,837) could significantly help this Soldier because it enables display of a virtual object of Part 2 on a TPMI upgraded IVAS in the exact correct position and orientation with respect to tangible Part 1. By enabling the Soldier an accurate visual understanding of how parts are supposed to connect, customized parts could be used more effectively throughout the Army.
Consider another use case of TPMI’s ‘837 patented technology. An engineer working with an Army engineering agency could be faced with the challenge of designing a new part that has to fit and function with an existing Army piece of equipment without having access to that existing Army piece of equipment. TPMI’s ‘873 patent improves integration of printed objects by geo-registering virtual object that faithfully replicates the actual equipment. This could be iterated as required and the result would be introducing the new part which could form, fit, and function with existing equipment at minimized cost and development time – thus improving Soldier acceptance. Thus, the ‘873 patent is not only helpful to the Soldier in the field using the part, but also for the Army engineer designing it.

Section 4 Increase lethality, survivability and training: Generative AI is predicted to be a super massive industry – $1.3 trillion by 2032. It is important that the U.S. Army leverage advanced technologies.

Challenge #7: It can be hard enough to react to an expected scenario. It can be even harder to react to an unexpected scenario. For example, preparing to fight against an unknown weapons system is extremely challenging.

Solution 7: TPMI has developed technology (US Patent 10,956,635), which enables AI generation of virtual threats into the OWT. This will have a profound impact in wargaming. A system with generative AI threats has the potential to yield a set of completely unexpected scenarios, which will better prepare US forces on all levels. An example would be wherein TPMI’s generative AI has equipped the simulated enemy with a new long range artillery system, which is placed in the OWT. Based on the range fan of this type of artillery, war planners could address key items including units at risk and actions that should be taken. These types of actions can be thought through and implemented with TPMI’s generative AI beforehand. Tactics, techniques, and procedures (TTPs) can be prepared and rehearsed. Performance would be second nature in combat. Such training will improve lethality and survivability.

Improvement 5 Superior Military Strategy: Military strategy can, of course, make the difference between winning and losing a war.

Challenge #8: The amount of sensor data being collected on the battlefield has rapidly increased in recent years. Sources of sensor data are all over the battlefield including the IVAS unit itself. A unit intel officer in a combat situation needs to appropriately utilize this data. This super massive data needs to be analyzed effectively to help the U.S. Army make better decisions.

Solution 8: TPMI developed technology specifically to help US Army improve understanding of the vast data (US Patent 11,728,035). The ‘035 patent provides a novel sub-volume based Artificial Intelligence algorithm. Practically speaking, the user of the ‘035 patented technology can move a 3D volume cursor to a location within the OWT and run an AI algorithm within the sub-volume enclosed by that 3D volume cursor. Selecting sub-volumes and tailoring AI to the selected sub-volume provides a practical solution to improve understanding and analysis of vast battlefield sensor data. The operations/ plans officer could then begin laying the tasks, wargaming sequences of fire and maneuvering. The unit commander would get involved and the overall strategy would emerge.

Challenge #9: Consider a situation wherein the battalion/ brigade S3 Operations Officer is tasked with preparing a training exercise for key members of the unit or for the entire unit itself. The commander wants the exercise to be as realistic as possible and the participants are to use their IVASs. A realistic OWT would improve training and understanding for participants in the S3’s exercise. The OWT incorporates battlefield sensor; however, it should be noted that battlefield data has limitations in its realism (e.g., LIDAR comprises range and reflectivity only).

Solution 9: TPMI developed technology to process battlefield data to generate a more realistic OWT data set by assigning material-type properties to segmented features in the OWT (US Patent 10,878,639). The OWT, improved with the ‘639 patent, would be comprised of virtual structures with material type properties corresponding to the tangible structures. For example, a virtual tree trunk would be assigned material property of wood. A virtual sidewalk would be assigned the material property of concrete. This will improve analysis and targeting for the S3 and add tremendous realism for Soldiers wearing the IVAS.

Challenge 10: If new sensor data is integrated into the OWT in real time, it will be a revolutionary way to analyze a battle in real time. This could significantly help the US military in its battlefield decision making. A challenge associated with this is being able to understand the key data for decision making.
Solution 10: TPMI developed technology (US Patent 10,950,338), which enables the Soldier wearing the IVAS a novel method for analyzing 3D data. Building on the ‘639 patented technology which improves realism of the OWT, TPMI ‘338 technology enables a user to sift through or dissect through the vast 3D data. Consider this scenario. Enemy forces dissipate and the US Army performs an updated airborne LIDAR scan of the region, which is processed with the ‘338 patented technology. The ‘338 patented technology enables structural alterations of the processed data (e.g., peeling back the layers of virtual trees to see what is beneath). By enabling “sifting through” and “dissection of” the data, TPMI’s technology will improve Military Strategy & Planning.
TPMI is committed to helping the US Military in any way it can. TPMI aims to work with PEO Soldier and PEO STRI to enhance both the IVAS system and the OWT.

About the author: Dr. Robert Douglas is one of the few known retired Infantrymen who have 80+ USPTO awarded patents. More to come on Artificial Intelligence (AI), Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR). Dr. Douglas is keeping at it – he is currently under a consulting contract to CESI, the SETA contractor for OneSAF, and using that model, with Army approval, to assess effectiveness of advanced sensor systems.

Dr. Robert Douglas
TPMI
email us here
Visit us on social media:
Other