Alert Vest – connected tinyML safety vest by Swanholm Technology

When former Scania intrapreneur Jonas Svanholm saw the statistics on how prone truck drivers were to accidents and injury at work, he and his team decided to take on the challenge of making such workplaces safer for all professionals, by making the safety vest the drivers are obliged to wear outside the vehicle into something smarter and more helpful in hazardous situations.

Being a professional truck driver tops the most hazardous work environments both in Sweden and across the globe; out of 404 fatal work accidents, truck drivers make up 66 deaths during the period 2011-2020 according to the Swedish Work environment authority.

tinyML and Fall detection

The need of connected safety goes well beyond just truck drivers, and the sheer number of workers in hazardous occupations became the driving force to found Swanholm Technology AB, with the mission to offer connected safety products for exposed professionals. The main purpose is very straight-forward: to make the end-user safer and provide the owners with insight how to build a safer work environment in close co-operation with companies to find new business models and new products to sell.

This objective was not without its challenges, as we needed to introduce a new security layer, without adding additional tasks to perform or things to bring along for the user.

Truck drivers often experience long hours, regularly with potentially stressful situations as well as demanding weather conditions, and it was vital to work out of the platform they already had. The vest needs to comply with the EN 471 class 2 safety standard, also when equipped with smart technology and become the Alert Vest. Needless to say, the questions were many and a lot of practical work needed to be done to reach the product range Swanholm today offer the market, with 100+ units sold and with operations in 5 European countries.

Some practical issues about such a device needed to be addressed, laying the foundation of a new, augmented type of safety product. This raised an array of practical questions such as;

• How is a safety vest used today, what is its actual purpose?
• What sensors are key, where are sensors best placed for optimal reading?
• How are the electronics best embedded in the fabric and sewing?
• How often would a user need to recharge the embedded lithium batteries?
• What is the best way to connect the smart vest to a smartphone, or even tablet?
• How would the device withstand years of dirt, rain, normal wear and tear?
• Can the electronics be easily removed when the vest needs washing?
• Will workers feel watched, even though the vest only alerts its position via the smartphone when the fall detection alarm is triggered?

This product line was developed in close collaboration with the vest manufacturers, embedded engineers, transport and construction companies and most importantly the end-users themselves. In parallel with these interviews and workshops, the software performing the detection of hazardous falls needed to be developed and trained, and this is when Jonas Svanholm contacted the team at Imagimob in order to explain the needs and wants of end-users, and thereby giving the assignment which initiated a string of development iteration cycles, leading to the final product and market introduction in 2020.

Getting back to the theories on sensing a person falling; the fall itself is registered as a split-second event; usually starting off with a small change in the motion vector, followed by milliseconds of apparent weightlessness, and then a sudden, greater rate of change in the resulting acceleration vector, such are the characteristic of a true fall.

Using a safety vest as platform, the nape of the neck proved to be a good placement of the sensor to get consistent readings, while everything else like the batteries and the embedded computer can be placed on either side, or both for weight symmetry. In machine learning there are the traditional series of steps to be taken, each affecting the efficiency and accuracy of the resulting model.

tinyML and datasets

Machine learning projects start with gathering data one way or the other, and in many application areas there are open datasets available for training. In specific applications, whereas a dataset covering the exact use case may be hard to find, hence additional training data has to be captured. This was done using a Bluetooth-connected accelerometer with concurrent video capture. The video capture, even though not used itself in the final application, allows labeling of the data both in real-time as well as in post-production, using the Imagimob mobile capture app for Android.

One of the more known available dataset for fall detection, is the UR fall detection set, developed by a team at University of Rzeszow in Poland [1]. The accelerometer used in their study, however, was placed at the waist and not at the neck as with the Connected Safety Vest’s sensor mount point, rendering such a third-party dataset less accurate for this purpose, but still useful to some degree. The sensor was also of another make, potentially resulting in that the characteristics of a person falling may still be as accurate, while the actual readings can differ from one sensor make to another. It’s naturally preferred to use a similar sensor element for the commercial product, as during the collection of training and testing data respectively. The conclusion was that more training data was needed.

Being a conscientious team, a large number of exercises and tests have been carried out by the Imagimob team, in the parking lot at the Swanholm Technology offices and on various customer sites in several countries.

Falling out from the driver’s seat of a truck, from the back of it when unloading, slipping on the ground, falling into a roadside ditch; the possibilities of falling are near endless and it’s important to cover as many cases as possible. At one point, a professional judo practitioner was hired to perform a large number of pre-determined falls, but just couldn’t trigger the algorithms of the smart vest, as the falls performed were too soft (judo meaning “the soft way” may be part of the explanation). Subsequently, a 70kg crash test dummy was acquired to be a less-skilled fall agent, and suffered through a number or ordeals, only too numerous to mention from a wide range of starting positions and fall heights.

The sensor used for this product is the Bosch BMI-160, which carries both 3-axis accelerometer and 3-axis gyroscope in a small form factor. It handles acceleration up to ±16g and its power consumption is 180 μA for the accelerometer in full operation and 3µA in suspended mode, making it a good candidate for embedded applications when powered by batteries.

After data collection and labeling, the model is generated and trained followed by verification using testing data. It’s a fancy way of saying there is parts of the training data that the AI hasn’t seen before, where it’s predictions is compared to the ground truth, i.e. the labels set during data capture. The last step is deployment into a executable binary library, which can be introduced as part of a new or existing application, see figure 4.

A false positive – fall detected when there isn’t one – is annoying at least, eventually leading to a Peter-and-the-wolf effect where eventually no one reacts to alarms from an ill-constructed system. An actual fall occurring undetected is considerably worse, and only needs one false negative occurrence to invoke distrust as well as dire consequences for the individual.

Naturally, a user doesn’t need to be lone worker in order to benefit from using the Connected Safety Vest. Many other groups of professional workers carry the vest today, construction sites, storage facilities, road workers, dock loaders et cetera.

One of Swanholm’s current customers had an incident prior to outfitting their crews with the smart vests. It was a most unfortunate event, which occurred at a construction site in the middle of a team operation with plenty of people nearby; one of the wheel loader operators gets dizzy and falls down over 2 meters height from his vehicle and after landing unconscious it takes the others a full hour to realize his situation with him lying behind one of the big wheels. So even when working together, it’s good to be able to continuously monitor everyone in case of sudden incidents.

Far from all falls, stumbles or jumps actually results in severe injury. Therefore, the device is equipped with a timer which starts right when a fall is detected, allowing for the user to reset the vest during a grace time, during which a set of warning beeps are heard. It only after a typical 30 seconds has passed, when an actual alarm is issued, sending notifications to peers, management and/or safety staff. If there’s an actual accident, the user doesn’t need to do or perhaps can’t do anything, so just waiting for the timer to time-out is sufficient and the alarm will execute as expected.

The connected vest also is equipped with an explicit pushbutton switch on the front, to raise attention should an operator become stuck, wedged or otherwise pinned in a work situation without being able to call someone using their phone. There might not be a fall at all, and this makes up a secondary way of calling for assistance to compliment the AI detection algorithm.

One other important standard function of the Swanholm product series, is the ability to securely modify the firmware over the air, FOTA, using the smartphone, allowing for new tinyML models and applications to be updated as the fall detection algorithms evolve without having to hand them in for an upgrade.

Learn more about the Alert Vest here.

[1] Bogdan Kwolek, Michal Kepski, Human fall detection on embedded platform using depth maps and wireless accelerometer, Computer Methods and Programs in Biomedicine, Volume 117, Issue 3, December 2014, Pages 489-501, ISSN 0169-2607

[2] Yu X, Jang J and Xiong S (2021) A Large-Scale Open Motion Dataset (KFall) and Benchmark Algorithms for Detecting Pre-impact Fall of the Elderly Using Wearable Inertial Sensors. Front. Aging Neurosci. 13:692865. doi: 10.3389/fnagi.2021.692865