Imagine a vehicle that continually collects data about the road surface as it drives along. The data is immediately analysed and sent to the local road authority which gets an ongoing real-time image and map of its asset. The highway authority can point to hundreds of thousands of dollars saved because of targeted maintenance.
This is not to mention possibly dozens of lives saved due to safer road conditions.
Meanwhile, the vehicle collects information about its own performance and wear and tear. The driver gets his or her own black ice warnings, tyre wear alerts, aquaplaning and other situational awareness updates.
At the same time, autonomous vehicles that have virtual sensors are getting the same data to increase their artificial intelligence learning of the highway environment to continually improve the safety of their passengers.
This virtual virtuous circle seemed far-fetched to Eitan Grosbard when he was first approached by Tactile Mobility, an Israeli virtual sensing technology and data provider, in 2018. “My first thought was that it was pure science fiction,” says Grosbard, now the company’s vice president of business development. “I said to myself that even if the company can do only 10 per cent of what they say they can, then it will be an interesting ride.”
As with any start-up company, the road to commercialisation can be a long and costly one after years of trialling and proof-of-concept arrangements. But late last year, Tactile announced its first major deal with an OEM – original equipment manufacturer - Germany’s BMW car maker. “Starting in the summer, all BMW group vehicles, including the Mini, Rolls Royce, the whole shebang, will have our technology embedded in the vehicle’s ECU [electronic control unit] right off the assembly line.”
Tactile Mobility was originally an aftermarket manufacturer of fuel consumption systems for trucks created in 2012 (see box). But with much lower fuel prices by 2016, the company changed its technological direction. “We realised that in-car software was creating data that our software’s algorithms could use to create further data outputs, in essence we created virtual sensors. Tactile Mobility is a software company with a component in the vehicle and the other in the cloud,” explains Grosbard.
Virtual sensors
Tactile Mobility’s software collects raw signals coming out of the existing sensors in the vehicle - wheel speed of all four wheels, wheel angle position in relation to the road, engine rpm, brake pedal position.
“We are not adding anything physical, but using technology that has been in some vehicles for several years now. We collect this date, process it and using physical modelling, machine learning and our own proprietary algorithms, create something new – virtual sensors,” says Grosbard. “Road grip estimation, the way that the vehicle is holding the ground, is one such sensor. Another is aquaplaning, how the vehicle behaves in conditions of water. Grip relates mainly to ice. Others are road surface conditions, such as potholes, can be measured.”
Tactile Mobility’s information from its sensors is fed back into the vehicle and the vehicle manufacturer is paying to use this in real time. But this only half the story, he says.
These same signals from millions of vehicles, starting this year, are sent to the cloud where they are analysed using two further mathematical models. “One is called SurfaceDNA, a digital representation of what the vehicles are feeling on the ground, what we call a tactile layer on the map. So I can provide information about the weather or infrastructure issues. I can distinguish between asphalt and gravel, between new asphalt and deteriorating asphalt. I can detect slipperiness, such as black ice,” he says.
“Other visual data gathering systems that use cameras can see almost everything but when it comes to black ice these cameras can’t distinguish between black ice and black asphalt. Our system can feel it, which distinguishes our technology from other technologies. It can also detect cracks in the pavement before that section of pavement needs to be reconstructed.”
SurfaceDNA creates what Grosbard describes as a digital twin of the road to indicate different grades, banking and curves. “It all goes into the tactile road map layer,” he says.
There is also a VehicleDNA. “This includes tyre tread depth, if it is wearing out over time and to what level. As a driver you will be able to receive a notification from the manufacturer saying that your tyre has worn out and within 400km you must replace it. Certain aspects are already here, while other aspects are being researched now.”
Asset inspection
Most road agencies send out their service vehicles to do a complete visual road surface inspection every few months or even just once a year, meaning months between inspections of a particular section of highway. “For example, you could inspect a section of road before winter and then during winter it deteriorates before you inspect it again.”
But spot checks on sections suspected of having deteriorated is also time and money consuming. “This is where we come into play because we would have a large number of vehicles in a city constantly driving around collecting data, we analyse this 24/7,” says Grosbard.
Since October 2019, Tactile has been running a proof-of-concept programme with the US city of Detroit, in the state of Michigan and home to major US and foreign car makers. Twenty Detroit city service vehicles, such as light duty pickup trucks, have been driving the streets collecting data for just over a year. Data is collected at any speed above 50-60kph, the normal speed for urban driving. Collection is actually possible at speeds much higher, 100kph or more. Grosbard says that Tactile is working to reduce the slowest speed to around 10kph.
“We are providing Detroit with pavement ratings of 1-to-10 to indicate structural integrity of the city’s roads, from city block to city block. So, 10, 9 and 8 are very good, down to 1 to 2 which is very poor.”
Importantly, the Tactile Mobility analysis is benchmarked against PASER, also a 1-10 rating system for road pavement condition developed by the University of Wisconsin-Madison Transportation Information Center. PASER is used by SEMCOG – the South East Michigan Council of Governments, a partnership in south-east Michigan state. But PASER is visual assessment which does not allow for measurements of individual distresses. While roads may be assessed quickly, possibly even by a windshield-type survey, the method has limits. PASER ratings can’t be disaggregated into component distress data, meaning that it cannot pinpoint individual stress or distress points.
“We now have a year-plus of data [for Detroit] and have shown our capability for locating potholes and pavement distress. We are aiming to have a follow-up proof-of-concept agreement for things like assisting them with winter maintenance,” says Grosbard. “For example, our real-time data can show that it is snowing more in one area than another and so the agency can send out a plough to the more heavily affected area first. In this way, we see Detroit as a design partner. They benefit from our data and we benefit from their knowledge.
“We are not looking to replace service vehicles that currently travel the roads collected condition data. It is a case of using our SurfaceDNA data to close up gaps in information to make maintenance more efficient and effective,” he says.
For about 15 months, Tactile has been doing a proof-of-concept in Singapore. “The roads are very good but the main problem is that it is a very rainy region. We map water collection and conduct research into what the structural and drainage issues are.”
Other studies have been set up in Haifa, Israel, and in Yorkshire, England, with highways services provider A1+ [pronounced ‘A one plus’], a joint venture between construction and maintenance engineering firms of Jacobs, Colas and Costain.
Data ownership
Grosbard acknowledges that data ownership is a complex issue. Raw data will belong to the owner of a vehicle. For example, the data collected by the service vehicles that use the sensors in the proof-of-concept trial belongs to the vehicle owner, in Detroit’s case the municipality’s roads department. “Everything that is being created by that vehicle belongs to the owner of that vehicle,” he explains.
“If you bought a BMW and you are driving on the streets of Philadelphia, everything that is being collected in real time belongs to the driver. But BMW, or whichever OEM, will have a consent from the driver that allows the OEM to use that data. The data moves from the vehicle to the OEM cloud and then to Tactile Mobile’s cloud. SurfaceDNA and VehicleDNA analyses are being created within Tactile Mobility’s cloud. So that end data – actually insights about the road and vehicle – belongs to Tactile Mobility.”
In the longer term, groups other than road agencies, such as insurance companies and road-mapping companies, could be interested in Tactile Mobility’s data and analysis.
Meanwhile, Grosbard hopes that its recently announced agreement with BMW will provide that essential big push towards commercialisation. Vehicles coming off the BMW assembly this summer will start generating data. “From then on we can produce data for German cities and then across Europe. We won’t have to rely on proof-of-concept programmes.” But he notes that a key issue is to get other OEM vehicle manufactures to carry Tactile’s virtual sensors.
“We will also have to open more communication with municipalities, insurance companies, with tyre manufactures. This will take some time and remember we are a start-up company of around only 30 people.”
At the end of 2019 two investors were brought on board – German car maker Porsche and Union Motors, an importer of Toyota and Lexus vehicles into Israel. Recently, an American vehicle sector tier one manufacturer and a Canadian venture capitalist firm invested in the company. “In 2021 we hope to introduce more vehicle and tier one manufacturing companies.”
Tactile Mobility
Tactile Mobility was co-founded in 2012 as MobiWize by Boaz Mizrachi, Yossi Shiri and Alex Ackerman. It is based in Haifa, Israel and has offices in San Jose, California, and Kirchheim unter Teck, near Stuttgart in southern Germany. The company’s software enables tactile virtual sensing technology for “actionable insights for autonomous vehicles, municipalities and fleet managers”. The technology collects anonymised real-time data generated from a vehicle’s existing non-visual sensors and turns it into images about road conditions, such as grades, grip and curves, as well as vehicle attributes such as weight, tyre parameters and more.
