Using blurring as privacy protection when self-driving cars are roaming our streets.
getty
Have you ever taken a picture and realized that a person in the snapshot appears blurry due to your camera being out of focus?
I’m sure you have.
In some cases, the blur happens by accident, whereby you should have set the focus but failed to do so. It was merely an oversight. If you perchance discovered the issue right away, hopefully, you were able to quickly take another photo and delete the remiss one. Problem solved.
There are also situations involving the use of a blur for intentional purposes.
Perhaps you have a few friends that want their pictures taken. Behind them are people partying and making quite a scene. You don’t want the background to overtake the attention to the foreground, namely the gaggle of your best friends that are aching to have a quality shot taken. So, you set the focus to make the background blurry and ensure that the foreground is nice and sharp.
Suppose that you indeed take such a picture and keep it around on your smartphone. A few weeks later, someone asks you if George or Samantha were in attendance at the event. You are pretty sure they were, though your memory is a bit hazy (potentially due to the numerous margaritas that you had).
Wait for a second, they might have been in the background of that photo that you took of your dearest friends. You go ahead and pull-up the picture to check it out. Unfortunately, the blur is overwhelming and there are no easy means to discern who else was captured in the snapshot.
Why all this discussion about blurs in images?
The use of a blurring technique can potentially be useful as a means of trying to attain a level of privacy for those that might be captured on an image or a video. This might entail blurring the face of someone. It could involve blurring their entire body and whatever kinds of motions they made. The blur is obscuring information and making it difficult to ascertain what was in the image or video. That can be a good thing if you are aiming to provide privacy to those caught on tape.
As with most things in life, there is also a downside involved too. In the example about trying to ferret out whether George or Samantha was at the party, the blurred portion of the photo was unable to help in answering the question about their presence. There are likely lots of instances whereby a blurred aspect makes some potentially useful information only marginally handy and possibly entirely unusable.
Your mileage will vary, as they say, depending upon whether you are desirous of having blurs or are downtrodden that blurs are at hand.
Shifting gears (I promise to come back to the blurs in a moment), let’s talk about cars.
The future of cars consists of self-driving cars (see my extensive coverage at this link here). These are cars that have an AI driving system at the wheel of vehicle. The driving actions are undertaken by the AI. No human driver is making use of the driving controls.
An important element of self-driving cars is the use of various sensors to detect the driving scene. These sensory devices are the veritable eyes and ears of the AI driving system. For most self-driving cars, the types of sensors encompass video cameras, radar units, LIDAR units, ultrasonic devices, and the like. The array of sensors are mounted on the vehicle and are used to figure out where the roadway is, where other cars are, where pedestrians are standing, and so on.
This seems relatively innocuous and there isn’t much attention being given to the plethora of sensors that self-driving cars contain. No big deal, it seems, since the sensors are logically required to sense the world that surrounds the vehicle. Without those sensory devices, the AI driving system would be blind to what is happening around the car.
Here’s the rub.
Those sensors can capture a lot more than you might at first imagine that they do.
Envision that you put a video camera on the top of your conventional car. You turned it on and set it to record, doing so continuously. You then decide to drive from your home to the local grocery store.
What happens during that rather routine drive?
Your video camera is capturing all the activity that you perchance come across. For example, after backing down your driveway, you drive down the block to the corner. Turns out that your neighbors next door were in their front yard. They were tossing a baseball back and forth with their children. That seemingly everyday activity is now captured onto your video camera recording.
I believe you get the gist of the matter.
The point being that with that merely one video camera mounted on your conventional car, it will be collecting videos about the daily lives of anyone that you happen to drive past. Let’s up the ante. Suppose that all of your neighbors put video cameras on the rooftops of their cars too. They will also now be recording anything that they encounter while on any driving journey.
Welcome to the emerging world of self-driving cars.
Those seemingly heralded self-driving cars are going to be capturing imagery and other data about whatever they detect, wherever they go, all of the time that they are underway. Keep in mind that there is an assumption that eventually, we will predominantly have zillions of self-driving cars on our roadways, and very few conventional cars.
Returning to your neighborhood and the idea of video cameras mounted on a few neighborhood conventional cars, ratchet this up to assume that all cars that come down your street will have a full suite of state-of-the-art sensors (because they are self-driving cars). Those advanced vehicles will be amassing a lot of information about the comings and goings on your block.
If self-driving cars were empty and simply roaming the community to be available for any ride requests, they would be capturing daily activity.
Whenever someone takes a ride in a self-driving car, it will capture the surroundings that are along the path to the stated destination. By riding in a self-driving car from your home to the local store, the self-driving car will record video and other data about whatever was taking place during that time period.
Please sit down for this next shocker.
I don’t want to get you on the edge of your seat, but imagine that this massive amount of data was collated and assembled to try and piece together the daily efforts in a city or town. In theory, you could pull together the data from all of the self-driving cars and pretty much recreate a semblance of where people were, when they were there, what they did while there (assuming they were outside or otherwise visible), etc.
I’ve referred to this as the roving eye of the coming era of self-driving cars (see my discussion at this link here).
You could say that this roving eye will be a marvelous addition to our society. There are numerous positive uses. For example, you want to see the latest real estate in an area that you are considering buying a home. It is conceivable that the data from self-driving cars could be used to see exactly what the homes look like, nearly up to the minute.
This can also be used for crime-fighting. A burglar tries to break into someone’s house. The crook scoots away before being caught. Turns out that there were self-driving cars that happened to be along that street during the time period of the criminal activity. The sensory data is examined, and the identity of the thief is figured out.
There are some notable downsides too.
Do you want just anyone to know where you were on last Monday or Tuesday? Presumably, an inspection of data from self-driving cars might show that you were in front of your house, mowing the lawn, on Monday morning. You then left your house and walked down the street to visit a friend at another house. You stayed there for about two hours. And so on.
Some people are worried about privacy intrusion from video cameras that are mounted on telephone poles or that are used by people as they carry their smartphones. Those are peanuts in comparison to the magnitude of video and other sensory data capturing those self-driving cars will undertake.
The more we adopt and utilize self-driving cars, the greater the amount of observing of our daily lives that will occur. It’s as simple as that.
Are we doomed to come under the crush of a Big Brother dystopian world by accepting self-driving cars as our preferred mode of transportation?
Sadly, not many are considering this issue and it won’t visibly arise until there are enough self-driving cars that the kind of overlapping and semi-continuous recording rises to a level of noticeability. Until then, we will be laying the seeds for the future that will seemingly catch us by “surprise” about what we have done to ourselves over time.
Shucks, you might be thinking, if this is a looming problem, perhaps something ought to be done, sooner rather than later. There must be some means to keep from digging a hole that appears to be a quite disturbing abyss.
Aha, allow me to bring up an old friend of sorts, namely the blur.
The earlier discussion about the blurring of information was in fact the “answer” before I had presented you with the question at hand.
Similar to how a blurring effect was able to mask whether George or Samantha was at the wild party, the same kind of notion and capacity could be used for dealing with the data that the roving eye detects and collects.
Here is an intriguing question to ponder: Will the advent of AI-based true self-driving cars and their roving eye be potentially made more societally palatable via the use of blurring?
Let’s unpack the matter and see.
Understanding The Levels Of Self-Driving Cars
As a clarification, true self-driving cars are ones that the AI drives the car entirely on its own and there isn’t any human assistance during the driving task.
These driverless vehicles are considered Level 4 and Level 5 (see my explanation at this link here), while a car that requires a human driver to co-share the driving effort is usually considered at Level 2 or Level 3. The cars that co-share the driving task are described as being semi-autonomous, and typically contain a variety of automated add-on’s that are referred to as ADAS (Advanced Driver-Assistance Systems).
There is not yet a true self-driving car at Level 5, which we don’t yet even know if this will be possible to achieve, and nor how long it will take to get there.
Meanwhile, the Level 4 efforts are gradually trying to get some traction by undergoing very narrow and selective public roadway trials, though there is controversy over whether this testing should be allowed per se (we are all life-or-death guinea pigs in an experiment taking place on our highways and byways, some contend, see my coverage at this link here).
Since semi-autonomous cars require a human driver, the adoption of those types of cars won’t be markedly different than driving conventional vehicles, so there’s not much new per se to cover about them on this topic (though, as you’ll see in a moment, the points next made are generally applicable).
For semi-autonomous cars, it is important that the public needs to be forewarned about a disturbing aspect that’s been arising lately, namely that despite those human drivers that keep posting videos of themselves falling asleep at the wheel of a Level 2 or Level 3 car, we all need to avoid being misled into believing that the driver can take away their attention from the driving task while driving a semi-autonomous car.
You are the responsible party for the driving actions of the vehicle, regardless of how much automation might be tossed into a Level 2 or Level 3.
Self-Driving Cars And Roving Eye Blurring
For Level 4 and Level 5 true self-driving vehicles, there won’t be a human driver involved in the driving task.
All occupants will be passengers.
The AI is doing the driving.
One aspect to immediately discuss entails the fact that today’s AI is not sentient.
In other words, the AI is altogether a collective of computer-based programming and algorithms, and most assuredly not able to reason in the same manner that humans can. I mention this aspect because many headlines boldly proclaim or imply that AI has turned the corner and become equal to human intelligence. As if that wasn’t bad enough, the outsized headlines seek to amp further the matter by contending that AI is reaching superhuman capabilities (for why the use of “superhuman” as a moniker is especially misleading and inappropriate, see my discussion at this link here).
Why this emphasis about the AI not being sentient?
Because I want to underscore that when discussing the role of the AI driving system, I am not ascribing human qualities to the AI. Please be aware that there is an ongoing and dangerous tendency these days to anthropomorphize AI. In essence, people are assigning human-like sentience to today’s AI, despite the undeniable and inarguable fact that no such AI exists as yet.
With that clarification, you can envision the AI driving system doesn’t natively somehow “know” that the sensors are capturing a lot of information that might be considered intrusive.
Those are facets that would need to be programmatically devised by the automaker or self-driving tech firm that makes the AI driving system. If they aren’t considering those facets, there won’t be anything somehow innately in the AI driving system that will “realize” that society doesn’t want that kind of pell-mell collecting of our daily activities.
With that important context, let’s dig into how this might work.
Recall that when discussing the act of taking a picture, one point made was that the background could be out of focus and thus blurry, and likewise the foreground could be out of focus and blurry.
There’s not much debate that the foreground of any sensory detection by a self-driving car is going to be crucial for the driving of the vehicle. As such, the foreground is ostensibly going to have to be kept in focus.
The more open-ended question is whether the background needs to be kept in focus too. In other words, suppose that the sensors were calibrated to ensure that anything in the background was blurry. This might help to overcome the otherwise wanton avid capturing of daily activities that are not particularly crucial to the driving of the vehicle.
Of course, you can readily argue that this dividing line between the foreground and the background is altogether untenable.
Suppose that a dog is running around in someone’s front yard. We probably would want the self-driving car to detect that a dog is up ahead, and though currently inside a yard, the dog might decide to dart into the street once the self-driving car comes along (for my analysis about self-driving cars and those chasing dogs, see the link here).
If there is a purposeful blurring when the sensors are trying to detect the driving scene, it could be that vital clues about the surroundings would no longer be readily usable. This in turn could mean that the AI driving systems will not be able to drive as safely as we would hope for.
Some would assert that it makes absolutely no sense to intentionally undercut the capabilities of the sensors. Indeed, those proponents would undoubtedly contend that we need even stronger sensors that have increasingly piercing capabilities, being able to do detection that is far above that of what humans might be able to do.
Under that rather strident thinking, we might wish to momentarily herein set aside the notion of trying to prevent the sensors from capturing whatever they can potentially detect. Assume that the sensors are going to be allowed to detect as much as they can.
The sensory data flows into the on-board processors of the self-driving car. At that juncture, the data is mathematically examined for purposes of driving the car. The AI driving system tries to computationally interpret the data to figure out the driving scene. Upon doing so, the AI driving system figures out the driving action to undertake and emits commands to the vehicle accordingly.
You could suggest that the data from the sensors could now be discarded since it has been used for its primary purpose. In that way of thinking, there is no need to worry about what is contained in the data. Just dump it out, after it has been used for the driving act. Ergo, the data cannot now presumably be used for any nefarious purposes since it isn’t sitting around anymore. The moment that the sensory data has been analyzed for driving purposes, make sure it gets deleted.
That is the end of the road for the sensory data.
Well, that presents a couple of challenges.
First, it means that you can’t potentially use the data for the other augmentable upside possibilities that were mentioned earlier. The data won’t be around and therefore it can’t be used to figure out the latest aspects of real estate or be used to catch those despicable criminals. Some would argue that you are possibly tossing out the baby with the bathwater (an old-time expression).
Secondly, you cannot necessarily guarantee that the data will be deleted. Once you’ve let the data into the onboard systems, this is like letting the horse out of the barn, or the cat out of the bag. Sure, you might believe that the data is going to be deleted, and the system might be programmed accordingly. Nonetheless, the data can potentially be kept, despite the otherwise desired notion of deleting it.
In that viewpoint, you either are going to not capture the data at all, or you are going to capture it and need to do something with it.
This is where an intentional blurring comes to play.
One approach consists of taking the data after it has been examined for driving purposes and then blurring the data so that those elements that are generally considered irrelevant to the driving act can no longer be readily discerned. You don’t necessarily delete those aspects, you blur them.
A difficult question arises about when the right timing is to do the blurring.
If you do so while the data is fresh and just brought into the on-board systems, this means that you need the computational resources on-board to do this type of blurring action while the car is underway. Some would argue that whatever computational processing you’ve got ought to go entirely towards the driving act. Do not usurp those precious processing cycles from the life-or-death matters of driving the vehicle.
Okay, in that perspective, we might have some background process that does the blurring when the self-driving car is parked and not underway. Or basically whenever there is spare processing time available.
Another notion is that you could do the blurring once the data has been uploaded into the cloud. You see, it turns out that self-driving cars are going to be using OTA (Over-The-Air) electronic communications to connect with the cloud of the fleet operator or automaker of the self-driving car. This would be used to readily push down crucial updates to the AI driving system. It can also be used to upload data from the self-driving car and into the cloud.
Thus, some would say that you shouldn’t use any processing on-board the self-driving car for the blurring and instead let it happen in the cloud. The self-driving car would upload whatever data it has collected. This data would be in its rawest form. The cloud processing by the fleet operator or automaker would be programmed to then blur the data.
Sorry to report that this chain of where the data is going and what its status consists of will open a bit of Pandora’s box.
For example, suppose that the raw data in its entirety is being kept on-board the self-driving car. Once it gets loaded up into the cloud, perhaps at that juncture it is deleted from the on-board processors (a copy now exists in the cloud). Unfortunately, this does mean that for some length of time, the data is sitting there in the vehicle, in all its glory. There is nothing blurred as yet. This leaves open the chance that the data could be somehow siphoned or copied and now be made available with everything it has to show.
That’s why some vehemently argue that the data ought to be blurred at the soonest possible opportunity.
Of course, there are other matters intertwined. The data is likely in an unencrypted format upon first flowing into the on-board systems. Some would urge that the data be right away encrypted. In that manner, you don’t necessarily need to worry about the blurring, since anyone that could surreptitiously get the data won’t have anything useful due to the encryption.
This brings up that there are two camps typically at logger’s heads here. One camp says that the full and unblurred data should never be allowed to leave the car. In that sense, it should not be allowed to be uploaded to the cloud. Only once it has been blurred, and possibly encrypted too, can it be uploaded. The other camp says that it is fine to upload the whole shebang, and as long as it is encrypted, you just blur it after getting into the cloud.
Conclusion
There are a lot more monkey wrenches that can be thrown into this thorny matter.
Let’s suppose that the data does get blurred. We are presuming this implies that there is no longer the qualm about being able to detect that your neighbors were playing catch with their kids in their front yard.
Sometimes, that which can be blurred can, later on, be unblurred.
This means that the blurring might be undone. If the images or video is allowed to be copied, you could use all sorts of unblurring techniques to try and turn the blurred aspects into something discernible. It might not be recast into its original pristine state, but at least given sufficient definition that perhaps it once again is intruding on privacy.
The cat and mouse gambit of the blurring algorithms is an ongoing battle. Someone comes up with a newer and better blurring routine. Someone else then comes out with a new-and-improved unblurring approach. Round and round, it goes.
At this time, few are worrying about the roving eye of self-driving cars.
Over time, perhaps my exhortations will only become a blur, though I am really hoping they become unblurred in time for appropriate thought and action to be taken about this mesmerizing and rather clear-cut dilemma.
There really is no blur about it.
