Woman pulled over twice after Flock-linked software connected her to homicide(guessingheadlights.com) |
Woman pulled over twice after Flock-linked software connected her to homicide(guessingheadlights.com) |
And? That's all? This is why no one wants the gov't. to run things.
He tells me the only thing that saved him that day for not getting shot was not having the vocabulary to ask questions or explain himself. He was kneed, arms twisted, humiliated, then tossed in the back of a cruiser. Our car was a white infiniti. The stolen car was a black honda.
When the cops get information there is zero investigation. It's "we got him!" And act immediately. It's like having a giant llm generated PR and you just comment "lgtm" and merge, only someone's life is on the line.
The woman probably has a good case for damages from the police department for doing it twice. That's sloppy police work.
Sometimes that means a tradeoff in terms of efficiency, cost, or something else. Often this tradeoff is not one that people in charge of the system are willing to make, unless they are forced to do so by a larger institution or force.
To really drive this home:
1. What steps did Flock Safety take to ensure this warrant was valid and current, and that the plate number was correctly entered?
2. What steps did the police officers take to validate the warrant against the APB hit from the camera?
3. What process is in place to ensure incorrect warrants are not entered or mistakenly kept live for longer than intended?
My guess is that the answer to all three of these is basically "none".
I’ve seen too many cases where people intuitively trust the magic technology instead of verifying it. The tool shouldn’t not present itself as magic or AI or whatever the current term is.
2. The vehicle was correctly identified by Flock as being the one flagged, and verified by the local PD that it was the one that was flagged. That said, they could have verified that the flag was still meaningful before converging on the vehicle.
3. This is a real problem and this is really the core of the oversight.
When the lawsuit over this is filed, it seems like 80% of the liability should attach to #3, and 20% to #2, and essentially none to #1. It's hard for me to see why Flock is even relevant to this discussion except to drum up outrage.
Qualified immunity makes accountability a distant mirage.
The police operate exactly as they are allowed and encouraged, by the system they labor within, to behave.
Your words 'sloppy' and 'error' imply that a 'careful, correct' approach fixes these ills. History and experience shows us that this hypothetical good will not be produced until laws and incentives more broadly, change.
Product, technology, implementation, police department, policing method, workflow. These aren't discreet. Incidents involve all of them.
Responsibility is on people.Technologies cannot be "found guilty." But... this isnt' a trial. There are consequences to doing policing any particular way, and that's the subject.
Doing it this way... using baby skynet for command and control has advantages, like response time. It has disadvantages, like false positives. "Law enforcement error" is technically correct, but not informative about the matter at hand. The error
There's enough blame to go around to hit Flock and the PD both.
Saying that the responsibility is only on the user is extremely myopic. It's also dismissing the role Flock has in how they make and design their software.
If they can't handle this maybe they should be disbanded as a company or nationalized for better public control.
I am not a fan of flock, nor police surveillance state, but what could they have done better here?
"Milwaukee police later admitted the alert should have been cleared out of the system days earlier, but an employee simply forgot to remove it. "
The system seemed to work as intended - that numberplate was flagged for investigation with homicide - and gave the alarm as intented, nothing more. The problem was wrong data given to them by humans.
People talk online as if this is a Flock technology issue, but it only peripherally is. The core problem is that the hotlist systems states set up were never intended for real-time enforcement; there's a great deal of imprecision in them that tends not to matter in human-scale enforcement, but matters a great deal when enforcement is automated.
This distinction is important because there's basically not much Flock can do about it. The source of truth about which cars should be curbed isn't Flock's and can't be Flock's; it's core law enforcement data that belongs to and is the responsibility of the state. Companies like Flock can presumably rig up Rube Goldberg devices to double-check and cross-check hotlist data, but their goals are in tension: they have a window of just minutes to detect and interdict a car. Our police department pledged to do voice confirmations on these hits, and everyone knew that wasn't going to work.
In our specific case, we curbed more innocent vehicles than stolen ones (though not at gunpoint). In each bogus case, Illinois LEADS had left a car recorded "stolen" weeks after it had been "recovered"; "recovered" in scare-quotes because many (most?) of these cars were never stolen to begin with, but rather driven off by a relative without notice. More context on why these data sources are unsuited for real-time enforcement!
Unfortunately, every law enforcement ALPR deployment in the country is a dragnet for all curb-able offenses, from stolen cars to failure-to-appear warrants (the plurality of arrests under our system, which turned our police force into municipal debt collectors for neighboring suburbs). The starting point for mitigating this problem would be to drastically reduce the set of offenses you alert on --- and that's tricky, because your instinct is to reduce them to the most serious offenses, but what you really need is the set of offenses least likely to result in false positives. We settled on "stolen license plate" (no innocent explanation) before giving up on the endeavor entirely.
This also helps provide restitution to victims who would otherwise just be screwed out of their time and legal defense fees
Make software as creepy as you want, intangibly creepy, and there’s not enough for normal people to latch on to.
(If I could only permaban one not sure whether I would choose Facebook or Flock)
see https://revealnews.org/podcast/what-cops-arent-learning/
There are also plenty of videos of drivers scamming others by cutting in front of them and suddenly stopping. All the way up to organized crime. Obviously, you are going to want to spend a couple hundred dollars on cameras and memory cards to prevent yourself from being a victim.
https://www.newyorker.com/magazine/2026/04/20/the-car-crash-...
Another common one is for dog owners to be able to prove someone else’s dog attacked theirs, so there might be a chance of recouping veterinary costs.
Finally, I don’t see why all daycares should not have cameras. As a parent, I want to know that the employees handling my kids are being monitored, at least by management, if not the parents. The kids cannot speak up for themselves, why would we not want to ensure there is proof of abuse or neglect if it happened?
Not sure I agree with the premise...
> Most of the time it does.
Citation needed
> But every so often, a computer system hiccups, and an ordinary person ends up staring down the barrel of a gun for no reason at all.
"Aww shucks, you know, bugs happen and sometimes guns get pointed at people because of them". Pardon my language but: What the actual fuck? How does a site write that with a straight face?
Just absolutely trash reporting and the site doesn't allow for copy/paste to top it off.
EDIT: And after reading further I can confirm this publication and author are not good at their jobs. The ending paragraph is absolutely absurd.
This isn't reporting, this is just saying what the police and the victim said while throwing in line at the end saying maybe we need more oversight.
This is bottom-of-the-barrel reporting and it make my blood boil for them to just shrug this off and make excuses for how the police acted here.
> Flock cameras work by reading plates and comparing them against active alerts
False, flock cameras are car identification and fingerprinting devices, they would ID your car even if you don’t have a license plate at all.
> A tool built to keep communities safer
This should be put as a quote taken from the corporate BS, because obviously it didn’t make any community safer
> As for the rest of us, it is a good reminder that these cameras are everywhere now
Trying to normalize a police state, is this journalism now or straight up paid shills?!
What officer in Milwaukee left a dead notice in the system even after there was one false positive, and what happened to him (I would be willing to bet actual money that the answer is nothing). Who set up the system in Brookfield that allows them to accidentally pull a gun on someone innocent for the exact same reason they pulled a gun on that exact same person a few days earlier.
There is no other profession where you can get away with repeated identical negligence like this. The reason for that is in almost every other profession, negligence is a liability for the parties involved. In policing it is a liability, but not for the negligent person or department.
https://www.youtube.com/live/1OmEmpL-7qY?t=720&is=z_mb9ZEVCs...
Looks like I have two serious opponents to privacy on the commission to deal with, and the commissioner seems to at least want to end the contract. I have no idea where the other two members sit.
Flock cameras... I mean, look: our society, without any central organization at all, has the capacity to record nearly every crime that occurs, just by dint of our camera phones and private surveillance systems. So Flock really just gives the police more information, creating a an asymmetry with the rest of society.
Given that we can already record all that we need to, and call for help from each other at a moment's notice, what good can police do for the next few centuries? And what possible hope is there that it will outweigh the bad?
Includes an example of a Flock camera flagging a number plate that the police pulled over, a family car SUV believing it to be stolen, but the stolen vehicle was actually a motorcycle with the same plate number but a different state!
This training is supposedly for "officer safety." But being a police officer in the US isn't even in the top-10 most dangerous fields, and the vast majority of deaths are from traffic accidents or health issues while on the job (e.g. heart attack/stroke), not gun-related. So it is more likely a cultural issue within the profession.
Then 9/11 happens a few years later and we end up with 1) police being trained in counterinsurgency tactics and antiterrorism response by the FBI, and 2) an ongoing military surplus program where the Army dumps all of its used weapons, armor, and armored vehicles on the police every time they upgrade.
As far as I'm personally concerned, all of these things were mistakes, but I'm not going to sit here and act like they were completely unjustified responses to nothing. My own experiences having been in the Army during the Iraq and Afghanistan years is suck it up. You volunteered for a job knowing you'd get shot at every now and again, but you're not a hostile occupying force. Your job is ultimately to serve the people, which does mean protecting them from some extremely violent criminals out there, but the bar for action should be a lot higher than it is. The overwhelming majority of the time I served, the rules of engagement stated you can't shoot unless fired upon first. It is legal to open carry in the countries we were in, so it was never acceptable to shoot someone simply because they were carrying or because you felt your life was in danger. I never understood why the police couldn't be held to the same standard.
These things ebb and flow in a series of counterveiling overreactions, but the Internet of today does a poor job of remembering just how violent the 80s and 90s actually were.
You don't think that has something to do with the exact policies you are objecting to? They call them "high risk stops" for a reason.
Citation needed.
I speak to folks about the USA sometimes, and guns come up. We are known for that. They think they want them. I get it... they do sound cool.
I try to explain that in the USA, the ratio of guns to people is far above 1:1. Therefore, if I was a police officer approaching any car, I would assume that the driver had a gun. That reality increases the chances of bad things happening due to human factors. "Do you really want every speeding pull-over to become a twitchy wild-west situation? Maybe your kids are in the car!"
Sometimes, this seems to wake people up to the reality of ubiquitous firearms. Please note, I am not making judgements here. These just seem like facts.
Except that majority of households (58%) do not have a firearm:
* https://www.pewresearch.org/short-reads/2024/07/24/key-facts...
and when it comes to individuals, it is the majority (70%) that do not own one. The numbers are 'skewed' by a few individuals that own a lot of them. Statistically you'd be more correct to assume the person in the car does not have one (unless, perhaps, they're a white guy):
* https://www.pewresearch.org/social-trends/2017/06/22/the-dem...
I would be absolutely terrified if people with guns came up to me, whether they're cops or not, and if I had a gun with me, I'd be much more likely to go for it in that scenario compared to a normal human interaction. But IDK, maybe I'm an outlier?
That sort of thing depends on what is "normal". Many states in the US could have around 10% of the population carrying a firearm. Armed interactions with the police are common and the vast majority are a non-issue. The typical police response to "I have a license to carry and a firearm in the vechicle" is typically "Where is it? Just don't touch it". Happens all the time.
Now, is there a statistical difference? Maybe. It's hard to say because you would need to apply the same policing culture, legal system, and population dynamics to a lesser armed country. I don't know of any good choices for that combination.
Why would every speeding pull-over become a wild-west situation? If possessing a gun is a right, then mere possession is no reason to suspect criminality is afoot.
Anyone can become a cop in 6 months. Not everyone is temperamentally setup to be a great cop. That, there, lies the rub.
In general cops err on the side of protecting themselves from danger and I can't fault them for that. How would you approach a car whose driver is suspected of homicide?
Obviously someone made a mistake here but I'm not sure this has anything to do with Flock. The car had an "APB" out. We want cops to pull over "wanted" cars, right?
Police could solve a ton of these issues if they investigated before they tried to apprehend. If we're going to live in a surveillance state, use the damn surveillance for situational awareness. Right now, the cops just get a hit like "APB - Homicide" with no retrospective analysis or attempt to verify. It's essentially an unverified tip - they should use some expertise and tools to try to validate it before they commit.
But they don't, because there's no cost or penalty to them for overreacting, and there's a risk if they underreact. It's really an incentives problem. I don't know if the right spot to address it is with the individual officers or the department, but someone should have some huge liability for this mistake.
If you dont accept this, consider the perfect surveillance state under president N, used by president N+1 given the inevitable flipflop in party control.
This case is a perfect demonstration of the Base Rate Fallacy, and shows what happens when you surveil EVERYONE hoping to find a small percentage of bad people you want to stop.
Almost every car that Flock scans is a normal, law abiding citizen. Even at very high levels of accuracy, any sort of flagging it does is going to most likely be an innocent person simply because of the Base Rate Fallacy.
There are going to be a LOT more of these types of stories than there will be where they actually find a bad guy.
There is enormous variance in performance between police departments. Where I live now the police have been nothing but helpful. In college our neighbor was a cop and was ten times cooler than he had any obligation to be. I've also lived in places where they were nothing but a hindrance.
In fact I was once pulled over something like three times in eight weeks by one police department and given a warning each time for "speeding". The third time it happened I told them that I wasn't speeding, had dash cam evidence, and that I knew something was up given the frequency with which it was happening (probably not a wise idea but I was pretty fed up with it). Officer went to his car and came back five minutes later with some nonsensical excuse about my "VIN number [sic] not coming back properly in the database" and said I'd stop being bothered from them on. In his defense he seemed like he knew they'd made a mistake and was stumbling over his words a bit but the whole thing made absolutely no sense. No retired LEOs I've asked about it had good theories either.
It seems like this woman dealt with a more extreme, AI-powered version of the same. She has my sympathies. This won't abate until the subset of police officers who treat their community as "civilians" stop spreading such dangerous rhetoric.
This meant there were no humans in the loop.
If AI isn’t working, why use it at all?
edit: Every single thing you guys have replied with are policy/law debates, not enforcement debates. I disagree with many laws on the books today. I want the laws changed, not selective/arbitrary enforcement of them.
The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized.
While I merely quoted the text of the Forth Amendment, the Fifth, Sixth, Seventh, and Eighth are also arguably limits on policing power, making fully half the original ten amendments about limiting policing.
I invite anyone and everyone to peruse the contents of https://constitution.congress.gov/constitution/
So the question isn't: "Should we take away the tools these cops use to serve the public?" but "Should we give even more powerful surveillance tools to a group of unaccountable thugs who routinely abuse what they already have?"
Flock does not force cops to be less arbitrary/selective any more than bodycams do.
“Sir, we are coming by today because we are concerned you are insufficiently supportive of our public servants in uniform. It has come to our attention that you have been disparaging some members of our law enforcement divisions to groups as large as 100 people! This is part of our new thought policing initiatives, we need everyone to support our policing efforts!”
“Yes, I work for Flock Online. We only index statements posted at publicly accessible URLs, which in principle anyone can see. We are able to profile every person at scale for a more effective community policing database. This new thought policing initiative is being carried out as a public-private partnership with the full support of your county’s representatives, as a way to preserve our community’s harmony and enhance social cohesion.”
“Please don’t be alarmed, the AI flagged you but I assure you, this is a real human in the loop wanting to talk to you about your comments. I am here because we know you’d be home around this time. Can you please come down to the station with us?”
> "Aww shucks, you know, bugs happen and sometimes guns get pointed at people because of them". Pardon my language but: What the actual fuck? How does a site write that with a straight face?
At the end of the day, the worst bug I could write might delete an object from a bucket. No bug I create will ever cause physical harm or death to the end user or anyone. No bug I create will ever cause a medical device to give erroneous results causing a patient to be misdiagnosed. This is by design. I don't want to ever be responsible for anything like that. Do people working on these kinds of products just dissociate their work from the potential negative effect their work could have?
I always find it interesting that everyone outside the industry focuses on the downside and completely ignores the positives like, you know, actually keeping people alive.
> Citation needed
This is the most damning part of the ALPR rollout for me. If cops were successfully using this technology they'd be beating that drum loud and proud at all hours.
See also anti-terrorism efforts. "Supporters" will argue they keep quiet about their successes for opsec-type reasons or to hide their methods. But we are perfectly happy to publicise blowing fishing boats out of the water so I don't believe that for a second. These people can't help but brag.
No, these cameras are used to erode our rights and let police stalk their ex's/women/etc without consequences.
Also, my sheriff's department isn't exactly known for respecting constitutional boundaries. We had the whole predictive policing national story a few years ago.
Hell, go for a stroll on Google maps in my neighborhood and maybe 1/4 of the neighbors are cops.
Is what it is. Can't be too scared to speak out when it matters.
I’ve always thought millions of camera-equipped cars on the road combined with AI/ML would be an excellent tool for catching criminals.
Edit: I just (had claude) looked up how much was spent on each elected seat across the candidates, and for each of the 5 county commissioner's seats, the election spend between the candidates for that seat was an average of 250k.
The wonderful lady in pink (salmon?), Mrs Starkey, has a bit of sway in the area: https://photos.app.goo.gl/WBPRWMduJ15k9Aj46
Running yourself would be a whole lotta work. Fear that they'll soon be out of office should be ample incentive for the anti-privacy commissioners to suffer sudden changes of heart. Or if not - are there any energetic local reporters, who might dig up some evidence of Flock greasing palms?
If you don't know how to design a system to prevent this, DON'T MAKE IT!
Good grief our industry is absolutely pathetic with these excuses.
https://www.youtube.com/@PoliceActivity
It's pretty unvarnished; there are good shootings and bad shootings, they just post everything released. There are ~2700 videos right now. Pretty big sample.
I think you might reconsider your "If I were going to approach" statement.
Possibly at a time facilitated by the security apparatus that puts the teeth in the surveillance state, because even if they theoretically have the capability autocrats rarely are interested in running their surveillance and security systems themselves, and want to spend time doing other things—which puts a lot of power in the hands to which those tedious bits are delegated.
(Actually yeah. Chicago Police uses an on-prem 3rd party blackbox fax service to avoid those records being FOIA'able. But that's beside the point.)
We'd need to take a look at what they have access to and how they're ingesting that info. I'd start with caring greatly about false positives and not automating department onboarding.
Working with various counties I've seen situations where someone needs to tell you a specific table dump exists to get an accurate count because that's how they published deletions back in 1983. There are no standards and the people who created these systems are all dead.
> The problem was wrong data given to them by humans.
This is always a problem. It's never not a problem. The question is how Flock chooses to address the problem.
The human in the loop did not properly delete an invalid license plate number from one of the databases the flock system checks against.
The issue issues are with:
1. Human in the loop
2. Third party database police use for homicide info
3. Police response
I don't see what the license plate reading technology has to do with any of this. It seems like the only thing working reliably.
Flock should stop existing. I hope people help them with that, enthusiastically.
There's a lot of money in cyberstalking and real stalking. see: Facebook's valuation
There are plenty of ways if you are creative enough.
Somebody could start putting Flock executive license plates in with erroneous police data.
Which means they wouldn't have flagged the car and pulled it over.
A couple of days go by and the error is out of the system.
Saying there is nothing we can do is pathetic behavior and just shows that maybe you (nor Flock) should be in charge of these things if these are the outcomes they find acceptable.
His insistence that "The police are the public and the public are the police" doesn't really hold up in many countries anymore.
* Driving a stolen car
* Driving a car used in a recent homicide
Policing in Mexico is ~100 deaths per 100,000 compared to 11.3 deaths per 100,000 in the US.
100 deaths/100,000 is on par with logging workers in the US, which is the #1 most dangerous field.
Do better research.
It depends on where you live, but some local offices are decided by as little as a few dozen voters.
A guy got shot after being swatted not so long ago and that would be a data point with no injuries for officers, masking the real victims.
[1] https://en.wikipedia.org/wiki/List_of_countries_with_annual_...
[2] https://worldpopulationreview.com/country-rankings/police-ki...
Fun fact; apparently that's just like a local police force's motto (LAPD's originally), and not a national policy for all law enforcement in the US. While reading about this some days ago, I came across this:
> The phrase “To Protect and to Serve” originated as the motto for the Los Angeles Police Department before being adopted by departments throughout the country. The popular phrase has turned out to be quite misleading. Through a series of former decisions, the courts have affirmed that the police have no legal obligation to prevent harm except under specific circumstances [...] The Supreme Court upheld the precedent set in DeShaney and affirmed that State actors, in this case police departments, have no legal duty to protect - https://guulr.com/2021/05/10/a-review-of-to-protect-and-serv...
I think overall the motto is simply a marketing term today.
It should be a quick bit of data work for records to be made of such stops based on duty logs (sans personally identifying information) and compare it to actual arrest/conviction rates to verify that at a minimum the perception of the police match reality. If they don't, then the local community should exert a suitable oversight to adjust police procedure.
To top that off there is almost no "suitable oversight" of any police departments in the US of any kind, let alone by the community.
There really are no simple solutions to this, even places not plagued by bigotry and racism have major structural power problems for the average citizen.
I expect this argument is actually a poison pill for some anarchist or libertarian fantasy of “if the laws fit onto a single sheet of paper all these problems would just go away”.
We just want the enforcement of this policy to be maximally successful! What’s wrong with that :-)
That kid in middle school didn't have a girlfriend in Canada, and the FBI (documented on multiple occasions to have fabricated and entrapped people into "terror plots") didn't quietly stop a ton of terrorist plots.
If you inspect rollercoasters I figure you could probably be pretty lackadaisical on the average inspection. If you don’t take each one deadly seriously though and there’s no way of knowing when it’s important to, maybe it comes flying off the track, so you follow your checklist each and every time without exception.
(So glad I’m not employed to or in a position where I need to own weapons.)
You reminded me of a salient line in BSG:
“There’s a reason you separate the military and the police. One fights the enemy of the state, the other serves and protects the people. When the military becomes both, then the enemies of the state tend to become the people.”
If I'm ever elected God Emperor of the United States I'm going to make beating XCOM 2 a high school graduation requirement just to impart some basic statistical intuition into the population. Everyone needs the experience of having a 99% chance to hit and missing the shot, let alone 70%.
I understand that thinking (see also 'would you play Russian roulette with those odds?'), but the "(>)1:1" comments hides the underlying distribution, which I think colours how people think about the (alleged) pervasiveness of firearms in the US.
As another example of perception versus reality, most people probably don't think of Canada as 'gun country', but we're in the top ten per capita (probably because of the history/culture of hunting up here):
* https://en.wikipedia.org/wiki/Estimated_number_of_civilian_g...
And if you're going to teach people about odds and statistics, perhaps it will help people understand that guns tend to make people's homes less safe:
* https://www.rand.org/pubs/commentary/2013/03/guns-in-homes-a...
Yes, merely owning a gun does make you statistically more likely to die of a gun shot, but only in the same sense that owning alcohol mean you're more likely to drive drunk, owning a car means you're more likely to die in a car crash, and having a house wired for electricity means you're more likely to die of electrocution.
For instance, that article says that the risk of suicide is 5x higher with a gun in the house. Well my risk of suicide is presently zero, and 5 x 0 = 0.
There's a great deal of individual control over the risk when it comes to personal ownership. Compare that to a cop who is professionally obligated to pull over a large sample of people they don't get to select, and this sample is disproportionately violent or otherwise untrustworthy.
"About as risky as low paid delivery work" doesn't seem unreasonably high to me basically. Most cops die in car crashes, not being shot by potential suspects as it stands.
(pretending my parents, wife, children are police officers and posting from that perspective, but do not like taste of boot nor Flock - just thinking logical on the ground procedure accounting for broken larger system)
Because when the other side gets in they're going to use those awesome powers to do "good" things. No one believes in institutions anymore.
I've been shouting this from the mountaintops.
you can mock my beliefs, you can try to convince how misguided they are, but its really outside the minimum rules of discourse to tell me what I actually believe
The problem with rats' nests of infrastructure is that it is effectively impossible to determine what wires are structurally load-bearing in unexpected ways, and there's literally nobody in a position of authority that would be even remotely trusted by a super-majority to construct a suitable replacement system from whole cloth.
I don’t know why the first thing that comes to mind is calling in for backup and having a cop in a mascot costume get out, with body armor and a completely concealed weapon at the ready underneath. They cheerfully chat up the adult driver, say hi to the kid & give a sticker, and once they assess it’s safe they can move onto the next call while the uniformed officer comes to discuss the situation. Doesn’t help the adult much though I guess, we’d know about the concealed weapon. … btw maybe sending a drone or a robot actually came to mind first, but those solutions are uncomfortable - OK w/a million caveats maybe
(Instead of posting something so stupid I could’ve left it at I hope there’s a happy medium!)
it's a pretty high bar to say that all three people the police kill every day were warranted.
most line of duty deaths involve not wearing a seatbelt.
Our constitution itself gives us a blueprint because it states, in no uncertain terms, a "well-regulated militia"; not "each individual citizen".
> the right of the people to keep and bear Arms, shall not be infringed.
what does "the people" mean to you?