How I learned about corporate firewalls(valcanbuild.tech) |
How I learned about corporate firewalls(valcanbuild.tech) |
Turns out that a substring of that product ID matched the client company's phone number and their security theatre intercepting proxy was replacing all occurrences of "sensitive" strings sent to the internet with asterisks.
The irony is, of course, that as the people running the site, I didn't know (and would never have wanted to know) the user's phone number until this incident.
How I loathe security theatre.
The right thing would be to add a lookup function to first verify the phone number is in use and then call the number to ask for permission to use it; followed by a webhook to send a confirmation back to the database to cache that info because this needs to be efficient!
/s
And now they can exfiltrate all the sensitive phone numbers -- just sending clients (you) long strings of numbers, and see what was replaced.
public Internet facing channel is rightfully scanned and screened for these kind of patterns to prevent unauthorized data loss
This is a end user making a request to an online shop and the POST request to "add product 123456 to the basket" gets changes to "add the product 12***6 to the basket" by a security* proxy between the end user and the web site.
This isn't specific to the site we run. This would have happened on any site they were posting to.
1) They install a root certificate on all machines and use that to MITM all TLS connections using a firewall appliance. They turn this MITM on one day without notifying any developer. Overnight, all our builds (run on-prem) fail because npm install, pip install etc fail and we spent a long time trying to figure it out. They are still failing to this day and I have to get off the VPN every time I need to run these simple commands. IT absolutely doesn't give a flying **** about developers.
2) They ban all non-Chrome browsers from being installed. As in, if you install such a browser and try to launch it, the system will say "browser X is banned. Contact IT." They would have banned Safari too had it not been part of the OS. Furthermore, they also disabled private browsing in Chrome (probably the ability to do this is why they allow Chrome). I think they're preventing people from hiding their internet browsing.
What classifies this as an "overzealous" act of network configuration? There may be a subjectively legitimate reason the user's network was configured this way.
"I had no idea I was ever going to get anything different."
There's an entire list of HTTP status codes. That was your clue that you would get something different. You made a decision to not have handling for them all. Not implementing handling for 418 is understandable, but forbidden and service unavailable responses are common enough.
Always check status codes. Don't assume that backend (even if it is your own server) behaves as you think it should - complain when the response is not what you would have expected.
This is why I hate those error responses that encode the error message into JSON and return status 200. Gee, thanks - your backend is so special that it is an honor to write custom error handling for it. /s
Glad OP solved it in the end, but I would suggest reacting to all 4xx and 5xx statuses. It's a standard, if you get 418 you know what "your" backend is saying.
Good thing I never put any of my money in the company accounts...
It makes simple things really hard - like a links checker, package dependencies, remote servers or integrations with Google.
We can't even run test scenarios on the machine because we're also locked _out_ of the server. Instead, we rely on their IT department to run test scripts that we send them via email.
We were debugging an elastic server connection for 2 weeks that was working perfectly fine in their "QA". It's a horrible existence.
Even then, if you were using certificate pinning, it wouldn't work as the HTTP proxy would serve a "are you sure you want to continue" HTML page, which is of course not expected.
SSH is out of the question.
it's amazing what "simple" things break; like kubectl, gcloud, go get.
So frustrating. Countless development hours lost to bypasses.
If you're running Linux, there's a utility called "tsocks" which wraps any other command and redirects all network servers through a SOCKS proxy defined in /etc/tsocks.conf, e.g.:
tsocks pip install somepackage
Reminds me of a friend who started a government job, and they went 6m before they were fully onboarded and able to work. ????
I wish more front-end devs recognized that they're building HTTP clients whenever they make HTTP requests. There's a whole specification written about how to do that well so one doesn't have to learn things like this the hard way. Specs may look old and esoteric, but following them bakes hard-earned wisdom into your apps for free.
I prefer to let the users actually see such errors, although that seems to be an anti-pattern today.
Usually any message receiver should first check the status code and only proceed if it is 2xx and handle errors in any other case.
But such edge case errors (a 403 usually isn't an edge case) getting swallowed still happens on the most prominent on thoroughly tested sites. Had similar stuff on Amazon and Microsoft pages for example. Saw the errors in the console but they weren't displayed to the user.
Will it fail silently? If so, is your code prepared for null to be returned instead of parsed JSON objects. You'll have to check for that.
Using a library isn't going to save you from handling all the error states and unexpected response bodies. It'll just change the documentation you're reading and the name of the abstractions you're dealing with (e.g. status codes -> exceptions)
I think you did a great job cementing the "why"—usually this topic is very hypothetical. I also liked how you tied it to real end users. After all, that's who the internet is for!! [1]
My intention wasn't to criticize your post. I hoped my comment would help one or two readers recognize the underlying problem space a little sooner, which might help them learn a more broadly applicable lesson when the time comes.
And they wonder why I prefer to work from home?
Cannot judge by not knowing how he displays errors. But a question to HN public: Is opening unknown HTML under my domain within another window safe? Or is there any possibility to strip down any "permissions" to cookies, requests, resources etc for that dedicated page?
I always hear people cry and moan about this but having worked on that side of the fence I would like you to know that I know of instances where people have been downloading illegal material (involving children) and running tor. That's not to mention the 75% of staff who willing give details during phishing campaigns.
Saying that, I find 60%+ of cyber businesses to be a waste of time at best, and at worse just frauds. Core firewalls with L7 capabilities from vendors such as Palo Alto and CheckPoint are legitimate security devices, especially suited for enterprise networks.
I do think it's pretty pointless running those in the cloud though, unless you have admin VMs on vnets for your production resources. But that way lies madness anyway.
This is hopefully a trend that is disappearing with a wave of modern transparent proxy solutions, but in general companies tend to set up proxies that get automatically authorized by your workstation. It may have some issues with less known browsers and your console tools will not be able to use that at all.
So when you build something locally, want to download a .deb, or a pypi package to have modern Python tools your are out of luck - you have to download it manually using a browser or not at all.
This is where such proxy comes into play.
Although I still think that breaking it up is a very bad idea in general and it is appalling that this became common practice. Especially because there are exceptions where it fails and you train users to just disregard TLS errors.
Even worse, the IT security industry shamelessly uses the data to spy on employees. For that alone it deserves its bad reputation. Still, there is no real solution to shield data from the the most careless users.
Ideally a subnet belonging to one of your competitors? I thought that nowadays only very ignorant people follow links or open attachments in spam emails. Certainly all the spam I've seen for a few years has been as plain as the nose on your face: only an ignorant person would mistake it for ham.
> Website blocked
> Not allowed to browse Shareware Download category
> You tried to visit:https://www.valcanbuild.tech/handling-corporate-firewalls/
The irony.
There are lots of reasons why the request would fail and returning a 403 or 503 from a corporate firewall is just one of them. What happens if the user's wifi is flaky and the HTTP request is canceled? What happens if the connection is slow and the request times out? What if, heaven forbid, the destination server is down or unreachable temporarily?
As a web developer, never let a user's action lead to nothing happening. Always give feedback. Whenever sending background HTTP requests, always provide a visible error message to the user when you encounter unexpected results or HTTP/network errors.
Firewall as bossware.
Firefox being banned is because it uses its own certificate store, so Firefox users would see a browser warning every time they visit any https site notifying them that their traffic is being MiM'd. Chrome and chrome reskins like MS Edge use the OS store which MS Windows centric organizations can easily (centrally using MS tools [GP]) add the trusted CA for MiM into. For the Macs, it probably wouldn't matter since the 3rd party mgmt tools could probably push out either.
FYI You can instruct FF to use system trust store: https://support.mozilla.org/en-US/kb/setting-certificate-aut...
They also used push notifications on the desktops to know when people were active or what they were doing, and had keyloggers installed/active. Once caught a manager's personal laptop on the network running mitm software. A friendly coworker in IT confirmed all of this with me in private.
Tried warning a couple coworkers, but got brushed off. People don't seem to care nor believe even though they're being manipulated.
That place was a nightmare to say the least
Our IT now blocks outbound SSH entirely. You know, the secure way to access VM's in, say, our cloud? Sigh. I'm sure there's a "jump" server somewhere that I'd have to log into, `sudo` to another account, THEN SSH to my target box. Whatever. I just avoid the VPN.
I used to use `cntlm` to tunnel requests through our firewall for things like Ruby's bundler, as it required NTLM authentication. Now they've also gone the additional mile, and installed a certificate (Cisco Umbrella) in all of our computers, and require its signature to pass the firewall. Unfortunately, it took me a long time to sort this out: why `cntlm` no longer worked, and why none of the usual suggestions on SO fixed it. I finally figured out that RubyInstaller for Windows included a nice facility to deal with this. You just place additional certs in a directory, run a Ruby script, and it will bundle the whole stack into a single .pem, which it will reference for all network-related commands. Thankfully, bundler's error messages were telling me the specific certs I needed, and I could download them from Cisco's web site.
Just about a month ago, my company started requiring that cert for ALL traffic, not just HTTP(S). Like for, say, Postgres connections on port 5432. I finally realized that I could reference that same SSL bundle in my Postgres client connections, and get through.
I've spent about 8 years here now, and it's been a cat-and-mouse game the whole time. I'm always wondering what's coming next.
They are not paid to. Their performance is judged against how close they get to zero compliance issues, not how close they get to zero times developers were unhappy!
> I think they're preventing people from hiding their internet browsing.
Without delving into the “do you have the right to privacy even on a company machine”, who would be daft enough to do something they want to hide from the company on a company machine, or the company network at all? Though there are valid useful uses of pron^H^H^H^Hicognito mode so it seems silly to ban it.
Maybe “you shouldn't be doing that anyway” is a key part of why they don't care to spend effort resolving the problem.
I've worked in various places from big finance, public sector, to privately owned. It was only the big finance institution (the biggest) that actually seemed to care about supply chain attacks. Everything was locked down super well and you could not! use a random library without it getting vetted by a central team. In fact, we were even locked down to specific versions of programming languages.
People see this as annoying and in the way of developers, but it really is the only way to secure your development "supply chain". When people cry about this I always ask: do you really want the entire financial industry grinding to a halt because someone took down left-pad?
I’m in charge of the IT goons somewhere. We aspire to provide a better level of service and maintain local repos of things you’re allowed to use. Stuff like Node isn’t allowed near anything important though.
I would be careful. An agency doing stuff like that is probably running an EDR that will detect and report on what you’re doing. If it catches what you’re up to, you’ll be jammed up.
If you don't have a budget for the vetted repositories, it means you don't have a budget for the project within the security requirements. You shouldn't be circumventing the security requirements, you should escalate the issue.
PS: of course I'm not talking about other things like MITM certs, that only reduces security.
There are portable apps for other browsers, Firefox for example. FF has its own certificate store that overwhelms IT.
about:config -> security.enterprise_roots.enabled and it uses the system store.
Overall Firefox is a very good browser to configure for different machines. The out of the box Chrome or Edge are just a bit more forgiving to MITM attacks. So the user doesn't notice perhaps? Aside from that they are horrible browsers with horrible priorities.
“It’s more secure” they said.
The “solution”? Disable certificate checking of course! What could go wrong?
Same place that ran a vulnerable instance of nexus (the package manager) for all internal npm and maven packages for a whole year before patching. It was publicly accessible. And it had a banner on the homepage that said “this version is vulnerable (severity 10/10 anonymous RCE), update NOW”. Anyone who went to https://nexus.1337company.com would see it.
That company did software for the government. I’m sure I wasn’t the only one who noticed the vulnerability and that some packages got tainted. But we’ll never know because no audit was ever performed and there were no backups of that server anyway.
Like I said, absolute joke of a workplace.
Or npm and pip use their own certificate stacks and refuse the firewall's cert, which is ... good I guess.
Corporate middleboxes come in all shades of stupid.
Combined with the fact that chrome is the only allowed browser, I suspect it is the other way around. Chrome uses its own certificate stack, and I would guess IT only added the MITM certificate to the chrome trusted CA list, not the system one.
you should use this command "pip install -i http://artifactory.mycompany.local pandas" and get url for artifactory from admins
> Overnight, all our builds (run on-prem) fail because npm install, pip install etc fail and we spent a long time trying to figure it out. They are still failing to this day and I have to get off the VPN every time I need to run these simple commands. IT absolutely doesn't give a flying ** about developers.
Add their cert to system store ? Won't help inside containers tho... without much fuckery
Security team should have provided you with golden image with hardened config, latest patches installed, and corporate certs installed in certificate store.
If they didnt, they aint doing correct DevSecOps/SecDevOps or whatever the fancy term is for integrating security within development team.
It is a big red flag that any developer can pull whatever image for container running in production, possibly with unpatched vulnerabilities and loose config and ports open, and running with root privileges, etc.
Usually stuff has to be vetted and checked prior to being deployed in production environment
That's the fun part: every technology and tool has its own bespoke way of handling certificates, and it often isn't as simple as adding a certificate to the system store.
Worked at a large FI.
Our corporate firewall used to block any website or payload that contained the word "hack". At one point, the security team decided to roll out a change that blocked all verbs except GET and POST without telling anyone. I could go on.
What you tend to see is the web firewall is administered by someone who has only one duty (manage this firewall) and very narrow set of skills (certification in this appliance). They probably have a very shallow understanding of the http protocol.
How else are you going to stop employees from downloading and playing NetHack at work?
I understand that some companies want to block certain websites. However, if you're in such a restricted network, I wouldn't expect a website like "Thankbox" to work at all.
An overzealous filter like this prevents normal POST requests (logging in to websites, etc.), lets through random websites (gift card website) and allows all manner of data exfiltration and other nasty stuff. The goal is laudable, the implementation is laughable.
No, just no.
In a world where many website use GraphQL (POST request with content) (or gRPC) that's complete garbage decision.
- This kind of brain-dead admin decision is exactly what bring protocol abuse: people would just use GET query with a ton of parameters and violate semantic just to avoid stupid middle box problems. Same goes with TLS which is used everywhere (even in VPN) just to bypass the crappiness of corporate firewall and stupid managerial decisions.
Dash (or its Windows equivalent, name escapes me) can be used to view and search these dumps (as well as dumps from GitHub, language docs, etc) offline: https://kapeli.com/dash
The problem with security people is that they think security is the most important thing.
A good security admin will work within the bounds of compliance to make the business work. And any good blocks will be apparent to the user. Trust me, security doesn't enjoy pissing people off, we just accept that it happens sometimes.
Decided the idea was not worth it to fight with infosec guys.
Turns out that a one-checkbox-tick fix was all it took to make that go away. The woman in charge of the web proxies panicked, thinking that this change had "broken something", reverted the change, and then refused to change it back.
Fun times.
The A stands for availability and if you don't make things available, you're failing at your own job.
Also, the link to "request an exception" lead to a 404 and the IT team responsible for the blocking didn't respond to email.
Oddly enough, I can get on imgur
For example I wonder if javadocs shows you how to convert an InputStream to a string, as per this question: https://stackoverflow.com/questions/309424/how-do-i-read-con...
CC: Management
With such a strategy for learning everyone would end up in IT security.
Regards,
hyperman1
Yeah, I can't believe how stupidly locked down some of these networks are.
I once had an employer said they needed a "whitelist" of websites we wanted to visit instead of a "blacklist" of ones we shouldn't. That was an interesting day...
We run a Saas and someone wrote an email saying that our server was down, and when we'd expect it to be up. Not having had a notification, I double checked from a couple of geographic locations that our application was indeed up and responding.
After a bit of investigation, it turns out that they have to whitelist every unique address with their corporate IT. And had only whitelisted our primary client-app URL (talks to a couple of different API endpoints), hence the strange error message.
It's been a long time since I've worked somewhere with whitelisting.
I currently no longer need to do so right this minute, but sometimes people do keep asking me why I still have that.
---
Not sure if this still works on modern corporate networks. These days I tether to a mobile phone with unlimited internet; which is all-around easier to work with.
If you don't have other relevant allow rules, your sshd traffic would just be dropped, regardless of port.
If the firewall administrator does things poorly, they will create an allow rule for port 443 and your sshd traffic on port 443 would be allowed (no inspection of traffic to determine if it is SSL or SSH).
BTW this is inspection, not decryption. Two very different things.
The business of developing algorithms to effectively detect various applications must be very interesting. You can see all the different "applications" here: https://applipedia.paloaltonetworks.com/
1. Disconnect from VPN and run `npm install` until it failed
2. Connect to VPN "Profile 1" and run the command again until it fails
3. Connect to VPN "Profile 2" and run the command again until it fails.
4. Disconnect from VPN and run the command another time to finish installing all dependencies.
5. Reconnect to VPN to actually run the app.
-> ᛯ ssh -T -p 443 git@ssh.github.com
Warning: Permanently added '[ssh.github.com]:443' (ED25519) to the list of known hosts.
Hi XANi! You've successfully authenticated, but GitHub does not provide shell access.If he put it in a sandboxed iframe, it will have the same kinds of access as the main page, because it comes from the same domain. Everything is already as messed up as it can be, and there isn't anything the frontend can do to improve it.
What's the safest way to handle this? Open it in an iframe?
What's the safest way to handle this? Open it in an iframe?
But still, I'd go with safer practices. Even in the slightly unlikely case someone manages to hack 3rd party (Stripe) and send your users arbitrary HTML for some periods of time... :)
Take for example the scenario in question here. Is it really legitimate to allow GET requests to a domain but block all POST requests? That sounds questionable at best. How many sites is it safe to view pages, download files, etc from, but POSTing to them is dangerous? There may be a few, but it is not particularly common. Far more common is sites where any request could be harmful. (Malware, sites spoofing other sites, etc).
I get fully blocking a domain. That can be reasonable sensible, especially for domains in a known blocklist of porn, malware, etc.
I can get inspecting content and blocking if there is clear evidence of maliciousness (but this must be done carefully, since false positives can cause a lot of headache!), but for other content-matching scenarios, you may well be better off generating an alert to be reviewed manually, rather than blocking things.
There have been cases where these system incorrectly block business critical functionality, causing a company to completely shut down, losing huge sums of money while figuring out what is breaking things, before getting it sorted.
Yes, blocking phishing mails can be impossible with some hosted providers' spam filtering. But, here the solution should be to push back on e.g., Microsoft to fix their dumpster fire spam filtering, or switch the organization to a different product that works.
I don't think IT should be pretending at being police. It isn't their job. And, any infrastructure that can be used to catch "criminals" can be used to abuse employees.
Also, there is absolutely nothing wrong with using tor. I've used it often, at work, to test things as if from off-site.
I believe the role of IT is to respectfully facilitate users to safely get their work done. This involves a balance of security measures that do not invade the users' privacy, pushing back against management when appropriate to protect the users from managerial overreach, and sometimes just allowing something that could be dangerous because the alternative is worse e.g., MiM provides limited protection from exfiltration, but also enables horrible abuse by management and should be pushed back against.
If you don’t have a starting point like that from prior experience or stackoverflow, you’re stuck clicking around the package lists, hoping to land on something useful
But you are correct as well; that is an uncommon yet hugely beneficial reason to have an internal artifact repository.
It seems that running the proxy leaves the original problem uncorrected. I'd be inclined to exercise a bit of malicious compliance to increase pressure for changes to the security configuration.
This being security theatre, it is entirely plausible that the "security" proxy actually decrypted trafic and required the user to have the certificate installed.
https://news.ycombinator.com/item?id=33095888
> I work at a government agency and here are my tales.
> 1) They install a root certificate on all machines and use that to MITM all TLS connections using a firewall appliance. They turn this MITM on one day without notifying any developer. Overnight, all our builds (run on-prem) fail because npm install, pip install etc fail and we spent a long time trying to figure it out. They are still failing to this day and I have to get off the VPN every time I need to run these simple commands. IT absolutely doesn't give a flying * about developers.
A tale as old as time: https://www.youtube.com/watch?v=uRGljemfwUE
I still maintain one on an internal QA network.
It allows whitelists.
https://avolio.com/fwtk-history/
https://www.ranum.com/security/computer_security/editorials/...
/s
Confidentiality,
Integrity,
Availability.
There are other IT protection goals, including authenticity, privacy, reliability, and (non)repudiation.
These fall under integrity, confidentiality, availability, and integrity respectively! The CIA triad is pretty comprehensive!
Authenticity and non-repudiation falls under integrity.
Privacy falls under confidentiality.
Reliability in context is another word for availability.
Unfortunately this is rarely true in practice. There is always some degree of friction or error that ought to be managed; ignoring it is how shadow IT proliferates, e.g. a dev is tired of their builds failing due to a false positive and decides to circumvent artifactory altogether.
You're spot-on otherwise.
It seems that Microsoft has built an internal tool that runs such a scan on NuGet (https://devblogs.microsoft.com/nuget/how-to-scan-nuget-packa...), at least against your individual app's packages. (That would be a very rare h/t to Microsoft from me.)
EDIT: Apparently, you can also do this with npm packages (https://docs.npmjs.com/auditing-package-dependencies-for-sec...). I don't see any facility to do this with Ruby gems.
It looks like the common practice would be to outsource the issue database to GitHub, and let whatever scanner you're using cross-reference that list?
What happens when it finds a reported problem? Does it automatically delete that mirrored package, and/or block it from being downloaded or used from the on-prem repo?
This is all new to me, and has helped put this in context, but what actual software are you talking about using for analysis?
EDIT EDIT: Running `yarn audit` in my main Rails app (just using webpacker to bundle the JS):
97 vulnerabilities found - Packages audited: 1074
Severity: 2 Low | 34 Moderate | 52 High | 9 Critical
I could talk at length about this; unfortunately, I'm on my phone with a shotty connection.
The tl;dr is that companies like Snyk make money by requiring companies to pay to check for vulnerabilities once they've been downloaded. There's not necessarily anything wrong with that, but a philanthropic company could make things significantly safer for everyone if they weren't concerned about making money. Initiatives like the OSSF will hopefully have a positive impact, for this reason.
He was on the phone with the CISO who was explaining it's impossible to give him access to SPLUNK because of the network segmentation.
While he's ON THE PHONE, he received an email from the IT group with credentials to access splunk.
And to be clear, I left specifically because of their security stance. I was once told they couldn't automate pulling data from production because of the same reason as mentioned above, the network segmentation wouldn't allow it.
So no, developers aren't just whining because they can't directly access PAN.
Security people always think their concerns should trump everything else. I would almost be willing to bet 70% of the mind-numbingly stupid decisions made across the industry had some security justification behind it.
If human beings took the same approach to safety that Security people do to security, they'd insist the wheels on your vehicle should only be able to turn straight and right. That the vehicle should _actively_ prevent you from turning your wheels left because left turns are more dangerous than right turns and they can show that you can _always_ get to your destination with just right turns.
One of my former employers has developers, network admins and security professionals working together to maintain a deployment pipeline using Github, terraform and AWS to let developers do as much as possible without having to request anything from security, ever. All the guardrails and checks are built in. Labs get to deploy just about anything, test and prod are identical, and prod has implicit restrictions on requiring encryption for data, prohibiting excessively powerful roles, and so on. But they've worked directly with development to get them everything they need ahead of time, in order to make IT and the business as effective as possible.
Security is necessary, and good security does what it can to stay out of the way.
For pulling data from prod - this is often discussed. Data in production should not be pulled in lower environments (dev and test), because of segmentation, but you can absolutely operate with Prod data within prod environment, like by using approved production datalake or data warehouse or something.
Believe for every security decision that you think is stupid - there are many incidents that happened, and every rule and ban has happened because of these incodents/breaches/data corruption, etc.
It is like workplace safety instructions, they were written because of workplace injury, same for traffic laws.
That's everything that's wrong with the security mindset.
This is why only commercial company can build a great product. The constant cat and mouse game between threat actors and defenders/researchers. Every new malware/trojan/cryptominer strain needs to be found, identified, signature written, and all clients need tk get latest signatures asap, and product has to work flawlessly with as few false positives as possible
Of course; it's just funny to think about how much money gets spent detecting and fixing things downstream instead of fixing it at the source.
Security, public trust, etc requires controls and audit to deliver. You don’t want a startup “fake it till you make it” mentality in government or banking.
It’s simply good practice not to use your work machine for anything personal at all ever. Because depending where in the world you are, anything stored or viewed on a work machine gives your employer de facto access to it, legally speaking.
And there are perfectly valid reasons for companies to monitor traffic: data exfiltration, accidental or malicious, is a significant concern for companies that hold and process PII and for the people who have their PII held/processed by those companies. It is not as black & white as “monitoring and surveillance bad” unless you only care about your personal privacy.
The organization also exposes itself to greater liability. E.g., a rogue employee could use the trusted MiM CA cert for their own MiM e.g., capturing banking credentials of co-workers or accessing user/employee PII they would otherwise not have access to.
Yes, monitoring traffic by MiM https to external sites can alert you to / possibly prevent accidental exfiltration, but it doesn't prevent intentional exfiltration. It is, however, very effective at monitoring employees. The thing it is best at, might be its true purpose in an organization.
It can prevent accidental exfiltration, or deliberate exfiltration by a relative incompetent, which are the majority of such problems.
You are right in that they will not stop deliberate actions by a competent disgruntled or a competent external attacker who has access (but you have a much wider set of problems in this latter case).
Maybe I'm old-fashioned (I am definitely a “working in an office, living at home” person which seems to mark me out as a dinosaur in the coming remote-work age!) but I don't think it is my employer's responsibility to provide me with unfettered unfiltered internet access to do personal stuff with. Work stuff on employer provided Internet which they can monitor all they like, personal stuff on my own devices & connections which they can keep the hell out of.
Palo Alto appliance should be configured with both Forward Trust and Forward Untrust CA certificates, and the issue you described will not exist. If some people misconfigure - thats their fault for not following instructions.
Secondly, rogue employee doesnt have access to CA key that is stored in Palo Alto appliance. Only your firewall admin will have it, but if your main firewall admin went rogue, capturing colleagues’s data is the least of your concerns. Insider threat of that calibre is equally applicable to rogue CEO or CFO stealing all money from the bank. Or your ActiveDirectory admin getting CFO’s credentials and corporate bank credentials.
Having PA MiM TLS connections is the organization back-dooring itself as well as the external sites the users connect to. This back-door is available for abuse by IT staff, management and/or an attacker(internal or external).
There is a rule that seems to eventually always be proven-- if you provide infrastructure that can enable abuse, eventually it will be abused. Even if you and everyone else involved in the decision at your organization have good intentions, your future coworkers / management may not. Presumably the FBI and NSA have more thorough back ground checks of their employees than the average employer, and both have had employees abuse their access to surveillance data to e.g., stalk ex-girlfriends. And, even if the employee isn't rogue themselves, when the order comes from above, many will obey immoral/illegal orders-- e.g., Ronald Reagan, as president, had the FBI spy on his daughter's boyfriend. The safest option is to not to install the back-door in the first place.
PA's ability to tie Internet activity to specific users' identities was central to their sales pitch-- our tech staff assumed this was targeted at windows shops, but we used non-MS stuff including our LDAP servers and hoped this could kill the surveillance project-- PA countered that they could, at a last resort, do things like e.g., scrape radius logs to associate identities.
PA appears to be a competently run company that probably knows what messages are most effective at selling their product, and they really pushed user surveillance. Therefore, I suspect that many organizations who purchased PA products based that decision on the user surveillance capabilities (explicitly to enable abuse by management).
PAs main feature seems analogous to an illegal phone wire tap, and IMO should be illegal (especially without notification to the victims-- both on-site and off-site). It is curious how corporate circumvention of encrypted communications without permission of the external site hasn't been seen as a CFAA violation while a simple 'view source' on a browser can result in SWAT pointing a gun at your child.
in company of thousands users nobody has the energy to spy on employees - it is simply not worth the effort. Why would company spy on own employees, it is not something that brings profit for the company.
The only purpose of SSL decryption is to decrypt traffic and enforce policies: prevent users from going to shady websites, downloading malware, clicking on phishing links, stop viruses, trojans and hackers' command&control comms. It is because majority of http traffic is TLS encrypted, that security vendors no other choice other than decrypt and inspect.
Nobody is looking over zillions of logs, looking at what pages a random employee is browsing in a given day - aint nobody got time, energy, nor infrastructure to do that.
User identity (also device identity, and app identity) is used as to classify traffic and it is then up to company admins to create policy for enforcement.
Whatever the policy is - it will be enforced, and it is the same policy&terms you agree to by signing employment contract.
Which says something like - your work laptop and corporate Internet connection can only be used for work related stuff and not personal stuff, etc, etc.
In more civilized areas of the world privacy rights are explicit, and even things like employers snooping on employee email accounts on company owned email servers is illegal. When at work, you are selling your time to your employer, but that doesn't imply that the employer owns you while you are at work.
As to the sibling comment about cameras in workplace bathrooms, yes employers did this and now there are laws prohibiting it. Now, employers just account all your time using bossware leading to e.g., folks at Amazon having to pee in bottles or wear diapers on the job to not get fired. There is no line that some capitalist employer will not cross unless we place limits with consequences to reign them in-- e.g., we no longer have employers forcing small children to crawl into running machine tools to clear a jam while risking a limb being sucked into the mechanism and turned into hamburger meat-- but, we did, it was common-- lives of the poor (especially children) were cheap, but stopping an assembly line was expensive.
And statistics of developers falling victim of phish attack, credentials stealing that leads to major breach - there are plenty. The most recent Uber hack or Okta hack - were all tied to developer clicking on stupid stuff, opening executables from Internet and getting his a$$ owned.
You just gotta accept the fact that developers are not security specialists, most of them cant even create a software without introducing plenty of bugs and vulnerabilities. They mostly google stuff and copypaste from stackoverflow, install shady barely working packages and copypaste directly into production whatever code snippet they found on the first page of Google results. Thats why they need extra control from security specialists
And now I'm going to quote myself from earlier to make it clear you're displaying exactly the silliness I was speaking to, with added emphasis.
"If human beings took the same approach to safety that Security people do to security, they'd insist the wheels on your vehicle should only be able to turn straight and right. That the vehicle should _actively_ prevent you from turning your wheels left because left turns are more dangerous than right turns and THEY CAN SHOW THAT YOU CAN ALWAYS GET TO YOUR DESTINATION WITH JUST RIGHT TURNS."
---
You see, you can still get to your destination with no left turns, it's just really damned inconvenient and has costs in terms of happiness and time.
It's a classic case of security people making decisions they themselves don't have to pay the cost of.
And don't get me wrong, you'll often hear security people _CLAIM_ they do, in fact, adhere to all of the security practices they insist on. And they may even do so.
But ...
THESE SECURITY PEOPLE ARE NOT DEVELOPERS.
There's no critical thinking in these decisions. A phone agent working in a very specific application all day doesn't need access to the PC the way a developer does.
---
> And statistics of developers falling victim of phish attack, credentials stealing that leads to major breach - there are plenty. The most recent Uber hack or Okta hack - were all tied to developer clicking on stupid stuff, opening executables from Internet and getting his a$$ owned.
uber hackers got through using slack, okta was a technician RDPing in.
Neither were developers, and unless you're prepared to claim slack wasn't sanctioned by the company, it's all just a long worded admission that removing local admin rights didn't actually help.
Then there's the question of, if someone steals a developers credentials, what do they have access to?
THAT is where the rubber hits the road. I've literally seen the following:
- Disallow developers from running powershell, but they can log directly into DB's with PII and PHI data ("they had a legitimate business need").
- Force developers making 6-figure salaries to "request access" for admin or the installation of software, said requests being granted by support teams of people making a little over minimum wage.
There's a reason why so many people call it security theatre.
> You just gotta accept the fact that developers are not security specialists, most of them cant even create a software without introducing plenty of bugs and vulnerabilities. They mostly google stuff and copypaste from stackoverflow, install shady barely working packages and copypaste directly into production whatever code snippet they found on the first page of Google results. Thats why they need extra control from security specialists
The reason your company is full of such developers is because you took away local admin rights and the ones with options left. You don't even have any left who could mentor the ones that need mentoring, they left too.
Put yourselves in the shoes of that developer who can access PHI at will, but cannot update their Visual Studio in the name of security because it requires local admin rights to do so.
One possible thought - they think very highly that you will do the proper thing, but they cannot and do not trust every single vendor out there. Have you heard of SolarWinds?
And honestly updating Visual Studio is something that can be arrange but would take probably 1 hour of IT time to solve and I'm sure they have other things they need to do.
Developers are not special to NEED admin access. they may WANT it because it's more convenient, but convenient is not secure. Maybe you're the most 1337 developer out there, or maybe you're a corporate spy.
Perhaps instead of lashing out and getting angry, approach this like a developer - what are they trying to achieve? Do they have technical debt like you? Is this a good enough solution for most use cases?
Inconveniencing you is not the main goal, so perhaps understand what their main goal is.
using your car analogy - car designers created steering wheel, blinkers, and mirrors to increase safety, but you insist that since you are power user - you want to be able to turn left/right by drifting using parking brake. This is obviously safety risk on public roads, and understandable how corporate fleet employer like Greyhound might not allow drifting when driving company bus with passengers.
You are free to drift on your personal equipment though, during non-work hours and without wearing company clothes.
Developers really dont need admin rights, Visual Studio and any other software is updated automatically these days using tools like SCCM. This is not an issue at all. If you need full control over OS - install free VirtualBox, or get a lab VM and do whatever you want inside that isolated VM, but not on the host machine. Because your machine is tied to AD, email, bunch of other corporate stuff - IT cannot risk giving admin rights, so that you can disable all necessary security protections.
Just because you are power user, doesn't mean your colleague in next cubicle is as smart and doesn't click on phish Linkedin emails.
PII is not an issue at all, because of security endpoint agents, network traffic inspection, data loss prevention, and network segmentation, and bunch of other security controls.
Just because you make over 6 figures doesnt make you any better than minimum wage IT support folks, they are following scripts and established procedures very well, most of them do their job well.
I agree that a lot of places have security theatre, because Security engineering is even rare skill than software engineering, it is much harder to find skilled seceng than SDE.
But things like SQL injection, shell command injection, url traversal, and zillion of other attacks - are made possible by software developers, and it becomes then SecEng's problem to protect company against whatever crap they coded and pushed to prod.
Most of the time, I genuinely wish it wasn't. There's so much I have access to, that I would never, ever need. And because I have that, I can't access things I actually need.
Just give me a iPad for all the corporate stuff and let me work on an open PC.
First you compare the risk of losing limbs to having admin rights, now it's drifting on public streets is like wanting to install python.
You can't find anything reasonable because there isn't any.
Consider the possibility that you may be wrong.