Amazon CodeGuru – Preview(aws.amazon.com) |
Amazon CodeGuru – Preview(aws.amazon.com) |
The best thing about it was the recommendations on how to use the AWS SDK better as that's probably got the most potential to drift or make mistakes on
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
> CodeGuru’s machine learning models are trained on Amazon’s code bases
> comprising hundreds of thousands of internal projects, as well as over
> 10,000 open source projects in GitHub. Tens of thousands of Amazon
> developers have contributed to CodeGuru’s training based on decades of
> experience in code review and application profiling.
Amazon packages open source software (Linux, Postgres etc) in a way that is an abstracted service (RDS, EBS, Elastic Load Balancer). They add so many abstracted building blocks that you need a special skill set to manage them (Aws Certified Solutions Architect) instead of knowing how to do this with bare metal or a container image running in your own data center.
And now that things are complicated and developers might make mistakes using those services, they add a profiler that inspects your code running in production and a reviewer that ties into the stage before deployment. All just to optimize the use of their own services.
From a business perspective this is an awesome way to get vendor lock-in to a much higher degree. They are basically the certifying authority that tells you if your intellectual property (your code) conforms to their own standard. Yes, they show examples of standard Java optimizations, but it clearly says it detects deviation from best practices for using AWS APIs and SDKs.
And people were mad at Microsoft for shipping a non standards compliant browser as default and enriching it with HTML tags and plugins that would only work in that browser. Little did we know.
I personally wait for the "Amazon Compliant Code" label in the not too distant future as a selling point for business people.
Sure, google and Microsoft and IBM joined the party, but AWS was first and remains the best holistically. This is their moment of domination, and eventually something will knock them down, but they have made so many companies so nimble and powerful in ways that were impossible before. Go Amazon.
WhatsApp Stats (2014):
- 450 million active users, and reached that number faster than any other company in history.
- 50 billion messages every day across seven platforms (inbound + outbound)
- 32 engineers, one developer supports 14 million active users
- $60 million investment from Sequoia Capital
Which they managed their own FreeBSD servers hosted on SoftLayer.
[1] http://highscalability.com/blog/2014/2/26/the-whatsapp-archi...
YouTube (2008):
"YouTube grew incredibly fast, to over 100 million video views per day, with only a handful of people responsible for scaling the site"
- 2 sysadmins, 2 scalability software architects
- 2 feature developers, 2 network engineers, 1 DBA
"They went to a colocation arrangement. Now they can customize everything and negotiate their own contracts."
"Sequoia invested a total of $11.5 million in two separate rounds and was the only venture firm to invest in the company." [3]
[2] http://highscalability.com/youtube-architecture
[3] https://www.nytimes.com/2006/10/09/business/09cnd-deal.html
The part of that solution which was related to making the system scalable was written by two of us, who also did other things (it involved a partionable backend storage service, and a user registration service, that combined to let us migrate users between servers to even out load and partition storage; everything else was stateless).
This idea that AWS is necessary to build to scale with small staff just does not match reality. My years of consulting also showed me that I'd earn more from clients who insisted on AWS - they typically spent far more time and resources on devops (and spent far more on hosting overall).
AWS is convenient, and it's great when you can afford it, but it's expensive and still requires substantial devops effort.
Indeed. Hopefully, soon they'll stop selling physical items in that online store they have and focus on their strengths, so that other companies, who might be able to do better at selling things that aren't so frequently counterfeit that I no longer buy anything from Amazon, can have a go.
Could you expand on this?
I can't tell if you mean to launch a service/company or if you're talking about some large scale.. thing.. I haven't heard of.
I'm being serious, how do you design an API and a set of distributed intercommunicating systems in a way that doesn't couple you with their specific APIs, communication channels and semantics?
I don't think it's possible.
I can see suggesting using open source solutions instead that you run on your own, but that still couples you to those specific solutions, except they're open source so in theory you could fork it and have more control over them. I get that. But this is a different argument I feel. Since the cost of maintaining these open source products on your own is high, and the cost of switching to a different open source solution is as much as moving to another cloud provider.
I think the only form of lock in right now that might seem designed by the business, and not an artifact of the tech itself, is the high price of exporting your data out.
I think AWS is basically a large SaaS that sells you solutions to problems you have at scale. I don't think the lock in is only in the effort to export data, the lock in is also that a company will use the same building blocks for every new project or new feature on existing projects because their current staff is already trained and new hires don't know how to do it without those services either.
So just like in the 90's nobody got fired for buying IBM, today nobody gets fired for using AWS, even though they don't have problems at the scale AWS is great at.
I believe this is what Rackspace was attempting a while ago -- leveraging OpenStack to provide cloud services, so you'd be free from lockin in the sense that you could move to some other OpenStack compatible provider.
Wow, this is a scary but very real thought.
Though, the "Certified Windows XP / 7" stickers on hardware and video games / other software was quite common back in the day and isn't too dissimilar.
I would argue that the level of vendor lock-in Amazon is going for is far greater than Microsoft's.
If there were an open standard for "way to upload and store and serve files on the web", and S3 happened to implement that standard, and other companies and open-source projects did as well, then it wouldn't matter to me if AWS was the bundling king or not.
There was nothing magical about the design of Unix, either. It's not the only way to make an operating system, or even the best way. It survived because we got many competing implementations which were basically source-compatible. It really took off when we got free clones that anyone could run on their PC.
There's lots of focus here in the comments on the code reviewer portion, but one of the things I'm most excited about is the profiler - https://aws.amazon.com/codeguru/features/
I do a lot of performance engineering work, and one of my go to tools for visualizing where programs are spending their time is flamegraphs. While you can certainly create them with profilers besides CodeGuru (and I do not work with Java, so I haven't yet had the chance to check out CodeGuru for any of my use cases), I'm super excited about anything that gets more people using them. They make it very easy to see where your optimization opportunities are, and I have personally found them very useful when working with our customers - they're way easier, in my opinion, to go through and explain than just looking at raw perf output or similar.
(The upsell price may be free for now, but who knows maybe they add a premium version or enterprise features in the future)
I imagine that the tool will be used for recommending more amazon services in the future, and this is possibly a poor POC of more to come.
https://github.com/corretto/amazon-corretto-crypto-provider
They claim to be 25% faster than standard implementation: https://aws.amazon.com/blogs/opensource/introducing-amazon-c...
https://github.com/pediredla/Algorithms/pull/3/files
It looks like a linter, but maybe there is more.
> You are using a `ConcurrentHashMap`, but your usage of `get()` and `put()` may not be thread-safe at lines: 110, 113, 135, and 137. Two threads can perform this same check at the same time and one thread can overwrite the value written by the other thread.
This one could be a fairly simple rule ConcurrentHashMap.get() followed by some code that branches on the result, followed by put() is unsafe. These warnings can be very helpful, but no fairy ML magic needed.
But due my Visa situation here in the US (H1B), I'll be never able to monetize it as it's illegal to have a side income. But I think this is just the start and there is an huge opportunity for new startups and projects.
DISCLAIMER: I am not an attorney. More importantly, I am not your attorney. The above is not legal advice. If you desire legal advice, consult a competent, licensed attorney in your area.
https://engineering.fb.com/developer-tools/getafix-how-faceb...
I think these things make the most sense for Java and Go where there tends to be lots of repetition and lower-order programming patterns.
Unlike say, Python, Lisp, or Rust.
I feel like this should have a generous free tier for open source projects. I feel that very, very strongly.
I might be wrong though.
How well it works is beyond me though
Trade offs sure but design and scaling need to be considered _before_ the code review. Maybe an architecture review of sorts? Once you hit code review it's a little too late to reconsider design and scale unless it's a serious issue.
> mentor others
Mentoring is mostly outside of a code review. Sure it can help with that but I don't think that really counts. IMO anyway.
Strongly disagree, at least for remote teams.
Working remotely, I've personally found code reviews to be a great way of mentoring less experienced team members.
I also encourage junior team members to review code of more experienced team members.
For big changes, we discuss proposal/API/code reviews as a team.
I've had several people provide feedback that they've learned a lot from reviews like this, and honestly I wish I'd had this kind of mentoring when I started out (I was basically a one-man cowboy-coder for the first 5 years or so of my career).
I know mentoring can be seen as a chore for many, but it can be seriously rewarding too!
https://engineering.fb.com/security/zoncolan/
https://www-wired-com.cdn.ampproject.org/c/s/www.wired.com/s...
Also reminds me of sonatype or findbugs which does something similar but works on a set of rules instead of on ML.
I can tell you that there were lots of managers, PMs, directors, etc involved and they considered tons of naming options. They took into account third party services/products, first party services/products, and other things that might have overlap. This was likely the situation here and they accepted this as a drawback.
That said, you're free to disagree and that doesn't mean it was the right choice, just wanted to point out that this was not an oversight.
With codeguru.com being a long established site, doesn't that make it worse?
Hence, it is pure garbage.
> For example, if you have a typical pull request with 500 lines of code, it would only cost $3.75 to run CodeGuru Reviewer on it
Wat?!
Come on, $4 per review is not inexpensive, especially for what is essentially a glorified SAST!
Why not something more obvious like one of AWS Code Auditor/Reviewer/Checker?
What's interesting is that this technique correctly handles inter-thread effects like blocking, locking, contention, so it can point out inter-thread issues that traditional profilers and flame graphs struggle with.
Summary: https://blog.acolyer.org/2015/10/14/coz-finding-code-that-co...
Video presentation: https://www.youtube.com/watch?v=jE0V-p1odPg&t=0m28s
Coz: https://github.com/plasma-umass/coz
JCoz (Java version): http://decave.github.io/JCoz/ and https://github.com/Decave/JCoz
A bit of an (almost) shameless plug is a project I have been working on at https://blunders.io. A bit similar to the Code Guru profiler, but with a different feature set.
I've used Blackfire for a while, and this type of visualization is definitely helpful for finding bottlenecks in web performance. I've been able to reduce page load by caching big chunks that I was able to see in the graph / timeline.
You can look at the introductory video [2] to get an idea
[1] https://github.com/Netflix/flamescope [2] https://www.youtube.com/watch?v=cFuI8SAAvJg
EDIT: missing anchor
For code review services I’d expect a level far above this. Maybe they are able to do that, but I don’t have any existing positive bias towards this, and a few things against it.
Just needded to add an AWS library in my code base and BAM! here is how my console will look on every reload from now on :
:8081/index.bundle?platform=ios&dev=true&minify=false:93 Require cycle: node_modules/aws-sdk/lib/react-native-loader.js -> node_modules/aws-sdk/lib/credentials/temporary_credentials.js -> node_modules/aws-sdk/clients/sts.js -> node_modules/aws-sdk/lib/react-native-loader.js
Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle. metroRequire @ :8081/index.bundle?platform=ios&dev=true&minify=false:93 :8081/index.bundle?platform=ios&dev=true&minify=false:93 Require cycle: node_modules/aws-sdk/lib/react-native-loader.js -> node_modules/aws-sdk/lib/credentials/cognito_identity_credentials.js -> node_modules/aws-sdk/clients/cognitoidentity.js -> node_modules/aws-sdk/lib/react-native-loader.js
Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle. metroRequire @ :8081/index.bundle?platform=ios&dev=true&minify=false:93 :8081/index.bundle?platform=ios&dev=true&minify=false:28851 Warning: AsyncStorage has been extracted from react-native core and will be removed in a future release. It can now be installed and imported from '@react-native-community/async-storage' instead of 'react-native'. See https://github.com/react-native-community/react-native-async... reactConsoleErrorHandler @ :8081/index.bundle?platform=ios&dev=true&minify=false:28851 :8081/index.bundle?platform=ios&dev=true&minify=false:93 Require cycle: node_modules/@aws-amplify/analytics/lib/Providers/index.js -> node_modules/@aws-amplify/analytics/lib/Providers/AWSKinesisFirehoseProvider.js -> node_modules/@aws-amplify/analytics/lib/Providers/index.js
Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle. metroRequire @ :8081/index.bundle?platform=ios&dev=true&minify=false:93 :8081/index.bundle?platform=ios&dev=true&minify=false:93 Require cycle: node_modules/@aws-amplify/predictions/lib/types/Providers/AbstractConvertPredictionsProvider.js -> node_modules/@aws-amplify/predictions/lib/types/Providers/index.js -> node_modules/@aws-amplify/predictions/lib/types/Providers/AbstractConvertPredictionsProvider.js
Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle. metroRequire @ :8081/index.bundle?platform=ios&dev=true&minify=false:93 :8081/index.bundle?platform=ios&dev=true&minify=false:93 Require cycle: node_modules/@aws-amplify/predictions/lib/types/Providers/index.js -> node_modules/@aws-amplify/predictions/lib/types/Providers/AbstractIdentifyPredictionsProvider.js -> node_modules/@aws-amplify/predictions/lib/types/Providers/index.js
Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle. metroRequire @ :8081/index.bundle?platform=ios&dev=true&minify=false:93 :8081/index.bundle?platform=ios&dev=true&minify=false:93 Require cycle: node_modules/@aws-amplify/predictions/lib/types/Providers/index.js -> node_modules/@aws-amplify/predictions/lib/types/Providers/AbstractInterpretPredictionsProvider.js -> node_modules/@aws-amplify/predictions/lib/types/Providers/index.js
Require cycles are allowed, but can result in uninitialized values. Consider refactoring to remove the need for a cycle. metroRequire @ :8081/index.bundle?platform=ios&dev=true&minify=false:93 :8081/index.bundle?platform=ios&dev=true&minify=false:93 Require cycle: node_modules/@aws-amplify/predictions/lib/Providers/index.js -> node_modules/@aws-amplify/predictions/lib/Providers/AmazonAIPredictionsProvider.js -> node_modules/@aws-amplify/predictions/lib/Providers/index.js
>It’s like having a distinguished engineer on call, 24x7
I don't believe that, regardless of how many times they sprinkle in the words "machine" and "learning".
My original comment was definitely unclear. I actually had two separate thoughts (that I didn't communicate well at all):
(1) if your team has occasional large, automated PRs (code generation, automated refactors, etc), you probably don't want to run this tool on them because of cost, so anyone that has these large PRs and uses CodeGuru probably needs to build a way into their automation to suppress CodeGuru (or build a way to invoke it for specific PRs)
(2) I also wonder if it's good enough to justify the price on regular PRs
We don't have many situation (1) PRs where I work now, but they do come up occasionally. For example, I've used IntelliJ IDEA's structural find-and-replace to do very large automated refactors where CodeGuru would be very expensive and probably provide little value. We also do check in some generated code (we usually don't do this, but there are a couple exceptions where we weighed the tradeoffs and decided checking in the generated code was a better solution, in our eyes).
A good code base is a team-created intellectual work. For that to happen, you need a ton of collaboration, shared learning, evolution of norms, interpersonal bonding, and practice of key social behaviors (e.g., principled negotiation, giving good feedback, recognizing and rewarding good actions). Automated code review gets at none of that.
If you divide the 50 bucks by that number, you get a cost ratio. If it's lower than the ratio of (benefit expected by automatic code review) / (benefit expected by manual code review), it's worth using.
I guess we can speculate all we want; in the end, only experience will show if the service is worth it or not.
There are lots of academic ML review/suggestion tools. Those people come to the table with trials and statistics to assess the quality of their results. Amazon probably copied one of those papers, added a rules-engine to recommend their own APIs, and slapped a hefty price tag on it.
If it can spot a lot of issues (performance, security, bug, etc), $3.75 is definitely a good deal to do it once a while but not on every single changes (e.g. fixing a typo in the code comment)
That's a pretty deep rabbit hole. But considering that old "IDEs" with crappy "Intellisense", "Quickfix" or similar were widely sold, there's potential there.
(Disclaimer: I'm working for GitHub, but on a different project)
There is a sweet spot where cloud is good and provides some benefit but, once you're serving hundreds of millions of people and have double-digit millions in investment, you can probably do significantly better cost-wise rolling your own servers. Worst case, you just throw your own hypervisor management system on them and have most of the same features you got from a cloud service. If you're smart, you can probably architect it so you have on-demand overflow capacity from a cloud provider in case there's a spike you can't account for, which is the best of both worlds.
and yeah, they were down all the time, but that didn't seem to matter to their growth.
Lots and lots of getters and setters in Java.
I am always fascinated by deducive ignorami who pretend to have done the research, like yourself.
Hard to imagine that without AWS.
The announcement says it can even analyze parts your code that are more computationally expensive than they need to be. I'm not sure I understand the skepticism--surely they have among the largest code repositories in the world. Why couldn't they train models on it to look at best practices and even compare code practices to different metrics.
It's a spectrum. Now isn't special.
This is like saying that using React will lead you to be locked into Facebook?
The name might be awkward, and codeguru might be slanted towards suggesting open source libraries written at Amazon, but it is about as neutral as can be, not even requiring you to use Amazon’s OpenJDK distro.
Why does Amazon have an OpenJDK district though? Because sometimes Amazon sees performance issues at scale that they have a hot fix for. Then they share the patch with the wider OpenJDK community and have discussions about if there are better approaches to fix. Amazon has been one of the top contributors to OpenJDK releases recently (typically in top 3 for contributions to a given release), so they really are upstreaming patches.
FYI, as I commented below, it's also available on Github under its own user and with a very permissive (GPLv2) license: https://github.com/corretto/corretto-8
More recent efforts have us check in generated code alongside the "config" files, and automated processes ensure you check in the generated code if you touch the config file. It's much better this way.
Only downside we've found is it can be a pain with searching for references in Github and you have to remember to generate the code, but for the most part it is seamless.
I guess how you manage it depends on your IDE, if you can configure it to work nicely, and how much the generated source changes/needs to be read.
* Hermetic builds are faster because code-gen only occurs when changes occur in the base code
* Lots of docgen tools don't support incremental compilation
* Diffs in generated code show up as diffs when you change the code-gen tool, easier to isolate changes that occur if your code-gen tool is upstream (say you want entire org on Thrift 0.9.2 from Thrift 0.8)
Downsides I can see:
* Large repo.
* Source of truth is now the generated code, not the source, so someone else using the source could get a different result.
Essentially acting in an empirical mode of operation (i.e. does it provide benefits for cost), and ignoring any philosophical objections, this seems like it could go either way depending on the situation.
This seems to apply to the other side, I think. Generating from source with different tooling or tool versions could create different results, whereas using the generated code guarantees consistent behavior.
Does the (sub)domain something is hosted on actually matter when the ownership situation is the same?
While the product is free it doesn't mean the patches will be beneficial anywhere else, in fact if it will work better anywhere it would be merged back to OpenJDK and we wouldn't need the fork.
Same thing with Amazon Linux, sure you can use it on premise, but it is tuned to work best on AWS and might actually work worse outside than other distros.
Additionally, the entire thing, again, is open source and with a permissive license meaning nothing is stopping anyone from forking it and doing what they'd wish with it.
You are in fact right that it was created to work well with AWS, but I fail to see how that is 'lock in', since most of those benefits are probably benefits on any modern cloud - since AWS does not generally run on a particularly unique architecture.