Tell HN: There’s no room for skunkworks in modern software development

17 points by 00000000005 4 years ago | 23 comments

In the old days, corporate rebels could almost secretly - but often simply openly and casually - run special projects to create new and innovative software.

Modern software processes make this impossible. It would be deeply frowned upon, likely a sacking offence, to create software outside the tightly controlled process.

Does this mean anything? I don’t know. But what it does mean is any company that runs an organized process has no chance of “random innovation”. That concept, which was common in “the old days”, is gone.

Modern organized software development kills innovation.

ch_123 4 years ago |

As someone who has created several skunkworks projects at large companies which got shipped, I'm wondering what exactly the OP's definition of "Modern organized software" is. I have definitely worked in an environment where creating a skunkworks project was next to impossible due to a toxic micromanagement culture, but in that case I left the company instead of assuming that it was representative of the software industry as a whole.

EDIT: I'm also confused about the talk of "corporate rebels" on one hand, but the complaints about dealing with processes on the other. What do you think corporate rebellion means? Do you think that "rebellion" means not having to deal with any attempt to stop what you are doing? Frankly, trying to sneak around processes is half the fun!

jasonpeacock 4 years ago |

Nope. Where have you worked to get that impression?

> But what it does mean is any company that runs an organized process

That organized process is for deliverables that the company is investing resources into (and managing the costs of). The existence of such a process does not preclude the development of software/products outside of that process.

I've delivered many "skunkwork" projects, some of which have become critical parts of the company, while still delivering on official projects. It's stuff that I've been passionate about, or a personal pain point, or an experiment.

I've even done it as part of official projects, e.g. "we need to deliver A, but that requires B which is really annoying/slow/expensive - I'm going to solve B first and then A will be trivial". No skunks required.

> It would be deeply frowned upon, likely a sacking offence, to create software outside the tightly controlled process.

Nope. The response is "let's adapt this to fit into our process", not "you're fired!". The worse response is usually "we don't want it" or "we already have a project in flight to build that" and you've just wasted a bunch of time.

> Modern organized software development kills innovation.

It sounds like you're recently frustrated about something? Many companies have delivered innovative software through modern, organized software development. Where do you think AWS came from?

austincheney 4 years ago |

I largely disagree not from a policy position but in practical terms. In most major employers modern software processes allow for major time gaps in delivery and productivity. That time can be spent on anything regardless of employer blessing. Although personal projects created in these time pockets do not carry the endorsement of the employer they are often silently blessed as no cost innovation.

00000000005 4 years ago | |

So in non-corporate speak, the company is so slow, and the workload on developers so light, you have time to do whatever you want?

Wouldn’t dedicating that time to a project benefiting the company be an admission that you’re under utilized and slacking off?

austincheney 4 years ago | | |

Under utilized yes, but slacking off no. An employee is beholden to the work assigned to them. Large pockets of availability occur for developers when either a developer is substantially more productive than their peers or when internal processes limit increased productivity. In either case the developer is doing all they can to do their job with plenty of availability remaining for personal work.

xg15 4 years ago |

"Corporate rebels", wow. What exactly are they rebelling against? The fact that the companies' ROI could still be higher?

00000000005 4 years ago | |

It’s hard to explain now, but in the old days it mattered. Sometimes software was written by employees just because they wanted to, because they thought it was a good idea, and it wasn’t planned and organized and ticketed and budgeted and scheduled.

And sometimes it saved the company.

emptyparadise 4 years ago | | |

Employment for life is long gone. People with good skills are best off applying their abilities to their own projects. The company will not save you, why should you save the company?

taylodl 4 years ago |

It's like boys and girls having sex - they'll always figure out a way regardless of what you do to try to prevent it. Skunkworks works the same way. If you want to do it then you'll figure out a way to do it. That's the way it's always been, and I've been doing this for decades.

glanzwulf 4 years ago |

But at the same time it's so much easier (and even encouraged) to do your own thing. There are startups popping up daily.

So whilst modern "cog in the machine" software development kills innovation within the company, we live in an age where "anyone" could build the next facebook.

00000000005 4 years ago | |

The message however is this: An organized software development process kills innovation

That’s true for startups as well as big companies.

A company of four people could have a dead software process just because it’s super organized, with jira, tickets, sprints, user stories, CI/CD, testing deployment and monitoring.

The more organized your software process, the more dead it is, in terms of creativity and innovation.

w_t_payne 4 years ago | | |

I fundamentally disagree. It's entirely a matter of how that process is manifested and the impact it has on the psychology of team members.

A good process can be empowering. It's just that 98%~ish of processes are not good.

salawat 4 years ago | | |

Bullshit. You don't scale without process and people being able to make safe assumptions about what's going on elsewhere. Process is what enables your maintainers to plug into never before seen code and handle it as just another piece in reasonable time.

You're coming off as someone whose never had to coordinate a large number of people to deliver anything concurrently before.

cheradenine_uk 4 years ago |

BS.

I've run skunkworks projects in every single organisation I've worked for, and it's vastly easier today to 'pad out' the way "Agile" is practiced these days than it was in the past.

28304283409234 4 years ago |

In Ops, I've been in a situation multiple times that I was frowned upon for fixing production outages, instead of working on "The Sprint".

Example: I noticed failing memory banks on the primary postgres server. And got reprimanded for for fixing it (failing over to standby server, replacing mem, etc...), instead of working on the sprint.

rad_gruchalski 4 years ago |

> Modern software processes make this impossible. It would be deeply frowned upon, likely a sacking offence, to create software outside the tightly controlled process.

That’s a hell of an assumption. I guess it depends on the company?

tony-allan 4 years ago |

I expect that away from central IT control "corporate rebels" in business units can still create mostly internal tools to improve productivity for smaller groups of users.

TruffleLabs 4 years ago |

Innovation is not random; innovation is a process that must be deliberate.

Randomness can occur in the process but a deliberate process is needed for "random" ideas to become reality.

szszrk 4 years ago |

You are right at many levels. There are still many reasons why I would not like to work in a place that does just that - it's chaos. But some time should be spent that way, mandatory!

I actually have a boss that literally made a "skunkworks" weekly meeting that was 4 hours and meant "turn off phones, email, IM, work on cool things". It transformed into a full day (weekly) for a while, but we failed to keep it up. Fortunately this transformed into a full time, modern, project-only job for me.

We are lead to believe that "skunkworks" is done by R&D departments, but all I saw were failures in that regard. It was one of:

- R&D does cool things, keep them for themselves and maybe produce some procedures to pass further to Architecture. Random IT person has no benefit and no knowledge from that work.

- R&D doesn't do R&D, it does projects and development, they just have less compliance, which is destroying morale.

- There is a huge startup within the company, which has hundreds of "old" people transfered, new cool cloud tech, new offices, agile flow or whatever else they want to do. It never goes beyond that very group and dies as soon as the product goes into stable production (successfully). No lessons learned, we did things the old way.

- My Business colleagues in one of the biggest banks worldwide were doing full time "project" work, which meant they were designing and testing complex business system. Company had cool bonuses for "innovation". They did a lot of real innovation, proposed and implemented different ways to do business. Never got bonuses as "your team is a project team, so innovation was part of your daily tasks".

- Same company, but in IT - also awesome bonuses for innovation. This was THE topic, your yearly goals were all around it. Can we make tools? No! It's forbidden. Can we code? Forbidden! Can we script? Maybe, if it's not coding. But even if it is you are forbidden to install new tools anywhere. All script languages are forbidden.

I'm not a coder, but can't stand stagnation and obsolete tech. Did several tools which were plainly a workaround compliance. Implemented small custom DNS query engine in VBscript, as this was the only thing I could use on a workstation. In 2015 or so. Did file processing automation for production systems in perl. NOT OFFICIALLY, of course! Perl was forbidden, like everything. So I used Perl build into Oracle Client package - old Oracle with much older Perl inside. Hide and seek game, all day long.

Skunkworks should be mandatory :)