Show HN: Anonymous Feedback Tool for Teams

Show HN: Anonymous Feedback Tool for Teams(runsignals.com)

46 points by techdiff 3 years ago | 59 comments

Hey there!

Signals is a survey system that collects feedback from staff (mostly) but also clients and stakeholders in a business. It does this via SMS on a weekly schedule (the cadence can be changed but it works best when done weekly).

My co-founder and I started working on this nearly a year ago, having run similar small builds for over four years. This time we’ve tried to do it properly. The execution is relatively simple - similar to an eNPS (Net Promoter Score - a common way of measuring how consumers like your product or service), but we wanted a way to anonymously pass candid feedback from employees to managers and executives about their workplace experience, their jobs, new ideas... anything. And for it to be regular and easy to do.

We maintain the anonymity of the staff member (and let them know how many people are in the team receiving the question being asked, so they understand their level of safety). We show the team leads and managers a sentiment score and verbatim comments), but we do not associate the respondent details with those in the backend. This is somewhat different to currently existing tools that tend to use the term ‘confidential’, which means you're not anonymous if the admin permissions are high enough. Even those systems which claim to be anonymous can often have ways of twisting the data to unmask the users.

One of our team is working on natural language processing to understand, summarise and report on sentiment, comment themes and trends. We’re making it easy to add AMAs, poll clients and partner businesses, and we’re experimenting with sports organisations, unions, and within schools and education.

It runs over SMS (the highest response rate of any method we tested). Unfortunately, it does need a signup and confirmation (apologies) to try, but it’s free to test, and there’s no credit card required. Unfortunately, we only support the US, Canada, Australia and the U.K. at the moment but are looking to expand support as soon as possible.

We have only really been live for eight weeks now. We would love any feedback you have for us and hope you find it useful! You can email us at hello@runsignals.com if you have feedback or want to chat.

rcaught 3 years ago |

To ensure anonymity, you could always reprocess feedback through something like ChatGPT.

> rewrite the following and remove personally identifying information: "you're not good at your job, you have a bad attitude, especially with Katelyn."

> "Performance in the job is not up to standards and there are concerns about attitude, particularly in interactions with colleagues."

techdiff 3 years ago | |

This is a super interesting idea. It might muddle things like sarcasm, in-jokes (we see a lot of those around Christmas party time) and things like that. But even if it was something the user could control perhaps, it might be a great addition.

dt3ft 3 years ago |

Employee surveys are never anonymous, even if you say they are. If I get a link containing an ID, it can be traced to me. If I had a bowl of tickets with these links, and was offered to select one (lottery style), only then would I trust in anonymity. Of course, the survey would have to be reachable from outside of the corporate network for all this to work.

beachy 3 years ago | |

> If I had a bowl of tickets with these links, and was offered to select one (lottery style), only then would I trust in anonymity.

Actually a really great idea perhaps.

In the HR space there is definitely a place for pseudonomous feedback/issue raising. Not anonymous where a disgruntled employee can raise 10 separate issues and appear as 10 disgruntled employees.

Not sure how your idea would work in practice but letting the employee choose a "ticket" sounds like a nice concept.

krageon 3 years ago | | |

"in the HR space" is where anonymity must exist, otherwise the feedback can never be safe. A "Human Resources" department is hostile to humans (it exists to better exploit you, as an agent of your employer), so you must act as if you have an adversarial relationship with it.

dt3ft 3 years ago | | |

I imagine it working like this:

1. A set of questions is created on which the employer would love to get answers to from employees.

2. If a company has 100 employees, 100 unique tickets are printed out on a piece of paper (it could contain a qr code which leads to the survey)

3. All paper tickets are placed in a bowl at the reception and employees are able to take a ticket each

4. Employee goes home, uses their phone to scan the QR code and fills out the survey, ensuring total anonymity. Each ticket has a unique ID, ensuring only one response per ticket. The empployer can not possibly know which ticket was taken from the bowl by which employee.

5. The results are in, and employers get 100% anonymous but 100% honest responses.

blowski 3 years ago | | |

We use OfficeVibe, and I have full employee access. There is absolutely no feature anywhere that shows me who submitted a comment. I could just about hack one together by putting each person into a one-person team, but the user would be able to see that themselves. Sure, we can imagine a conspiracy where my CEO has a special button, but I strongly doubt it.

That said, it's still frequently obvious due to people's writing styles, the nature of their complaints, timing.

jedberg 3 years ago |

How do I as an employee know that I'm truly anonymous and that my employer won't be able to pay you/sue you and get you to tell them who wrote something?

techdiff 3 years ago | |

Thanks for the question! Absolutely understand the scepticism. We have a user ID that we match to the Twilio response, so staff can't submit multiple answers. We delete our data from Twilio after seven days at the moment (Twilio mandated min); we’re working on a 1-hour retention policy via their Rest API. We’re always working toward minimising any end-user identifiers back to comments.

For me, there's also our reputation - if we claim we're protecting users and then don't, our product would be over quickly.

neilv 3 years ago | | |

Have your lawyers looked into whether anything can be done in the customer contracts, to help avert having to disclose this info to them (and all the damage that might mean for your trust-based business)?

underwater 3 years ago |

I really don't understand the paranoia around employee surveys. In 20 years as both an IC and a manager I've never seen them used in a way other than described.

If your manager wants to fire you they don't need to create a complicated ruse to get evidence that you're unhappy, or that you have opinions or think that they are a bad manager. None of which are anywhere near a justification for firing, anyway.

In fact, at most companies, telling your manager that you're unhappy and looking for work elsewhere is more likely to get you rewarded with a promo or bonus.

techdiff 3 years ago | |

Absolutely, I have a friend who forwards all his linkedin approaches to his boss, just to keep them on their toes. Ahaha.

I've been part of teams and seen exit interview data reviewed openly with names, supposedly anonymous feedback twisted to unmask people and D&I surveys where really personal information has been shared in areas it shouldn't be. But that's just where I've been. I think part of what we're doing is a response to that.

The bigger part of it is rapid, in-the-moment responses to issues impacting the team, clients or partner businesses. Because this ideally works weekly, and responses need to be fast and short, it sort of fulfils a different need than a traditional survey. At least, we're seeing different data back with our test group so far anyway.

Operyl 3 years ago |

> and let them know how many people are in the team receiving the question being asked, so they understand their level of safety

How do you prevent attacks that involve the employer having 209 fake respondents, and 1 actual respondent?

techdiff 3 years ago | |

It’s a fair concern that’s certainly on our very near roadmap. We’re adding an MFA confirmation on each phone number which helps validate respondents. Aware that it’s not a bulletproof solution, but it will close the gap another few inches.

Operyl 3 years ago | | |

I guess being even more realistic here: your team is 15 people, your organization is 1000+ people. It's not hard to get a phone number, just a prepaid SIM away. If questions are only going out to your immediate team, how is isolation achieved here? I just don't see how this is a plus to the employee, it can be gamed by the employer, it can be gamed by cliques of people submitting fake feedback about you or others.

throwaway75 3 years ago |

All I see when I navigate to the page is this:

https://imgur.com/a/HmBC9fj

Just a blank page with "No Signals" in the center. Only when I allow stripe.com to load, do I see the webpage. I get it that stripe is required for subscription, but to prevent the site loading if stripe isn't enabled is not exactly nice I think.

Note: jquery.com, jsdeliver.net, and prismic.io allowed to load. licdn.com and stripe.com disallowed.

techdiff 3 years ago | |

Agreed. Thanks for this one! Will look into it.

drewrbaker 3 years ago |

I’d like to chime in and say we’ve been using Signals for probably 4 months now. We have it setup to send every two weeks. My company is 20 people.

It’s been really good to get quick feedback on the happiness of our employees. It’s been interesting the ups and downs that come through. A lesson I’ve learned is you don’t want to be too reactive, as things change quickly depending on the week. But you can pick up trends over a few iterations (for us, it was people wanted more social events organized).

One thing we really wanted to solve was allowing people a chance to break out of management hierarchies and go direct to founders. Even in a small company like ours, we had some middle management that blocked feedback. This has allowed people to bypass that. It’s a modern “my door is always open” system that seems to work so far.

On the anonymity discussed here, I’d say it’s anonymous in terms of the platform hides everything from us. But when you’re small like us, you tend to be able to guess who said what based on tone or writing style. But I think the point about the kind of company that wants to use this is a good one. We are using it because we care about what our employees think, not as a way to trap people and use it against them.

bigmattystyles 3 years ago |

Maybe I'm getting old and/or too sensitive, but I'd get rid of the 'on crack' in the flatter hierarchy section. On a practical reason, not everyone will understand what it means, and on another, yes, I too was a kid of the 90s/00s and used that expression, but now I'm in my 40s and the expression is somewhat loaded.

techdiff 3 years ago | |

This is actually a good call. I'm a similar age and didn't really look at it like that. Will get it changed though. Thank you!

braingenious 3 years ago |

Gotta love an SMS-based offering called Signals, surely this won’t cause any confusion.

>There are two sorts of businesses; The ones that know they need Signals, and the ones that don’t know they need Signals.

Also, that is quite a lofty proclamation!

techdiff 3 years ago | |

Ahaha! That line on the site came from our awesome designer. Definitely a big call.

And yeah, we know the Signals thing might cause some confusion. We're in a different world as a product but it's sure to come up. Funnily enough, when we registered the business we had shortened Signals Intelligence down and Intel's lawyers here got in touch within days to ask us about it. Ahaha.

29athrowaway 3 years ago |

Your peer Mickey Mouse gave you a rating of 1/5 stars. Your rating is now 1/5 stars. Now we can fire you legally without repercusions.

Mickey Mouse's feedback is fully anonymous, and real. Trust me, this is good for you.

techdiff 3 years ago | |

That's a risk for sure - we're based in Australia, you couldn't fire someone in the way you described here, but I understand that's not the case everywhere. It's something we're thinking about.

In all the testing we've done; the feedback is generally honest but professional. The more pointed feedback is often a call for help from the team, and the manager needs help/training to improve. For example, in one test, we uncovered a manager forcing their team to work insane hours, on their days off, etc., without any additional pay or time in lieu - IMO, they deserved to be fired but were just given coaching.

29athrowaway 3 years ago | | |

Sure. But then the business starts taking off, and the MBAs start joining. In no time, they will find a way to maximize profit in a non-sustainable way, and now Mickey Mouse is the top reviewer in Fortune 500s.

joegahona 3 years ago |

> Unfortunately, we only support the US, Canada, Australia and the U.K. at the moment

Hi, techdiff! When you say "U.K.," do you mean strictly the United Kingdom of Great Britain and Northern Ireland, or does this include the Republic of Ireland also? I ask because a large number of our team is in Dublin, and I'm not sure if this would be viable.

techdiff 3 years ago | |

Good question! And something I should explain better. This iteration of it uses the HQ location to generate the sending number. So if you're admin sets up within England, we grab a +44 number to send from. Irelands +353 code should be able to receive a message with some limits on Meteor Ireland network.

If you wanted to test it from Ireland, let me know and we will add support for it.

On the roadmap, we have a task to base the number on the end-user location (based on the dialling code of their phone number), which will also allow us to vary the sending times based on their location. Currently, the survey will deliver based on the HQ location time.

charles_f 3 years ago |

I like the idea of having a steady stream of feedback, Amazon has its "daily connections" that's asks you one question, with a desktop client, that works very well.

However, the general consensus is distrust when it comes to that, especially if things are not going well. You can claim how much you want that it's anonymous, people won't believe you.

techdiff 3 years ago | |

That's totally true. The initial version of this is still running at the company I set it up in. That was years ago. And I have spoken to people in recent weeks who have said they didn't trust it - they were shocked when I talked them through how and why it was. Ahaha. And they felt they'd missed an opportunity to say things.

Convincing people it's a safe space is one of the biggest challenges for us. It's part of the reason we think SMS is great (it's not a company-owned platform), and we make an effort to describe our process for the staff so they can understand it. But it's definitely something we will need to work hard at.

wespiser_2018 3 years ago |

eNPS is a bad metric for a lot of reasons, but in this the concern would be that in low sample sizes, you are unable to quantify uncertainty. The results for small (around 30 but depends on what the mean is) will also be noisy. Don't use NPS unless you have to, ie, you want to release a number for a marketing campaign because your score is a selling point.

"Our future vision is building towards an ideal mix of automated analysis working with machine learning, natural language processing and sentiment analysis, alongside best-in-class consultancy via human-in-the-loop systems when issues or opportunities are identified." - from the blog.

Interesting. I could see this tech being a good on ramp to a consulting practice, but the applications of sentiment analysis are much trickier, since almost anyone writing in a corporate environment is going to have roughly the same detectable sentiment, independent of content. (look at sentiment samples if you need to convince yourself). I'm sure there's signal there, but reliably deriving insights for heterogeneous companies out of anonymous survey data is definitely a tall order. The big consultancy groups in the US certainly are using deep learning to determine topic trends in their client communications. That's a very straightforward thing to measure compared to sentiment, which is trying to measure how people feel based off word choice, so I'd encourage you to focus on that first.

The only other issue see with this, from a survey product perspective, is how you get people to actually complete the survey if it's anonymous. If not enough people complete the survey, it'll skew towards the people most motivated to complete it, but without knowing who completed the survey and who didn't, you can't follow up individually with reminders to ensures surveys get completed so the data is representative. Maybe non-responses aren't actually a big deal, but those response rates (and your incentive schemes) will likely impact any downstream analysis on that data set.

Anyway, fun to think about this stuff. Good luck folks!

zxienin 3 years ago |

This is a very helpful use case, specially for leaders who really want to bypass oddness and get unfiltered feedback. Such a feedback is the only way to stay in reality.

I am not sure why you made it (or started with) SMS. Why not simply use web app?

techdiff 3 years ago | |

Thank you! We tested several ways of sending but the highest response rates and the most candid data came back via SMS.

On response rates, it's around 20% more responses via SMS than email or chat apps. This is the same for marketing delivered via SMS - the percentage is lower but compared to email, etc. it's relatively high.

In terms of the data quality - we think it's because SMS is not a "company owned" platform that could keep logs, etc. Maybe it feels safer for the end user so they're more honest.

Also, the interaction is fast and fun. We get responses initially via emoji and within the text comment you can hashtag and mention other staff. We're working on MMS support so people can send photos too. It's not globally supported but it works in the US at the moment.

abadger9 3 years ago |

the problem I have with this is that I'm a lead in a team of 12 engineers + 1 manager + 2 pm. There's too much bloat and everyone knows it, but the moment I make a succinct argument to the effectiveness of our pm, the director of product will jump in and screw the engineering team. anonymous feedback normally isn't the problem when it matters, it's the structure of the org and powers at play.

techdiff 3 years ago | |

There's so many troubles with org structures and models of operation. My day job is in this area, in an industry that is criminally inefficient. Any field that sells hours isn't incentivised to be quick about anything.

Signals can have an impact if it's run executive level to all staff. It gives people a voice who can see inefficiency but don't have a forum to talk about it, and move past layers that are trying to preserve their roles / status.

eof 3 years ago |

You lost me at SMS.

techdiff 3 years ago | |

Ahaha! Fair call. We're looking to support WhatsApp but SMS is handy because it's not an org owned platform. I've previously been in large corporates where chat logs etc. are maintained and might present a risk to the end user that's beyond our control. Also, response rates are way, way higher through SMS than any other delivery method.

emschwartz 3 years ago | | |

What about a dedicated site or app? For teams that use something like Slack, you could send an automated reminder to click a link and fill out the survey.

(Sorry if I missed that you had already tried or are planning those)

eof 3 years ago | | |

integration with imessage would be the most valuable. where internet is strong, but cell signal is weak, SMS is really brutal. that's my life. any data that can't flow "over the internet" is just ridiculous at this point

langsoul-com 3 years ago |

How do you find the data is used? Do people actually change their behaviour based on feedback?

techdiff 3 years ago | |

Good question! We encourage a weekly cadence - send the survey Friday morning, collect responses over the next few days. The exec team review the comments on Sunday night and present back in the all-hands on the Monday. This is the ideal.

The best companies using it talk to the comment themes (always starting with the most pointed and critical comments) and discuss the details around them and how they're going to remediate the issue, or take advantage of the opportunity.

The worst ones ignore the hard comments. This guts the trust the staff have in the exec team (because the staff WILL talk about what they say in it) and the culture of the company is impacted.

The companies who genuinely buy into this, change their workplaces quite a bit over time. From micro things like the wrong coffee in the kitchen, through to how they're being treated by managers, how the team works together or how they create their product or service.

I come from a professional services background and the people closest to the clients had the best answers to their issues. But due to hierarchy, I often didn't get to hear them. This gave me a direct line.

zxienin 3 years ago |

Is there an open source alternative to this, for personal usage at smaller scale?

techdiff 3 years ago | |

Not that I've found. If you have time, let me know what you're thinking and happy to see if we can help.