Show HN: I built testelf - app testing service for your iOS app

Show HN: I built testelf - app testing service for your iOS app(testelf.com)

77 points by ssong 13 years ago | 28 comments

ssong 13 years ago |

OP here. I quit my full-time job almost a year ago to travel and work on startup ideas, and this is the fruit of my labor in the past six months that I hope you'll find useful. I wanted to make it easy for startups and indie devs to test their iOS applications. How does testelf work? You upload a build of your app without providing any additional instructions, and we'll test your app from a user's perspective, logging what we did or saw. The goal is to charge a flat price per test, provide good coverage and actionable feedback, and have a 36-hour turnaround time.

Over the past few months, I recruited and trained a small team of testers, developed internal testing tools, and worked with a handful of startups who are piloting our service. The service is now open for beta -- I would love to hear feedback from hackers and startups on HN who can give it a shot.

matthuggins 13 years ago | |

"You upload a build of your app without providing any additional instructions, and we'll test your app from a user's perspective..."

It seems like a user should have at least some idea of what the app is. For example, I'll read the first couple sentences of an app's description and look at some screenshots/video before I decide to download it. This might be something you want to consider instead of having absolutely no idea what the app is before running it, since I assume most users have at least some context when opening an app.

Cushman 13 years ago | | |

It's usually a good idea when user testing to tell testers as little as possible about the product. It's true that real-world users have more context than that, but it can be worth it to get more unfiltered feedback— the first few moments you watch a user flounder to even understand what they're looking at can be the most valuable.

ssong 13 years ago | | |

Good point! There is an option to attach additional information (description, notes, link to website/app store) when uploading an individual build. We do ask for a little bit of information about the app in general when the first build is uploaded so that we have some context.

jmatthew3 13 years ago | |

This is a pretty cool idea, and something I would have considered trying to get my boss to pay for when I was a developer. I didn't see on your website - do you test on a wide variety of hardware platforms, or just iPhone 5 and iPad? Different versions of iOS?

Also, do you have any terms, in particular, regarding confidentiality?

ssong 13 years ago | | |

Thanks for asking! We do test on common devices (retina/non-retina iPad, iOS 5/6), and we're going to carefully consider which devices to cover as we grow. I am also very aware of the importance of confidentiality, such as keeping test results private -- I think the service wouldn't make sense without it. Our exact terms will be available before we publicly launch.

kevining 13 years ago | |

Brilliant idea for a business. I would love if you guys would do simple games some day. Hoping to give your service a try soon.

umut 13 years ago |

Congrats, looks really nice! (already forwarded to our testing team)

If you add a 'proxy layer' to your solution, that would be awesome. To explain the scenario, it is really very valuable for us app startups to see how it behaves in China, in India, in Europe, in US etc. You may keep your current system intact, and 'if' the customer desires so, you can route your connection over those proxies(from real ISS'es) from all over the world and test it 'as if...' This is really important and not so hard to implement... But sometimes, you can catch bugs and usability horrors with slow or dropped connection or some weird proxy mambo jambo played by ISS providers.

cstrat 13 years ago | |

I think this is a great idea too! It would be an easy upsell to international developers.

thomseddon 13 years ago |

Looks excellently useful, good job. I must say the graph on the test page (http://beta.testelf.com/builds/share/0b78d05c4f5d11e2ab5a220...) is particularly wonderful, great use of D3.

Also, would be very interested to know if you have plans as to how (or if) you will overcome the issues you would have if you wanted to scale such a bespoke service?

timdorr 13 years ago |

Any reason this is limited to iOS? It seems like it could extend to Android/Windows/etc. extremely easily.

ssong 13 years ago | |

Thanks for the comment! Android is probably the next platform for us if all goes well. I started off with iOS because I'm more familiar with it and it has less device variations. Extending the offering won't be trivial though. Before we start support for a new platform, we'd need to build the testing tools, develop new training materials, and acquire new hardware.

gmig 13 years ago |

Why is "No provisioning necessary"?

akaru 13 years ago | |

This is a good question.

speeder 13 years ago |

You wrote that you don't test games...

Well, I hope we can send our more non-gameish apps :)

king_magic 13 years ago |

This looks great, and I think it's a really good price point. $200 is quite reasonable, IMO, for the kind of detail you guys are getting into.

Definitely going to consider this for future testing :)

swanify 13 years ago |

Looks great, i run a mobile app agency and we are forever having issues due to developers not testing adequately, having some external company test would be really useful!

ianstallings 13 years ago |

This is a great idea sir. I signed up and hope to hear from you guys. I can use more eyeballs looking over our work. I really like not having to worry about provisions.

PanMan 13 years ago |

How do you handle UUID signing? Do you tell me the UUID's I need to add? Just upload seems to not work with iOS apps, right?

pooriaazimi 13 years ago | |

You can sign ipa files with your developer (private) key. There's an app called InstaSign that I use for "testing" apps before purchasing them. You download a cracked version, sign it with your own key (so iPad thinks YOU have created this app, like when you do in Xcode) and see if it's worthwhile.

It's saved me a lot of money that otherwise would've wasted on scammy, buggy, "not-working-as-advertised" apps. If you're a developer (i.e., can sign apps and load them unto your device), give InstaSign a try.

They're probably using a similar method (as they seemingly test on iPad/iPhone, retina/non-retina, iOS5/6 devices and it might be a bit more work for you to add at least 6 new UDIDs).

akaru 13 years ago | | |

I was wondering how you did that. How do we know you're not simply cracking these apps to resell? It sounds a bit fishy is all.

tachim 13 years ago |

Awesome! This looks super useful, I'll definitely give it a shot with my own apps.

thelarry 13 years ago |

Pretty cool, I have a rough time getting my lazy friends to test my iphone apps :/

jplehmann 13 years ago |

Good. I'm tired of monkeys. We need something more intelligent.

eclipxe 13 years ago |

This is very cool. Simple but useful. Great work!

hariseshadri 13 years ago |

This looks fantastic. Great work.

shawiz 13 years ago |

Congrats! Very useful service.