Review my startup idea: One API to rule them all.

levicampbell · 2013-01-02T21:20:40.000Z

There are more than 8,000 known web APIs in existence, ofOfering from weather, to financial data, to the ability to search inside libraries. One thing I haven't found, however, is one API that lets the skilled developer manipulate and search multiple APIs at once.A few questions. * I'm currently using a SQL style language to make adding more advanced features easier on my part, but if someone has a better idea, I'm all ears. * If a service requires user authentication (i.e. Twitter), my current plan has users passing in auth data in the body of a POST request, is this secure or is there a better way to do this?If someone has any comments, questions, or ideas, please let me know. Thank you for your time and consideration.

eclipxe 13 years ago |

"One thing I haven't found, however, is one API that lets the skilled developer manipulate and search multiple APIs at once."

There is one, it is called "the internet"

evv 13 years ago | |

Yes, but HTML+JS+CSS has helped turn HTTP into an application environment, and not the content transport protocol it once was.

REST is popular now, but it's not a real protocol. There's SOAP, but nobody really cares or wants to use it anymore.

The web is undoubtedly missing a social content protocol. I predict one will gain popularity in the next couple of years.

EDIT tldr: Until my blog has an open and commonly accepted way to 'friend' or 'follow' your blog, how can we possibly say that we are "done" with online social protocols?

Someone 13 years ago | | |

"There's SOAP, but nobody really cares or wants to use it anymore."

Do you know why? What makes you think we won't end up at

"There's OneAPItoRuleThemAll, but nobody really cares or wants to use it anymore."

or even at

"There's OneAPItoRuleThemAll, but nobody really cares or wants to use it."

If you cannot answer that, I wouldn't bother working on something like this.

Also, who do you think will be interested in having a common protocol? I think the answer will be "those with little data of their own". Big players do not want you to build tools that work together with 'their' data (clear example: Twitter: open while small, but now closing down its API)

evv 13 years ago | | |

> Also, who do you think will be interested in having a common protocol? I think the answer will be "those with little data of their own". Big players do not want you to build tools that work together with 'their' data (clear example: Twitter: open while small, but now closing down its API)

This is some scary logic. By that logic, why would great open protocols like HTTP or email have ever gained popularity over proprietary solutions? Do you really think those two protocols are the only perfect and final solutions which will win?

Couldn't you at least try to have a democratic mindset? This is the web, how do you think Twitter ever came to "own" all that data in the first place?

Someone 13 years ago | | |

HTML? By having many small players, plus by virtue of the US department of Justice.

E-mail? By having many small players, or at least not one player able to call the shots.

Even today, many parties try to make carve out their own communication kingdoms. Examples: BlackBerry, MS Exchange, Apple's iMessage and FaceTime, GMail (doesn't do push email anymore via Exchange emulation)

And yes, HTTP is open, as is TCP/IP. That, IMO, is because of two things: lower layers get commoditized and network effects. Even disregarding the latter, there is too little money to make in such low-level stuff (that also is why Apple builds on BSD and Google on Linux, and both use OpenGL.)

And that democratic mindset? I tried to make an observation, not to tell you what I thought the world should look like.

eclipxe 13 years ago | | |

Hmm. I'm not sure I agree that the web is missing a "social content protocol". What would such a protocol actually do?

REST+HTTP are not without their issues, but I'd argue they've been successful in allowing numerous heterogeneous applications to speak with each other, in a reliable and standard fashion for many years.

Compared to more complicated protocols such as SOAP, COBRA, DCOM, etc...REST+HTTP is as reliable and ubiquitous today because of its simplicity and generic nature. Adding features and utilities for a particular aspect (i.e. "social") would serve to weaken the protocol, not make it better.

The argument that the HTML+JS+CSS side of things has turned HTTP into an application environment is a bad thing is puzzling. Besides, what is the point of a content transport protocol if not to provide valuable services and use cases on top of it? We have TCP and UDP to push bits around...

evv 13 years ago | | |

Don't get me wrong, I LOVE how HTML5/JS/CSS is now a full application environment. We desperately needed networked applications, but now we have them. It is a great way of creating an interface for users.

But we still need application protocols to allow our apps to communicate with each other, and protocols for such have withered as of late. There are a few patterns most APIs need to re-implement:

- Authentication for actions on behalf of users (user XYZ, who I represent, says this..)

- Rich content format

- Content posting (send a link to somebody)

- Simple feedback (like/follow/upvote/downvote)

For these common inter-application actions, HTTP+HTML falls short.

kristjan 13 years ago |

Singly's working on this, especially as it applies to personal and social data: http://singly.com.

[Disclosure: I work at Singly]

mehdim 13 years ago |

Will you do it for non-developers like

http://Zapier.com, http://IFTTT.com, http://rules.io and http://elastic.io, http://ritc.io , http://kynetx.com/

or more focused on developers like :

http://temboo.com, http://singly.com, http://webscript.io, http://cloudsnap.com, http://script.google.com , http://webshell.io

What would be your main paradigm and what do you want to bring to the ecosystem?

Disclosure : I work for webshell.io and lol to see that your post make reactions of lots of API of APIs company (Singly, Cloudsnap, Ritc.io)

If some one in this page wants to reach us for open discussion for making the web programmable in an easier way team[at]webshell.io

kevindykes 13 years ago | |

Hey, Kevin here from rules.io. Thanks for the mention. I'd love to connect with you guys - will send an email.

slajax 13 years ago |

Seems like it might touch on aspects of http://zapier.com which is a bit more of a mashup mentality but definitely allows you to combine multiple APIs to do interesting things.

styts 13 years ago |

Immediately thought of this cartoon: http://xkcd.com/927/ ;) <br/> This could be a relevant resource: http://www.programmableweb.com/

kenneth_reitz 13 years ago |

- http://foauth.org - http://developer.yahoo.com/yql/

aioprisan 13 years ago |

As far as security is concerned, you probably don't want user authentication like a username/password combo to be passed with the requests, but rather acquire an oauth token that expires after a certain period of time and have specific rights within a scope to read-only vs write based on that token.

One question with concurrent API queries is how would you handle the race condition? Say I query 3 APIs at the same time, will I have to wait until the slowest one returns the results to get all results? Will it instead push results to a pre-determined results URL?

What are some cases when you would query multiple APIs and want the results at the same time? Why not just wait for each result, parse it myself, then pass some piece of that computed result to the next query, and so on? What would be amazing would be a universal API processor that can do some of this API to API call parsing and processing in the backend, store all credentials, and just get the final results of a long chain of API calls. Now that's something that would make developers' lives easier and I'd pay money for..

Gobitron 13 years ago | |

We're building something similar to what you're looking for and we're getting ready to launch soon. It's a Rule Engine as a Service (basically a Rule Engine API that connects to Web APIs). I'd love to get your feedback. Let me know if we can connect to chat you can reach me at davejonathangoldberg at gmail.

thekevinjones 13 years ago |

A lot of people are trying to fix this right now. A lot of people think that most API's, even though guided by Oauth or Oauth 2.0, are easy to implement, and it's a huge misconception. So many of the API's have quirks and issues all the time, making it incredibly time consuming to implement, learn, and keep up to date.

Making the process easier is a huge need right now. There are a lot of apps, and it's all becoming more separated. People want to integrate, and it's a pain in the ass to integrate quickly and cleanly.

EmergentOne and Apigee are helping people create standard API's.

Zapier and IFTTT are giving you a UI to meld them together.

Cloudsnap, Singly, and Webshell are all doing what you're describing, each with a different target of apps. Singly and Webshell are seemingly more social, whereas Cloudsnap is more related to business apps.

And of course, YQL, which has been around a long time.

--

Disclosure: I'm a designer for Cloudsnap, and above is my opinion and not necessarily the views of the company.

benologist 13 years ago |

I'm not very familiar with them but check out

https://www.mashape.com/

polemic 13 years ago | |

Yeah, very impressed with their API documentation and example building tools. As an API publisher, it's an excellent product.

zalambar 13 years ago |

What do you imagine this API would look like if it existed?

What use case do you believe it serves (who would use it, what existing behavior would it replace)?

If you're going to call this a "startup" then what is the business plan beyond some set of technical features which may or may not be feasible?

lr 13 years ago |

For the IdM side of things, there is SCIM:

http://www.simplecloud.info/

It started as Simple Cloud Identity Management, but once it got to IETF, it was changed to "System for Cross-domain Identity Management". This initiative has been backed by Google, Ping Identity, Salesforce, Cisco, SailPoint, UnboundID, and others. You can see some of these names on one of the RFCs:

http://tools.ietf.org/html/draft-ietf-scim-core-schema-00

This API is just for identity management, and it has taken years to get to this point. I think it would take many more to get to a common API for everything else.

mehdim 13 years ago | |

This is not always the "standard" which wins. You may have also de-facto standards like MS-DOS in 1980' which was an "shell to rule them all" and which has made Microsoft a multi-billionar company and has been more used than UNIX which was better and open!

Either is the standard, either it the the most adapted product/company (not necesseraly technologically) which wins the pot. This is technology AND business model, user community, strategic partnerships which will make an API to rule them all a defacto standard which will lead the programmable web. But you're right that IETF standards for APIs may be so long that I consider it is not possible untill decades !(or a leader which will open and free its winning standart to all only)

misframer 13 years ago |

Reminds me of YQL by Yahoo.

thingylab 13 years ago |

Do you have any use cases in mind ?

It seems to me that you would mostly act as a proxy between a user and a service, which really doesn't sound too useful unless one actually needs to access dozens of APIs (which doesn't seem to be something that happens often).

Or is your idea really an aggregator (e.g. "find images using such and such keywords" and it returns results from flickr, 500px, ...) ?

rckrd 13 years ago |

Sure, there are over 8,000 web APIs, but how many will be reasonably be within the scope of your project? How many times do we need a weather app that also takes photos, tweets, and gives stock prices?

Even though this seems to try to solve that problem, I think it obfuscates whats usable more than the internet does in simply just searching for the API.

web007 13 years ago |

Sounds like Yahoo Pipes, or YQL (as already mentioned).

Auth should be linked separately from the individual requests if you can. It would be a lot better to auth against $yourService, and $yourService has the cached / OAuth'd credentials for $otherService and handles the auth dance / request signing / whatnot.

mehdim 13 years ago |

You say 8000 APIs, but for example on http://apihub.com , you have more than 13000 APIs registered. And OpenAPIs are only the emerged part of the iceberg. Soooooooo many services ran by private APIs in big companies...more than you expect.

mtraven 13 years ago |

See SPARQL and the semantic web, though I think the xkcd cartoon is the better response.

mehdim 13 years ago |

For better understanding why we need an "API to rule them all" you can check the http://apijoy.tumblr.com about everyday's developer pain with APIs.

roycehaynes 13 years ago |

You should take a look at Zapier. They do most of what your getting at.

ville 13 years ago |

Have you looked at Yahoo! Query Language? http://developer.yahoo.com/yql/console/

kevindykes 13 years ago |

the team behind rules.io are leading an open source API framework project called Geekier which is exactly this. Take a look at the details here in Github https://github.com/rulesio/geekier/wiki - there is a link to the Google group for the project as well.

arthurlenoir 13 years ago |

Look at http://webshell.io I know they are hiring! ;-)

mmanfrin 13 years ago |

Like Zapier?

plaguuuuuu 13 years ago |

In what way does this simplify joining data from multiple APIs?

escaped_reddit 13 years ago |

a certain xkcd comes to mind.