Joan Didion, "On Keeping a Notebook"

Apologies for the long quote but this is pretty much how I feel about all attempts at "personal analytics" so far. As other posters have mentioned, analytics are great for businesses who have their FOMs and TPMs and KPPs and whatever to maximize already, but on a personal level the best use of diaries and journals is to force yourself to realize what your goals are, not to help achieve them.

Its exactly focused on insight and de-cluttering the mind and finding focus. Being happier and more creative.

It really works and lead me to significant periods of peace and contentment and working on meaningful things.

In the last year I did a lot of work to turn it into a proper business, but got caught up in another startup which took up all my time.

I am planning on making it into a product with great design.

That's really the primary work now: crafting a design that is communicates the data in a clear way.

Some messy bells and whistles thing with all the data and graphs and features.... nope. Not interested. I want clarity.

I use DidSum to track running, blogging, eating healthy, sex, coffee consumption, and more.

DidSum allows you to define the actions you want to improve at. You can track publicly with friends and family and encourage/compare/compete with each other. You can also track privately (some actions I only track with my wife). It has basic analytics, which will get more powerful with time.

We're in a closed beta right now, if you're interested in helping test I'd love anyone interested in this thread's feedback: http://didsum.com/auth/register.html

You need to separate the signal from the noise, and ramping up the noise isn't the solution. Quality takes precedence over quantity. Benjamin Franklin identified thirteen 'virtues', and those signals seemed to work for him.

> I propose a simple data entry framework: click a button, and that event gets tracked.

The problem with data entry is that it requires the motivation and initiative of the user, which is precisely what the user lacks. That's the reason people get personal trainers and life coaches: you need someone else to keep you accountable and motivated.

> Machine Learning Matters

Here's where I think almost all of these personal-analytic type applications go wrong: it's trying to solve a very subjective human problem with a very objective machine program.

Here's an alternative startup idea: a web application for life coaches and their clients, focused on providing them the tools they need to communicate better with one another. I'm betting that life coaches use email, text messages, phone calls, a calendar, and maybe a spreadsheet. They probably could use some help integrating all those tools and all the data they are collecting.

Everyone wants to improve something (or everything!) about themselves. Most people have a sense of what that is, and what they should be doing. Stop smoking, go to the gym, get more sleep, eat better, spend less money, make more money, etc. A personal analytics system may help someone optimize their life from seeing the hidden details, but I claim that the steps to self-improvement for most people are fairly obvious. The problem in self-improvement is not lack of data, it's how to execute on the obvious known fixes (how to stop smoking, etc).

friends <- [user.alice, user.bob, user.carol] # array of friends friends_movies <- apply(friends, movies)

When a friend updates its movies everything is recalculated like in a spreadsheet. The core idea behind this is to end with emergent behaviour from the small scripts of millions of people.

To simplify the programming language everything can be written as an s-expression. I've published the parser (using o-meta) with an article here: http://blog.databigbang.com/parsing-s-expressions-in-c-using... more food for thought in:

i) Egont, A Web Orchestration Language: http://blog.databigbang.com/ideas-egont-a-web-orchestration-...

ii) Egont Part II: http://blog.databigbang.com/egont-part-ii/

For the moment we're starting small and just trying to pull in activity tracker data (Fitbit, Jawbone, Withings etc) and some secondary services that provide useful quantifiable data - things like Foursquare checkins, Gmail, task management tools, productivity trackers, etc. It's hard to interface with every service when you're writing custom API clients for each of them. And on the other hand, if you try to dictate a common standard without any clout to back it up, you just end up with http://xkcd.com/927/. We're working on an API to export all your data too, so people can build visualisations and apps on the aggregates of their data.

Tracking things has to be as frictionless as possible, so I'm wary of manual tracking unless you're already quite committed, or it takes mere seconds - the interface in #2 looks like a good way to approach this. But we're avoiding this for now.

The other thing we strongly agree on is #4 - you need insights, not just data! Numbers alone are of limited value, and a lot of people already into Quantified Self end up with "number fatigue" where they're no longer motivated to put in the effort to track things, let alone work towards a goal, just so they can see the number go up. People need to qualify their data and make it part of a narrative about their life. They also need actionable feedback, which is where we're trying to focus on providing value - if you're trying to be more productive, on which days is that working, and what are the factors behind it? We'll try and tell you so you can focus on that. Same process for becoming more active or anything else.

It's a big problem to solve, but so many possibilities too.

Other than that, I totally agree. Someone please make this app-- then let me do the machine learning for you. :)

Yes Fitbit is a great pedometer that gets people to walk more but last time I checked they weren't recording that much data and linking to connected scales is useful but its barely scratching the surface.

One issue with Personal Analytics is we need more data and it needs to be EFFORTLESS to record. The ROI is only high enough when the effort is very low. This is a challenging problem that requires "true innovation". I don't think a tracking app on my mobile phone is going to be the answer.

Another issue is the ability to improve some facet of my life with this data(weight, sleep, etc). Even if I can correlate drinking coffee in the evening is causing me sleep problems unless I can change my habits its not useful information. We need to be able to change our habits and processes based on this data. There are scientific strategies to get people to change habits and simply telling them they need to walk more or eat less do not pass scientific muster. Trigger -> Habit -> Reward. If we need to change Habits we must replace them something. If I need to stop drinking coffee then I need to replace it with something.

If someone can create a system that nearly automates the tracking of what you eat you'll make gazillions and help a lot of people along the way. Maybe a bluetooth food scale in conjunction with a tablet that can do "facial recognition" on a plate of food to estimate what I'm eating and the carb/protein/fat/calorie breakdown. Then based on my physical activity for the day and calorie consumption it could recommend I skip desert or have a carrot instead.

If a web platform is developed to provide storage and an API, the only way I would trust it is if it were completely open source and users were encouraged to run their own nodes (as well as supporting easy data migration between nodes).

I've written a response, but I suppose I am hellbanned for new posts. No clue why unfortunately.

The basic idea: prompting is of great importance in this field. You can't get GREAT data through passive measures, but you can get much more consistent data if you consistently prompt. (This is my personal method, at least.)

[edit] It's not so much of a response as it is related - a way to capture data about self. Not so much related to the startup viability discussed in OP's article.

And therein lies a potential headache. What if the API for those 100 services break? What if just 10 of them change their API every month? Are you going to update them? And oh btw, you're gonna keep doing this before you actually have paying customers.

I want to track how much spend, consume electricity or gas, what are my goals.

The problem I found that I forget to enter the data into the app so it is fairly sparse. I can only get meaningful data on a monthly base.

I imagined it would be capable to compare gas (heating) consumption with the weather on the same day but it is not really working (without some automatic data input, that I don't have).

Among others I though about adding mood tracking, attaching 1-3 picture/day so when you want to watch back on your vacation you are not overwhelmed with 5000 photos, tracking locations. I would like to think about it like a social network where I am alone or something like this. (Codename: Track My Sheet :-) )

Showing trends, adding watchers like "you now need 1 year 3 months on your Y Account to reach Z amount at your current rate +Q USD / day, but these kind of things are faster with an excel sheet. Here is a dashboard mockup: http://snag.gy/OozTi.jpg

It is similar to the reportr.io app, actually I started mine on the same week before I first saw it on HN. :-)

Honestly I think this is too much data to make use of. I'm just one person. How much meaning can you get out of what I do? If it is a lot, do I really want that platform to exist on the web?

If the data entry is quick and easy, then I'm not gathering empirical usable data. If I try to gather good data the user will become annoyed.

I think data gathering apps work for a small scope when the user is engaged enough to provide the data. Fitness or body building for example. Or sports car races etc.

This was making me think that maybe a generic statistics app with user generated values/measures/timers might be interesting. But once again, non technical folks probably aren't interested in statics too much.

Maybe gamification and a great UI is the answer here or maybe one should start with a specific problem (workout, drinking, sex or whatever) and broaden the scope as he analyzes user interaction.

I would give such an app a shot, but I'm not sure if I wouldn't forget about it too quickly. Just as I forget about using Lift (the app) and other interesting stuff.

Completely agree that "graphs without the ability to manipulate data are worse than nothing"--it's hard to gain much insight without being able to drill down and filter the data. Simple questions such as "how strong is the correlation between room temperature and sleep" end up having to be refined e.g. with "ignore weekends" and "correct for different amounts of physical activity".

Being able to "get as much data as possible" sounds good, but garbage in, garbage out applies...

Finally, I don't think it's possible to create a single mobile app that suits everyone, which is why Zenobase has a flexible, generic data model and an API.

http://blog.stephenwolfram.com/2012/03/the-personal-analytic...

The biggest challenge imo is privacy and security: How do you safely collect and process everything (from emails to texts)? While I could implement something for my personal use, would anyone give access to all their communications to a third-party service? I wouldn't.

Maybe there is some better way to collect this information (just metadata? do some analytics and store results instead of raw messages?).

http://activitystrea.ms/ is a good first step in opening some of this stuff up for consumption.

The way it is now, each front-end service has to do a ton of work for each data source they want to integrate with. There should be services that translate data into a shared format. And then front-end services can focus on their idea instead of having to parse all the data.

Where do you get your revenue for personal analytics like this? Therein comes super targeted ads with pinpoint accuracy, using predictive analytics at a personal level. If you buy a brand of milk only on Saturdays, rivals have a higher incentive to strut their wares on next Saturday.

I know some people are trying food tracking via taking a photo. Taking a photo per meal is a lot of overhead! That said, 4 Hour Body recommends it to stick to a nutrition goal.

Also, I disagree with your premise. I know what I shouldn't eat, and the main questions I want answered are what correlates to my making bad decisions. How does sleep relate? Alcohol? Being around friends? Tracking specifically what I eat seems irrelevant.

Basis although is lagging in the software department. Their mobile apps are not very useful and their API is not open so people can't grab their data without doing unofficial hacks that accesses the web apps json data directly.

I think although people are going to be using something like the iPhone 5S's M7 motion tracker more than something like a fitbit, because they carry their phones everywhere and don't have to buy or manage a separate device.

This would be the best current combo if basis opened it's data silo:

1. A basis watch for sleep and general activity.

2. A wifi weight scale for weight & easy to measure BF%

3. Runkeeper, etc to track your cardio activities. Basis & fitbit doesn't track actual fitness activities (like biking or hiking) too well compared to runkeeper.

4. Also lets you manually input personal measurements such as your navel, hips, waist, etc from the mobile app. Fitbit does not do this.

All of these items in one cloud-managed website & mobile app will be pretty powerful. The closest you have for this is fitbit right now.

In the future, a credit card / payment system that automatically tracks nutrition purchasing info will be very powerful. This needs to be combined with more nutrition estimates in restaurants although.

Passive 24/7 location tracking can also help co-relate events. It can track the effects of vacations, ski trips, trips to the pool, more grocery shopping less restaurants, etc to put more meaning into your graphs. It can co-relate you going to the gym frequently or not going to burger king anymore with you losing weight.

I think just awareness that your slowly gaining weight can help snip weight gain trends in the bud, along with warnings your eating a lot of high calorie crap that you might not be aware just how high calorie it actually is.

But, with the new wave of smart watches, glasses and other smart technology there is an opportunity for that data to be easily imported into a personal analytics application.