My problem: I can't keep focus on a few selected areas. Part of the problem is that I want to learn too many things simultaneously. But another, more serious problem is that I can't keep the same goals on my "radar" most of the time.
My interests are located mostly in several domains (software development, history of technology, politics, cognitive flaws of human mind, workings and failures inherent in complex organizations, economics). But within those, I can't keep focus especially in things I know I should master but somehow can't keep this for extended period of time: databases make me yawn (apart from NoSQL ones), Java is boring to me and even though I know I should learn those I can't keep working on it up. About the only subject I have maintained steady interest in and regular work on is Python programming language.
With other things, even if I'm interested enough in them, I can't bring them to completion because there's another hundred of things I start working on with energy, but soon something comes up that is a blocker or a problem and I switch to that and do not come back to original domain (it's often justified, like program or language X hiccups bc of say os kernel issues A and B so I need to resolve that too, but I have trouble even remembering that I should resume X).
How to keep working on things I know I should be working on without losing focus (and it's not like I'm completely uninterested in those other things)?
The problem is not that I'm bored in new subjects: on the contrary, I keep interest in old ones so much I have trouble selecting those I should work on and so I jump randomly between them! In paradoxical way, losing interest actually helps in focus: you lose interest in most stuff and work only on 1 or a few selected issues.
I found money can't motivate me to work on boring, tedious or futile problem. My trouble arises partly from thinking too far ahead. Example: at this meeting we're trying to shoehorn our growing model into SQL db schema. At the back of my head I have "this would be so neat to model using graph database! it would work so much better!", but that's not a solution we can choose (legacy, team skills), so I sigh and keep working on this. But this thought at the back of my head can't leave me so off I go and read on graph dbs for a few hrs. in the meantime, my deadline is closing by those several hrs. Aargh! And again, I need to get on with this SQL but this shoehorning is so silly I can't even motivate myself to do it, so I'm struggling to get anything done. Another task is say, data conversion. What would be the best tool? Ruby reportedly has some neat tricks that could help. So off I go and learn a bit of Ruby for a day. But I have to stop bc of deadline, or some other problem needs solving and off I go and do not get back to Ruby even though strategically speaking I'd want to!
Iterate this dozens of times and you get the idea about final result: I'm not very good at any subject, not SQL nor graph dbs nor Ruby nor hundreds of other subjects I dived into but did not master due to simple lack of time. I'm spending my scarce time budget way too thinly. I feel like informational hamster, biting way too much to chew something effectively. And it's not caused by lack of interest, on the contrary, by being interested in too many subjects at once. In kanban you have an idea of self-imposed WIP limit (Work In Progress): I can't keep myself from violating my personal WIP limit all the time.
Another example how long-term thinking can harm you: I never bothered learning how to optimize SQL queries, for a good reason: with volumes of data as big as they are now and data so diverse and complex it does not make much sense to heap everything on a single server, for reasons of both modelling flexibility and db scalability. Sharding! Great, but after thorough evaluation of replication solutions for certain db I concluded they all suck. So I rolled out my very own solution, using RabbitMQ as middleware for sharding + replication (data safety was important consideration). It works great, except I still don't know SQL optimisation.. Another day: web system with lotsa queries. A nifty ORM to the rescue! It paid off in the long run, shaving down the time of development for a single query to minimum. Great, except at last stage of job interview CTO of large and interesting (specialist niche) company asked me just such questions (optimisation). Oh s**. I failed to get that job, even though all the previous interviewers were reportedly very happy with me.
Another guy in my place would never bother to look at all those interesting and productive tools, he'd just learn the damn plain vanilla O*a*c*l* SQL and get that job.
My problem is partially that I work in a truly big corp. People who think that issues are analyzed here in depth bc resources are available, are very much mistaken: the analysis is surprisingly short and shallow, modelling is ad hoc, planning is astoundingly primitive, and product design runs on shallow intuitions instead of being based on hard data, testing and knowing customer very well. As result, products are garbage (and don't tell me customers like them, they don't, they just put up with all the pain of using them, I'm ashamed to even be associated with those). This is also heavily demotivating for me. Paul Krugman was reportedly dumbstruck when working at US Treasury how incredibly primitive and inaccurate econ analysis was there (they did not even bother to account for inflation in analysis, for instance). I'm no Krugman of software but even I find this self-imposed lobotomization utterly demotivating. That contributes to lack of focus..
The other day I've done some work for a tiny startup. I felt in my element! In 1 hr we were covering ten times as much ground re analysis, planning, innovation, design and considering different scenarios as we are in Big Corp. It was like time machine! Guys who work there (as employers, not founders) are viciously smart and they could make literally 3-4 times as much money in a corp, but they prefer to work in that startup bc they can program in their favorite functional language and MongoDB (try to propose that in a corp, that would be like telling a nun she could be a temp stripper from time to time) instead of being bored out of their skulls by Java. But that's not an option for me, sadly: I'm too old and with child and wife to support I can't risk jumping ship..