Prioritization is hard because: (1) Relationship between time spent on a task and impact is non-linear, full of step jumps, and hard to predict (2) Switching between projects has non-negligible cognitive costs on your productivity (3) Every hour of your time isn’t equally productive, so prioritization is about managing your time AND your energy But we can get better at it: (a) Think through the impact ceilings for each project on your roadmap. What would change about the product/company/strategy if this project was widely successful? Leverage your peers, XFN partners, and manager as sounding boards. (b) Try to break ambiguous, hard-to-predict projects into smaller more actionable chunks. Be ruthless about how you manage your time and your energy. Learn how and when you work best and schedule your highest impact projects around that.
A framework for thinking about prioritization
This is a note about how I think about prioritization of individual time and why it’s a hard problem to solve. There are no magical formulas, but there are frameworks we can use to get better at it. I do think that some of what follows extends to prioritization of other non-individual resources (eg. staffing, product/feature complexity), but not necessarily all of it.
Prioritization = optimizing the impact output of each marginal minute spent working
For every project x, assume that impact(x) = f(time, other stuff) where “other stuff” is skill, tool access, etc, every other non-time resource you can apply to get project x done. For the purpose of this note, we are just going to focus on time. Usually the relationship between time and impact for a given project is one that hits decreasing marginal returns at some point. After a certain amount of time has been invested, investing an additional unit of time will lead to less incremental impact than previous time invested. This is the 80/20 rule restated. Ideally, this is what we would like to see a smooth relationship like this:
A dream world where you are working on only one thing
If you only have one project, all you have to do is pick a point in that line where you stop investing time. You probably want to stay on the left-hand side of the curve as much as you can (ie. putting 20% of the time to get 80% of the impact). And usually your time investment will have an externally imposed ceiling (i.e. a project deadline, PSC timelines). But in reality, we are more likely to be juggling multiple projects at once:
A bit more realistic
Taken to the extreme, prioritization is essentially allocating the next day or hour of work to the project with the most potential marginal impact. What project would yield the highest impact if I were to spend an additional hour on it?
Orange dots represent where you stand in terms of time investment and realized impact for each project
On a particular day, this is what your graph might look like. The orange dots represent where you stand on each project’s curve. In this case, Task C has the highest marginal impact and you should spend your day working on it and ignore Task A and Task B.
So far, so good, right? If this was the case, a golden algorithm for prioritization would be:
- Look at your entire backlog of work.
- Find the project with the highest marginal impact as a function of time.
- Work on that project until you sense that marginal impact is decreasing.
- Stop; go to the microkitchen; have tea. Repeat from 1.
Well, if you’ve been alive for more than 15 seconds, you know that this framework fails miserably. Here’s why:
a/ The relationship between time and impact isn’t a nice concave continuous function. Oftentimes it’s a weird, jagged, non-linear, discontinuous step-function. The 80/20 rule looks continuous in the long-run and over large enough samples of people, but for any individual project 80% of the impact can come from 20% of the time invested, just not the first 20%.
A little closer to reality
All of us probably have experienced this before. When you are working on a project for hours at length, making little-to-no progress and then suddenly an eureka! moment and you make a breakthrough. You finally fix that pesky bug, a difficult relationship suddenly improves, or a new roadmap-changing insight is uncovered.
b/ It’s difficult to know when you are close to a jump in the step-function. When you are deep in the weeds of a project, it’s tough to see the bigger picture and identify whether you are close to a major breakthrough. Not knowing the shape of the impact curve before starting a project often leads folks to overprioritize low impact/low risk projects for which the impact curve is more certain. This is widespread in data science and usually manifests itself as DS spending a disproportionate amount of time on operational work rather than bigger, riskier strategic projects.
c/ Switching between projects has non-negligible costs. Context switching can lead to lower productivity as your mind wastes time and energy ramping up on each project. This complicates step 4 in our golden algorithm. It’s not always net-positive on impact to switch projects when you sense decreasing marginal returns because the cognitive cost of context-switching can distort the shape of the curves.
d/ Every hour of your time isn’t equally resourceful. Energy management is a huge part of this puzzle. Your productivity is probably highest at certain times of the day or on certain days of the week. This further distorts the impact vs time curves for each task (and it likely affects different types of tasks in different magnitudes). For example, I’m most energetic for deep solo work in the morning but much more energetic for project meetings and 1:1s in the afternoon. Energy is often also non-linear — events throughout the day might affect your energy unexpectedly — adding further complexity to our optimization problem.
An applied example:
Data scientists usually complain that they don’t have enough uninterrupted time to do deep, complex, strategic analyses of a product ecosystem, Too many meetings, ad hoc requests, and operational work getting in their way. Product managers usually hit back with “well, can’t you just do 20% of the work to get 80% of the result? Do you really need 20 hours for that analysis? Couldn’t we get something impactful in 4 hours?”.
Well, really what the data scientist means is: I need 20 hours because: a) I’ll spend the majority of that time trying things that won’t produce impact, but in the midst of that I might hit gold. b) I just don’t know if that will happen in the first 5 hours or in the first 20 hours. c) Uninterrupted time will make me more productive for this particular task by avoiding context switching. d) And will allow me to manage my energy properly to tackle it at my highest productivity level.
How to solve for this?
Find impact ceilings for proposed projects For data science, this usually takes the form of asking yourself the following questions:
- What specific changes would this work bring about in the best case scenario (perfect success)?
- What product or strategy recommendations would come out of this?
This still doesn’t reveal the shape of the curve for any given project, but it should help you estimate the overall impact ceiling — the highest point in the curve. It goes a long way in helping you move away from overprioritizing low impact/low risk projects.
Break large ambiguous projects into smaller, more actionable chunks After you understand the impact ceiling, try to break down the task into smaller parts and find checkpoints along the way to validate the path you are going down.
This is essentially turning Task A in the graph above into something like Task B. Break large monolithic projects into smaller hypotheses that can help you approximate the shape of the impact curve. This is how we build products — what’s the MVP and how will you validate that the path you are going down is good and worth continued investment? For data science, this could take the form of:
- Feedback from other DS — if my hypotheses resonate with other data scientists, it probably makes sense to dig deeper
- Analytical validation — if my first take of this analysis finds X instead of Z, it probably makes sense to dig deeper
- XFN validation — if my initial findings resonate with PM, it probably makes sense to dig deeper
Build a prioritization process with your XFN partners Teams are how we get things done and your XFN partners are usually your most important stakeholders. A few ways to do this:
- Build a roadmap for yourself — what projects are you considering tackling this month/quarter/half? Get feedback from XFN early. This will help you identify the macro areas where you should be spending your time.
- Set up recurring biweekly meetings with your XFN partners to catch up on prioritization. Plans change over time so static roadmaps aren’t a foolproof solution. Set expectations that new requests or projects should be introduced in these meetings. This reduces thrash and helps you deflect low ROI work more effectively.
Focus blocks: Book time for deep, concentrated work on your calendar This works best if your entire team buys into it and decides to implement it across functions. If your team can’t agree to it, do it yourself and set expectations with XFN peers accordingly. Tips on finding the best time for this:
- Manage to your energy. Do you do your best work in the morning? Then schedule your focus blocks in the morning.
- If the space around your desk is hectic, go somewhere else for this time. Find a quieter space on campus (or a louder space if it that’s what makes you tick). Work from home for part of the day if your commuting infrastructure allows.
- Folks still interrupting you? Set expectations. Auto-replies on Workchat, a signaling system at your desk (eg. headphones in = focus mode).
- Be ruthless about how this time and use it to drive forward your most impactful projects.
Avoid calendar fragmentation: Keep meetings bundled together
TL;DR: This minimizes context switching and, like focus blocks, gives you time to focus on deep, concentrated work.
Know thyself: Understand how your energy changes throughout the week Observe yourself. Keep a journal.
How do you handle prioritization? What has (and hasn’t) worked for you?