Five years ago we began an ambitious software project to create the de facto system of record for investment research at hedge funds. It’s impossible to take on a task of that nature without some sort of project management methodology. And these days, given the choice, most teams would favour a flexible Agile approach over a more rigorous one like Waterfall.
Since its inception in 2001 Agile has come to encompass a set of frameworks including SCRUM, XP and Kanban which mould the principles of the original Agile manifesto into an applicable concept. These approaches are popular because they give teams something to adhere to; a set of rules which promote efficiency and regular delivery, all the while keeping the customer happy and the team motivated and fresh. Who wouldn’t want that?
As with anything though, these frameworks demand not just a fair amount of discipline from everyone in the team to employ and maintain, but also the intelligence and confidence to depart from the rulebook when certain aspects just don’t work.
So how have we made Agile work for us?
The Good Old Days
Back in 2012 we were a fresh startup with hundreds of things to think about. Wanting to get going as quickly as possible we set up JIRA, loaded it with our fledgling roadmap, and eagerly started writing code. In those days we had a handful of employees and everyone had a rough area of responsibility; sales, app development, test infrastructure, and so on.
In this early configuration our communication was focussed and often involved everyone in the company. Assigning work was an almost trivial, obvious affair, as tasks fell naturally to the person or persons qualified to do them. With few clients we weren’t inundated with support or feature requests, and could largely dictate our roadmap and our path through it.
In this sort of situation – experimenting with a product, trying to pinpoint a market fit – we found that the tenets of Agile development fit so naturally that we barely noticed we were following them.
We had to constantly communicate because our fledgling tasks were incredibly entwined. We had to prioritise the development of our software over its documentation because the rate of change was such that most docs were quickly rendered irrelevant. And we had to release early, and often, to interest companies with whom we could then collaborate to better understand the hedge fund domain, and shape our product to it.
These are a few broad examples but one could find many more instances of Agile practices cropping up in our approach during the early days Bipsync, even though we wouldn’t have considered ourselves to have been following an explicit Agile methodology. An approximation of our process would have looked like this:
However, as time progressed it became clear that this serendipitous way of working was not sustainable. As the company tripled in size, communication became more challenging and development more arduous; it was clear that a more defined strategy was in order.
Flirting With SCRUM
While never consciously following the SCRUM methodology, our approach shared many of its features; we organised our releases into two weeks cycles in the manner of sprints, conducted retrospective meetings, and were largely driven by our CEO’s vision of the product, making him a de facto product owner.
In an effort to better organise ourselves we took a step closer to SCRUM by introducing “sprints” in which we strictly planned our near-term work.
The theory was that this would improve our workflow by allowing us to identify dependencies between tasks ahead of time, and therefore schedule who would work on what to ensure a given feature was ready for the release at the end of the sprint. It would also allow us to better communicate our roadmap to our clients since we could extrapolate where we expected to be n sprints from a given date.
In practice, it didn’t pan out that way. As we took on more clients, each with a handful of feature requests and the occasional support item, we found ourselves prioritising these incoming tasks over our planned ones as they were often judged to represent greater value to the business.
We realised that while this imbalance was justified and maintainable in the short term, in the long term it would impact on our vision to build a focused enterprise solution that was consumer grade to use and fairly generic, yet inclusive to all approaches to research management.
As tasks were pushed back until they fell out of the scope of the sprint, the concept of the sprint itself became meaningless and eventually we considered it an unnecessary inconvenience. Framing our work in two-week cycles exposed bottlenecks within the company. Sometimes we’d see development tasks remain untouched at the end of the sprint because they were blocked by UX considerations, or required assets from our design team. Developers were often assigned tasks that they were infeasibly able to complete, which just served to distract.
We were also mindful of one of our core business values which is to avoid unnecessary complexity in everything we do, from the code we write to the structure and organisation of the business itself. Introducing stricter rules around who would work on what, and for how long, felt like a step away from that ethos.
Embracing Kanban
Recognising that we needed a more flexible, dynamic way to schedule our work we decided to instead align ourselves closer to another popular Agile framework: Kanban.
Kanban is more aligned with the Lean methodology, which aims to improve efficiency in any process, rather than being derived from the tenets of Agile. It allows us to keep our concept of “near term” prioritised work separate from our “long term” backlog, but advocates working within the team’s capacity, only taking tasks on as they have the ability to work on them. It’s suited to a fast-moving start-up environment in which there are many tasks whose priorities change frequently, which we often see at Bipsync.
Unlike SCRUM, Kanban is quite light in that it doesn’t prescribe team member roles like “Product Owner” or “Scrum Master”, for example. While we have a number of people who can essentially fill those roles, we didn’t feel like we needed to formalise it as part of our process.
Employees are free to work remotely and we offer unlimited vacation, so the more the team is prepared and conditioned to be flexible, the better. Our team structure is almost flat, and wherever possible we encourage our staff to think for themselves and seek approval later, if necessary.
In the workflow of Kanban, tasks move from the backlog through a queue to completion. We’re free to define the queue; we have a sequence of stages which take a task through prioritisation, planning (typically UX and/or functional specification), development and QA. Since different people will pick up a task at each stage we provide contextual Kanban boards in JIRA.
So developers, for example, will work off a “Development” board in which tasks are in one of the “Ready for Development”, “In Development” or “Ready for QA” stages. The diagram below illustrates how this all fits together:
We’re free to skip elements of the process if we choose – internal tickets will often go straight from “Incoming” to “In Development” to “Ready for QA” – but generally this sequence maps our workflow really accurately. It has the added benefit of exposing bottlenecks – if the “Planning” stage is regularly full of tickets but “Development” is empty, then we know we need to prioritise expanding the number of people in our UX/Design team.
It’s important to stress that Kanban’s no silver bullet.
Its beauty is in its simplicity, but as it is it is not sufficient to provide a holistic solution for product management. We still value face-to-face standup meetings, scheduled at least once a week; we still get together to retrospectively analyse the outcome of a complex feature; we hold video calls to discuss the intricacies of a nasty software bug. The important thing for us is that these events happen on an ad-hoc basis depending on our circumstances, and not because a framework tells us we should be doing them. We let the team organise what happens when – which as it happens is a cornerstone feature of the Agile method.
Getting Our Priorities Straight
Our process is most effective when the priorities of tasks are well controlled and to manage this we have a buffer in JIRA within which we place tasks that as a company we consider important. We review this buffer twice a week to keep it stocked with tasks and ensure that all the information we need to work on them is available.
This buffer forms our “near term” work, and we take care to ensure it has a mixture of tasks. First and foremost we make sure that includes each of our clients with development needs. We also devote space to bugs or improvements to existing features, to ensure the product quality remains high. Finally we’re also sure to include a couple of items from our internal wishlist – these could be features from our product roadmap, or crucial infrastructure requirements like upgrades to libraries or programming languages.
Our hope is by managing our priorities in this fashion, we’re nourishing the product while keeping our clients happy and our staff supplied with work in a consistent, efficient manner.
So that’s how we approach Agile development at Bipsync. We’re keen to revisit this topic in the future as our processes evolve, so let us know if there’s something you’re interested in or would like to discuss.
