Agile Forest

Version One have released their annual State of Agile Development Survey for 2012. Co-inciding with this they also released a blog titled the ‘Top 10 Things the State of Agile Development Survey Won’t Tell You’ which I excitedly opened only to find it was a joke blog post. This was slightly disappointing as I love the effort and professionalism that Version One goes through to produce their survey and felt the blog cheapened it. I had hoped that the blog would outline the known deficiencies in the survey, but alas no. So I decided to write what I felt the  blog post may have covered if it took the topic seriously, so here it is – the Top 10 (okay maybe 13) Things the State of Agile Development Survey REALLY Won’t Tell You:

1.  What the co-relation between those with Agile Development Practice Experience and their role as an Agile Practitioner. I suspect that the 19% of Agile Coaches/Consultants/Trainers would make up a high portion of the 25% group that have 5+ years experience (It would be very scary if it wasn’t true). 
2. Why is it that 60% of respondents were managers/leaders or consultants – are these the only people that have time to fill in surveys?
3. Who knows what about Agile? Asking the ‘most knowledgeable’ is a good question, but it only tells a portion of the tale. What we really need to know is the extent of knowledge that each role has in general. Whilst the Product Owner might be considered the most knowledgeable in 1% of teams, overall what is their Agile knowledge – poor, sound, good, excellent? How do we know as a community where we might need to focus improvement without knowing each role’s understanding?
4. What is the business’s role in all this? A lot of the questions are focused at an IT layer and don’t allow answering and splitting responses based upon business versus IT – for example, are any of the Agile Champions actually from the business?
5. Where does Lean Startup fit into the Agile Methodology used? I know of a number of teams going down this path and whilst you could argue it isn’t a methodology (nor are most of the options officially), it would be worthwhile having this approach added in.
6. Where is ‘name your technique’ in the list of those employed? I still have no idea what ‘Integrated Dev/QA’ means – I am guessing it is eluding to a cross-functional team, but why not have the BA’s too? I don’t get Agile Games either. There is no practice or technique called Agile Games, games are a way we learn, it is a learning technique and has no direct relationship with Agile. Then we have the missing techniques – two fundamental ones beings skipped: Product Demonstrations/Showcase as part of Sprint Reviews and Backlog Refinement (formally known as Backlog Grooming). I wouldn’t put Burndown and Team-Based Estimation together either as they are two different things to me. It would be lovely to see Release vs Sprint Burndown split too.
7. Where is ‘Cost:Benefit ratio no longer being acceptable’ as the major cause of Agile Project failures? Most Agile projects that I know get canned do so because the assumptions that were made at the start of the project no longer stack up and consequently it is no longer worthwhile to continue the project resulting in the project being cancelled. It seems to me that the ‘Leading causes of failed agile projects’ is actually talking about ‘Leading causes of failed Agile Transformations’.
8. Where are the Kanban practices/techniques? Cycle time is there, what about limiting work in progress, pulling work, visualise and  manage flow, making policies explicit?
9. Where are distributed teams as an organisational issue? I see this commonly as one of the biggest issues – is ‘failure to integrate people’ the same thing?
10. How many Scrum Masters also have a ‘Project Manager’ title or responsibilities?  How many Scrum Masters also actually help the team to deliver? I would love to know the answers to these questions.
11. How effective is the role of the Product Owner? There don’t seem to be any questions around this and I am curious as to whether Product Owners out there are answering questions in a timely manner, with the right information and how many are proxies?
12. What is the difference in people’s minds between a Bug Tracker, a Taskboard and a Kanban board? To me they have always been one and the same. Where do portfolio management tools fit? What about retrospective tools? How many people are using a physical board vs a virtual (or both)?
13. How many people are doing Agile versus being Agile? This is the question that I would love answered dearest of all.

So what would you like the State of Agile Development Survey tell you that it currently doesn’t?