Innovation and Rigour

by Kent Aitken

---

There's a general rule in academic research: you know you're coming to the end of your literature review when you've already read all the references in the articles you're reading. The second line of defence is that this gets validated with an advisor who has been studying the field for decades.

We don't have that luxury in public administration, and particularly in public sector innovation. There's no such systematic record of experiential and practitioner knowledge. How do we know when we know enough about a skill, project, or field?

The four stages of competence

You may be familiar with the four stages of competence model for a given skill. The idea is that we start incompetent, but don't truly know it. We know so little about the skill that we can't even meaningfully assess our own ability: (1) unconscious incompetence. As we learn more, we realize how little we know, reaching (2) conscious incompetence. Eventually we become adept and know it, the level of (3) conscious competence. When we master something, we can do it on autopilot, without really thinking: (4) unconscious competence.

I wrote last week that, in general, people are probably unconsciously incompetent at facilitating online collaboration (see: The Promise of Online Collaboration). Which isn't trivial: if someone has a bad experience collaborating with a group, they'll disengage and not return (just as they would attending a poorly designed meeting or conference). So where we fit in this competence rubric for a given project is important. We're making decisions in the public trust. How do we know when we're prepared to do so? When to move forward rather than signal-checking with others or conducting additional research?

The dark side of experimentation

Experimenting is good. Experimenting without truly knowing what we're experimenting on is not: it leads people to skimp on the equivalent of the 'literature review' (what's been done before? Who can I talk to for advice?), skip setting markers to know if they're on the right track, and fail at critically assessing and sharing the knowledge gained (see: Standardizing Innovation).

It's tempting to use a baseline of zero: "Before we did X, nothing was happening, then we did X and something happened: ergo, success." Unconscious incompetence applies both to implementation and measurement. Falsely declaring success creates complacency (see: Pilot Projects and Problems). It slows the move towards competence. And it represents underdelivery.

Get meta about innovation

So what's the equivalent of academic rigour for public sector innovation? Could we work out useful heuristics to ask ourselves? Something like:

• Who are the leaders in this field? What are they doing? Can I talk to them?
• Could I give a hour-long presentation on this field tomorrow?
• Where would this project fit in the Cynefin framework for problem complexity?
• What will this project impact?Is the level of rigour in designing the project commensurate with its potential impacts? (Future prospects for the organization? Groups of stakeholders? How many people?) 

I don't know what would work, but I'd love to hear ideas. Because every time we try something that hasn't been done before and succeed, the culture needle moves a tiny bit from risk-adverse to innovative. Every time we fail, it wiggles back. We owe it to ourselves, our colleagues, the next generation of driven public servants, and to Canadians to be thoughtful and purposeful. But, also to avoid the dark side of being unnecessarily rigorous - one project's reckless abandon could be another's costly analysis paralysis.

This is the get meta about innovation: how do we move from asking ourselves Is this a good idea? to How do I know this is a good idea?

On Gamification, Portal, and the Public Sector

A few weeks ago I shared Valve's employee handbook, positioning as the defacto office culture that all other office cultures are competing with. Last week I wrote why I believe that organizational learning plans are crutch. What I failed to do was explicitly connect the two in some meaningful way.


Consider what follows my attempt to do just that

Organizations like Valve don't bother with formalized learning plans, rather they choose to position learning as a cornerstone of their business. They recognize that creativity is their competitive advantage and organize around that very principle. Here is how the handbook positions learning:

If you were hired as a software engineer, you’re now surrounded by a multidisciplinary group of experts in all kinds of fields—creative, legal, financial, even psychological. Many of these people are probably sitting in the same room as you every day, so the opportunities for learning are huge. Take advantage of this fact whenever possible: the more you can learn about the mechanics, vocabulary, and analysis within other disciplines, the more valuable you become. (p39)

The approach at Valve is, I think, akin to what Rands recently described as "sandbox learning" in a great piece entitled "Two Universes: Design Well-Informed Improvisation"; a piece that explains the importance of gamification for learning using Valve's popular video game Portal as its primary example. Gamification, according to Rands, isn't "pulling the worst aspects out of games and shoving them into an application" but simply "clever strategies to motivate someone to learn so they can have fun being productive".

I want to pull in some longer quotations from the entire article; but I want you to think about them in the context of organizations and not in the context of video games. Don't worry, I will be chiming along the way to help you draw some comparisons.

As you progress through the increasingly complex puzzles, Portal does something [cool]. It teaches you the game, it teaches you how to improvise solutions to the puzzles, and it eventually makes you a master of the Portal gun and its associated physics — without a single page of documentation. You learn about the Portal universe intimately, but you don’t notice the learning because you’re too busy playing.

This is precisely the type of learning I was referring to above and in previous posts. Learning on the fly, no documentation, just a fully immersive environment culture.

In addition to not knowing what the hell is going on in terms of the plot, the first time you play Portal, you have no idea how to play it. Like all games, the initial levels teach you the very basics: how to move, how to pick up an item, and how to use items to get things done. Yes, there is a heads up display indicating how to move, but it’s up to you to learn. Oh look, when I put the cube on the button, it opens the door… to where? The plaques at the beginning of each level seem important, but I don’t know why. Why do I feel something sinister is going down?

The mystery of the player not having a clue what the hell is going on is the initial incentive to learn. It’s the desire to discover the story that situates the player in the Portal universe. It’s a difficult balance to strike in designing a game or application. How much do you explain versus how much do you let them discover? Too much explanation and you get this [image of a complicated user interface]. Too much reliance on exploration and they may never discover what they can actually do.

Think back to your initial on boarding at your organization. You probably didn't have a clue what was going on. You learned the basics first, how to move around, how to get and use your security pass, how to negotiate early relationships and other general housecleaning items.

Not knowing how your organization worked was incentive to learn. As was trying to figure out your own role within that organization. The key difference here is that no one actively thought through how much to explain to you versus how much to leave you to discover. You probably got the overwhelming walk around, the deluge of forms and the quiet desk time in (probably in that order).

Unlike the video game, very little forethought was given to how you'd react to the system around you. Which, on your first day at the office, is just as much of an unknown as a first encounter with the Portal universe.  However, much like the video game, too much explanation (in the form of "hey read this enormous pile of documents, they will help you make sense of the environment") and you'd tune out.  While too much freedom to discover everything on your own would leave you wandering aimlessly in and out of people's cubicles for days (I've experienced both of these during different organizational on-boardings).

More on this from Rands:

As the game progresses, the increasing complexity of the puzzles introduces a bevy of hazards, including high energy pellets, goo, and turrets. The goal remains the same: get out - don’t die. This is a tricky inflection point for any game: the arrival of the puzzle which is no longer a straightforward challenge, and I believe Portal’s developers have solved for this moment in two ways.

First, Valve play tests the hell out their games. They are intimately aware of when a chamber is too laborious, too complex, or introduced before the player has learned the lessons they need to satisfyingly solve the puzzle in a reasonable amount of time. This is essential testing that must be performed again and again to find a delicate balance providing a sense of progress and accomplishment with just enough challenge.

This is a critical inflection point where the user is weighing the following: is the amount of investment I’ve made to date worth banging my head against the screen trying to figure out what to do next?

As you progress through your career your work becomes more complicated, but the goals remain the same: get your work done - don't screw it up. Similarly you are likely to reach that key inflection point where work is no longer straightforward. Perhaps you no longer have a boss who hands you the work with a clear set of expectations, you are moved in a reorganization, etc.

The key difference here is again that no one actively designed and tested your work flow (or for that matter your career path). No one is intimately aware if your workflow is too laborious, too complex, or if you've been put into a situation before you learned all the prerequisites. As with the video game, this may be essential testing, yet unlike the video game it isn't performed rigorously - it probably isn't even tested at all.

As a question, I'd frame it thus: do those who have set your organizational structure have any expertise in organizational design? Have they tested multiple workflows, solicited feedback and chosen the most effective blend from the options available to them?

I think we all know the answer to that.

Yet again, as Rands points out, you will eventually reach a critical inflection point where you, the employee, is weighing your options: is the amount of investment I've made to date worth banging my head against the wall trying to figure out what to do next? Is it time to escape cubicle nation?

I suppose it depends on whether or not you apt at what Rands calls "well-informed improvisation"

Inevitably, you’re going to need to make a split-second decision in Portal. The floor will literally be vanishing from under your feet and you’ll have no time to consider your options; you will just improvise. It’s these moments of well-informed improvisation that I believe are Portal’s greatest accomplishment and best design. See, while you were busily having fun you had no idea that you were becoming an expert in the ways of the Portal universe. You now have experience using each of the individual tools and their behaviors to be able to combine them to handle the unexpected. The result: you are now able to effectively deal with novel and unknown situations.

It’s incredibly satisfying to sneak out of a tight spot by performing an action you didn’t know you could do, but created instinctively because of your experience.

That’s how I want to learn. Don’t give me a book; I don’t want a lecture, and I don’t want a list of topics to memorize. Give me ample reason to memorize them and a sandbox where I can safely play. Test me when I least expect it, shock me with the unknown, but make sure you’ve given me enough understanding and practice with my tools that I have a high chance of handling the unexpected.

Amen!

A video game has a very different goal than [a program like] Photoshop. A video game is designed to be pure entertainment, while Photoshop is a tool by which you get work done. A game designer knows that if a game isn’t both immediately entertaining and usable, the folks sitting in front of the Xbox 360 are going to stand up, toss their controllers on their beanbags and declare, “Screw it.” Worse, they are going to tell every single one of their friends about this gaming disaster because they feel stupid for wasting their time and money on something that was supposed to be fun, but turned out to be lame. This is game death.

Photoshop’s goal isn’t entertaining unless you think the national pastime of bitching about Photoshop is a sport. Photoshop has no points or leaderboards because Photoshop is a tool and the perception of tools is that you must be willing to supply blood, sweat, and tears in order to acquire the skills to become any good at using them.

Bullshit.

Make a list. Tell me the number of applications you use on a daily basis where there is a decent chance that you’ll end up in a foaming at the mouth homicidal rage because of crap workflow, bad UI, and bugginess. Is Photoshop on that list? Yeah, me too.

I know, I know, you are thinking: "A video game has a very different purpose than an organization, especially a public sector organization. Video games are about having fun, organizations are about getting shit done. Fun isn't important, and in order to get things done we must be willing to accept and work within the confines of the status quo.

Bullshit (to quote Rands).

List the organizations that you've worked for where on a daily basis there is a decent chance that someone inside the organization might choose to just walk away because of crap workflow, non-nonsensical rules and a lack of follow through. Is your current employer on that list?

Rands on why design is important for learning:

Great design makes learning frictionless. The brilliance of the iPhone and iPad is how little time you spend learning. Designers’ livelihood is based on how quickly and cleverly they can introduce to and teach a user how a particular tool works in a particular universe.

Perhaps there is something to be said about creating a similar economy where the livelihood of the powers that be would be based on how quickly and cleverly they could introduce to, and teach us how to solve, the big challenges this sector is facing?

If we put as much effort into designing our organizations and their workflow as Valve does with its games perhaps our work could be as engaging as playing a game like Portal.

Final note I highly recommend adding Rands to your required reading; he consistently publishes thoughtful long form writing and is one of the few writers that have been reading consistently since I myself started writing.




Originally published by Nick Charney at cpsrenewal.ca subscribe/connect

Learning plans are a crutch and I'd rather learn without one

First let me start by saying that the line of reasoning I am about to present to you may go somewhat against the grain; feel free to take issue or prove me wrong, as you will see below, its been done before.


I am not a fan of learning plans

My main contention is that organizations that are serious about learning don't have learning plans they have learning cultures. They don't have forms that need to be filled out, justifications that need writing, and approvals that need approving. Rather they position learning a cultural cornerstone of everyday activity, encourage inquisitive minds and guard against permission based barriers.

my crutches by dimtri_66

Learning plans are inherently antagonistic

I also think that learning plans reinforce organizational power differentials by reducing employees to applicants and managers to approvers.  This leads to skirmishes over justifications and shrinking resource pools.



Learning plans are biased towards a particular model of learning

In general, organizational training authorizations forms (the one's I have seen) are biased towards classroom or conference (learning event) based training. Where, for example, is the box on the form that allows me to express my desire for a more nuanced approach to learning?

They are also biased towards the here and now, I was recently asked to complete a 3-year training plan. How can I honestly complete it when the things I want to learn - the prescient things in my field - haven't even been invented yet?


Let me tell you a little story about learning plans

A few years ago, my boss at the time informed me that I needed to fill out a learning plan. We had an excellent relationship and while we agreed that we didn't really need a learning plan that is was mandatory.

So, I decided to have a bit of fun.

I took out a stamp my wife got me when I was a teacher's assistant and stamped the learning plan with it; the stamp read "Complete and Utter Bullshit". Signed it, turned it in, cajoled the admin into giving it to my Director General (DG), knowing full well it would get a good laugh but wind up back on my desk.

We were a small team (about 7 in total) so it didn't take long (I literally split a single office with the admin and our DG sat in the office next to ours).  A few hours later the learning plan was on my desk with a little note asking me to redo it. This time I decided to write a single line on the training form:

"All I want is the flexibility to discuss reasonable learning opportunities as they arise."

Again, I signed it and sent it up the pipe. Again it came down with a redo post it affixed to it. Finally I acquiesced. I wrote down a conference that was a few weeks away knowing full well that it would be near impossible to get it through the approval chain given that it was in DC (technically an international conference and thus subject to greater approvals).

Long story short, I submitted the form trying to prove that learning plans don't actually work and my DG proved me wrong.

She approved my training plan, and personally shepherded the request through the system and wound up in DC attending an O'Reilly Media Government 2.0 conference.  That woman went to the wall to get me to that conference. But I doubt proving me wrong was her motivation, she just wanted to honour her commitment because that's the type of person she was (and probably still is - we no longer work together).

I suppose in retrospect that learning plan did teach me something: that a learning plan is less important than the relationship between the people negotiating it.

That the former is only as strong as the latter.

That I ought invest more time in building relationships than filling out forms.

That a learning plan is a kind of crutch, and I'd rather learn without one.





Originally published by Nick Charney at cpsrenewal.casubscribe/connect

All I really need to know about Public Policy I learned from Lego

I love Star Wars.

When I was a kid I used to queue up the entire trilogy (yes, the trilogy) empty my vast Lego collection out onto the floor and spend the entire day in the Star Wars universe building things I'd seen on the screen before me.  It shouldn't come as a surprise then that I have passed those interests on to my children, both of whom love Star Wars (although they know 6 movies, animated series and countless video game versions) and Lego.

My son Kohl (4) with our new Lego AT-ST

Last weekend we watched Return of the Jedi then retired to the play room to play with the Lego (yes, I still play with Lego).  I somehow got it stuck in my head that I wanted to build an AT-ST walker. But before I did wanted to gauge the kids (stakeholders) interest in the project; they were obviously on board.  So with their blessing, I built the AT-ST pictured above completely from scratch (no instructions, no sets) in about a week's time.  It's far more complicated than anything I ever built as a child and I am incredibly proud of how it turned out.

That said, what I never anticipated was just how much the experience would teach me about public policy.  Here's what I mean.


Public Policy, like Lego, requires vision

You aren't going to get very far if you don't have vision.  In my case it was fairly easy, as a long time fan I have a good sense of what the end product should look like.  I think this speaks to the value that subject matter experts bring to the public policy discourse.  But even then, I repeatedly went online to look at pictures of AT-ST walkers, I needed to check my assumptions against hard facts.  The same applies to policy analysts who need to consciously check their assumptions against the data available to them.  Moreover, not only did I look at photos of AT-ST's but I purposely sought out photos that exposed different sides of the walker to me or that were done in different medium (sketches, paintings, computer generated graphics, other Lego builds, etc).  Policy makers need a well rounded understanding of their medium and that often requires expanding inputs beyond a small handful of expected sources.

But periodically checking in with photographs for direction also gave me the intelligence I needed regarding my resource pool.  Given what I've already completed and that which I have yet to complete, do I have enough resources at my disposal or did I need to go back for more?


Public Policy, like Lego, requires the right resources

I started by building a small chunk of the AT-ST but soon realized that I didn't have easy access to the pieces I needed to work on it project efficiently.  I have a huge (I mean huge) Lego collection.  My kids have not only the benefit of all the Lego they've been purchased over the years, but also everything I was purchased as a kid as well (thanks mom!).  Rummaging through an enormous bin of pieces can be incredibly time consuming.  After all, I had very specific needs for the project in front me.  I didn't need all the resources at my disposal, only a subset.  So I decided to slowly source the parts I needed before continuing to assemble the walker.  My requirements were fairly simple. Anything that was gray was in with the exception of wheels and really large platforms and priority was to be given to any parts that moved or could be used to make joints or connect angles. 

But I also had to be realistic. I knew that I wasn't going to need every piece of Lego I collected, so I didn't need to go through the entire collection routing out all of they gray pieces, I just needed enough resources to complete the job.  The rest could easily be returned for another project.  Having the right resources was incredibly important; but knowing what resources were in my collection but not yet directly accessible to me (buried pieces) was even more important.  The familiarity with the resource pool meant I knew whether or not my approach - an approach that hinged on a particular type of resource or its abundance within the collection - was plausible or not. For example, smaller pieces are generally the hardest to reach resources because they usually wind up at the bottom of the bin and as such require additional effort to secure.  At one point in the process I even had to build a makeshift scaffold from some small buckets to hold back pieces from collapsing back into the large container in order to reach these exact pieces at the bottom of the bin. Additional work pays dividends when you are certain the resources you hope to uncover actually exist. But sometimes those pieces don't exist, you simply can't find them, or you can't access them without huge cost. That's why you need to be able to pivot your approach, why you need to be flexible. You need to be prepared in the event that the end product may not look exactly like your vision.  Vision is, after all, limited to resources.  Remember that a roadblock often means an improvement, not a detraction of the quality of the end product. 


Public Policy, like Lego, has to stand up to scrutiny

The more complex the undertaking the more fragile it is.  This is especially the case during the initial build when you are unfamiliar with how to best handle your creation with care.  Quite frankly while you may have an idea of where the weak points are you aren't quite sure.  Typically we expect breaks at the joints but tragically, the only way to know for sure is to purposely break that which you have just created. 

So don't be afraid to knock it over, apply pressure, or add some weight.  Make note of where it breaks, then rebuild it, re-test it, and see if it breaks in the same spot again. Repeating this process a couple of times not only shows you its strengths and weaknesses but also teaches you about the act of rebuilding itself; which is to say that it breeds a familiarity and expert understanding that is important.  It's important because it will frame your reaction.  It will determine how you reinforce the joints, what pieces you swap out or in or lead to a complete redesign that circumvents a problem that cannot otherwise be fixed.

In the end you want to build something that can support its own weight.  A task that is made more difficult when you have moving parts.  My walker for example has a moving head that moves both horizontally and vertically. Moving parts are complicated, awesome, but complicated.  The more of them you have the more additional support you need, but the support you add should create a sense of robustness without undermining mission.  In my experience the best support mechanism in this regard is transparency. 

For example, the mobility of the head combined with its weight meant that I needed additional pieces to hold it in place. If I used coloured pieces it wouldn't be a walker anymore but something else entirely.  Using transparent bricks maintains the mission while offering critical support to the most complex piece of the undertaking. There is greater willingness to accept additional support when that support mechanism is transparent and minimally invasive; you are also quick to accept it when you compare it directly to less palatable (or more intrusive) alternatives. 


Public Policy, like Lego, should be a creative endeavour

Having fun is an inherent part of crafting.  It's incredibly important, so don't be afraid use your imagination and be creative.  If something doesn't work out right away, don't panic and don't abandon ship, its just bricks, there is nothing that can't be fixed, altered or tweaked, and there is always a work around. You just need to be willing to look for it. 


Public Policy, like Lego, could be exciting to next generation

After I finally completed the project, the look on my kids faces was incredible.  They watched me craft something amazing yet tangible; so much in fact, that it left them wanting more.

Shouldn't our governments strive for the same thing?





Originally published by Nick Charney at cpsrenewal.casubscribe/connect