Redux: Visualizing the Entire TBS Policy Suite

by Nick Charney

---

Last week I shared a visualization of the Treasury Board Policy Suite that I put together (See: The Web of Rules: Visualizing the Entire Treasury Board Policy Suite). I didn't have the time to write it up properly but in the spirit of the Internet I thought I'd just throw it out there and do some spaghetti testing. The visualization garnered some significant attention and a bunch of questions from people trying to parse what it all means. This week I wanted to share the next iteration of the visualization (v.2), explain how I put it together and elaborate on some of my key takeaways and next steps.

Version 2: TBS Policy Suite

But first, as a reminder, this is how the TBS website characterizes the policy suite:

... and here is what it actually looks like. You will note that this version has a swanky legend and that the size of a particular instrument is now reflective of the number of connections it has to other instruments (tip: click the three dots in the centre of the visualization to expand the viewing pane, or click here to open it in another tab).



How I put it together

First, I signed up for a free Kumu.io account.

Second, I painstakingly clicked on each of the 273 policy instruments listed on the TBS website and manually connected each instrument to the other instruments listed under the 'related instruments' section on the right hand side of the page.

Third, I made sure to colour code each type of instrument (e.g. framework, policy, standard, directive and guideline) differently.

Fourth, I let Kumu do the rest. Where the instruments are located on the map and the distance between them is a function of the software itself.  While there are obviously some ways to tweak proximity and weighting, I haven't manipulated the data in any way that would unduly shift or bias it's emphasis. The folks at Kumu have been working with me to improve the overall quality of the visualization and I can assure you they have no bias one way or another about what it shows, only that it shows well.

Caveats

I've gotten a couple of comments about how some of the instruments have been moved elsewhere (e.g. National Joint Council) but my approach has been to work strictly with the information posted on the TBS website. I had to draw some boundaries around the scope of the work, those are them.

While I've managed to export, clean, import and rebuild the visualization in a way that improves its overall quality, there may still be some elements of human error. If you see something that's amiss, please let me know and I will look into it.


Key takeaways

The policy suite 'as is' bears no resemblance to the official artist rendering. The former lacks the sense of balance, coherence and wrap around support that the latter somehow conveys so clearly (and incorrectly). I don't have an explanation for this and I doubt that there is anything sinister at play here. It's as likely to be the result of best laid plans being twisted over time, sprinkled with with extenuating circumstances, mandate creep and a culture that papers over problems with more paper.

Given that we now have a clearer picture of what's actually at play within the policy suite, it's far less surprising that our culture is what it is. The most prominent instruments are (in order, based on the number of connections):

1. Policy on Government Security (88 references)
2. Policy on Internal Control (71 references)
3. Policy on Management of Information Technology (59 references)
4. Policy on Privacy Protection (56 references)
5. Policy on Information Management (55 references)

That said, any attempt to improve the web of rules needs to not only consider the number of relationships between instruments but the complexity of the interrelated contents within them. One of the distinct advantages of this visualization is that it makes a strong case for a complete policy reset; it can help illustrate that incremental reductions to the web of rules can take us only so far. It can be used to model the magnitude of the actual effects of reductions on the web itself. Besides, while de-linking one instrument from another or removing an instrument altogether may reduce the overall complexity of the web, it will accomplish little if we simply give into the temptation to shift its contents into another instrument. Let's avoid the policy shell game.

The model also helps drive home the case for mandatory reviews of policy instruments at regular intervals to ensure that they reflect the realities of the context within which they are being applied and not only the realities within which they were originally written. Any new approach needs to be cradle-to-cradle.


Next Steps

1. Constant improvement: if you see something that can improve my work, let me know.
2. Collaboration: I'll be at CodeFest in a couple of weeks sporting this visualization and the dataset I built to produce it.
3. Education: I'll be meeting with some of the officials at Treasury Board to walk them through the model and methodology; and hopefully whatever else we build at CodeFest.
4. Historical analysis: I'm contemplating working on a companion piece that shows how the suite has evolved over time. 
5. Positive Spillover: I'm looking for other applications of my methodology.

If you are interested in any of the above, please let me know. I'm heading on vacation for the week. I'll still be plugged in but corresponding at a far more casual pace.

Cheers

The Web of Rules: Visualizing the Entire Treasury Board Policy Suite

by Nick Charney

---

Here's the deal. This post is to the point and factual. It's short on commentary and long on crunchy bits to chew on. I've wanted to visualize the Treasury Board Policy Suite for a long time but I didn't know how. I've talked to a lot of people about the challenge, then I bumped into Sarah Chan who told me about Kumu, a relatively new piece of software; none of this would have been possible without her pointing me in the right direction. Thanks Sarah.

The Treasury Board Policy (TBS) Suite

If you aren't familiar with the TBS policy suite you can find it here; essentially it represents everything public servants need to comply with rule wise. It's often called the web of rules and after visualizing it, I can see why.

The web of rules constitutes:

• 1 Code of Values and Ethics; 
• 8 policy frameworks; 
• 73 policies; 
• 76 directives; 
• 56 standards; and 
• 59 guidelines.

Meaning that all total, the web of rules is made up of 273 distinct documents that (mostly) relate to one another and need to be considered in tandem. If you take the time to look at any individual framework, policy, directive, standard or guideline online you will see that there is a section on each of their webpages that show you what other frameworks, policies, directives, standards or guidelines you need to read in conjunction with them. Hence, the web of rules. The whole thing is incredibly complicated, but luckily TBS has a great graphical representation of the 'constantly evolving' policy suite (click to enlarge):

And, according to the TBS website, the above diagram:

"... presents the structure and the organization of the evolving Treasury Board (TB) policy suite. The policy instruments will be organized as a cascade of instruments within policy frameworks in key management areas: People; Compensation; Information and ; Technology; Financial Management; Assets and; Acquired Services; Service; and Governance and Expenditure Management:"

Only it doesn't ... at least not when you painstakingly (manually) verify the relationships between all 273 documents by hand using software that can calculate and visualize the actual relationships between them. When you do that, you get something that look more like this:

Now, given that I have never tried to embed a Kumu visualization before, if the above embed didn't work click here, or try opening it in a different window or just drop me a line. Note: there are 3 little dots between the description as the visualization that will expand the visualization while collapsing the description.

If you can see the visualization, here's a couple of things worth noting:

• The Code of Values and Ethics is represented by the biggest dot; note that it's no where near the centre of the visualization despite it being at the centre of the TBS diagram
• The noticeable / largest circular cluster is what I would term the Information, Technology and Communications cluster
• The second noticeable oval cluster is financial cluster (i.e. how we spend money) 
• There are a number of things that are in fact floating (i.e. not connected to anything else)

You can manipulate and navigate the visualization ...

• You can zoom in and select any particular node by clicking and holding it; any unrelated nodes should disappear.
• The nodes are colour coded and you can opt to look at only the relationships between frameworks, policies, directives, standards, or guidelines.

I'm still processing what it all means 

But I do have a couple of thoughts I'd ask you to chew on:

1. What we say we are about (values and ethics) and what we are actually about (information, technology, communication and, separately, financial disbursements) seem to be very different
2. If 'web of rules' is an apt metaphor then ought we not focus efforts on reducing the number of nodes in it so as to not get caught up in it so easily? Wait, we already tried that.
3. Given that the above already reflects a concerted effort to reduce the web of rules, maybe we ought to consider a more aggressive approach, rather then trying to reduce the web, why not reset it completely and build something new from the ground up?

As always, if you'd like to talk about the web of rules or this visualization please feel free to drop me a line; otherwise I am eager to hear your comments.

Cheers