Wednesday, September 4, 2019

Even more NarraFirma

I have just finished another NarraFirma release, again based on commissions, with more new features. Chiefly this:


I am calling it a "correlation map." It shows relationships among all the scale questions in your project in one graph. When you hover over a link you can see the scatterplot of combined values (as in this example). When you hover over a bubble you can see a histogram of values for that question.

You can also see how relationships among scale questions appear for subsets of stories based on answers to choice questions, like this:

 
Also included in the new release are more improvements to the catalysis process based on user feedback. For details see the blog post on narrafirma.com.

About improvements to NarraFirma

If you are a NarraFirma user, you should know that (a) I am interested to hear about your experiences and suggestions; and (b) I offer an excellent rate for open-source development commissions. You might want to look over the NF commissions page to see some of the ideas I am considering for the future of NarraFirma.

About this blog

I have not been posting on the blog much this year. I am not sure if anybody is reading it. Also, when I can find time between consulting gigs, I am working on two (nested) book projects, and I want to save my precious writing time for those. I will continue to announce things on the blog when I have things to announce; but the days of "feeding the blog" are, I think, over. (Unless a really great idea pops up that demands to be written about. That's different.)

Friday, May 17, 2019

NarraFirma keeps getting better

Last month I received some more development commissions to improve NarraFirma, so I have been working on it again, and I released a new version today.

The focus of this release has been on improving the flexibility and usability of the catalysis process.
  • You can now easily set up multiple observations per pattern and multiple patterns per observation. 
  • Many new options make your graphs and reports look the way you want them to.
  • There are new fields and new report types. 
  • The interface has been improved.
You can review the full list of changes at the NarraFirma blog.

By the way, if there is anything you would like to see improved in NarraFirma, I offer a pretty low development rate (as long as the changes are open source and reasonably useful to everyone). I am happy to talk about any improvements you would like to see.

Tuesday, April 9, 2019

Mail bag: Stepping stones

Hello readers. I have been very busy in the past few months with paying projects. This has been a wonderful thing for my prospects of continuing to do this work! But it has led to the neglect of this blog, and of the book project. I have a few partially written blog posts in development, because I've been learning a lot lately, but I like to wait until they are ready to be of use before I post them. As for the book, I should be able to get back to it soon. At the moment I'm working my way through a new set of development commissions for NarraFirma, plus another ongoing story collection.

However, I am recovering from a stomach bug today, so I wanted to do something simple and fun, so I started writing an email reply to a question somebody asked me last week. As I was writing, it occurred to me that some of you might like to read the question and my answer, and that my correspondent probably wouldn't mind if I posted their question here. This is an off-the-top-of-my-head answer, mind you, without any preparation, to take a break on a sick day, but you still might find it interesting.

The question was:
I have a question about how you relate sensemaking to ontology. I have heard the term multi-ontology used in reference to Cynefin, and I am wondering if Confluence as you use it is more useful for describing the environment. Do you have a working term around this? I am trying to connect this to the latest thinking in psychology around ontological pluralism of perception.
Here is the way I understand it. Ontology is the study of existence, of what-is. People used to talk about ontology as one thing, like there can only ever be one ontology to cover everything for everyone. Then some people realized that ontology wasn't really the study of what-is; it was the study of what-is-as-seen-by-those-who-get-to-say-what-is. And then the term began to be used in the plural, and people started to talk about ontological pluralism. (This is my vague sense of what happened. If I had the time and inclination I could look up all kinds of things and write pages about what exactly happened, but ... it was probably something like that. Now you see why it takes months for my blog posts to develop.)

My guess is that the term "multi-ontology" has been used to describe the Cynefin framework because it attempts to give people a way to represent multiple perspectives on what-is. That is, people can use Cynefin to talk about not only what-is in a general sense, but also with respect to various worldviews or mindsets on what-is. The Confluence framework also attempts to do this, but in a way that makes more sense to me.

In the white paper I wrote when I was on the verge of breaking Confluence away from Cynefin, I talked about visualizing "clouds" on Cynefin space (that is, on the "dimensional form" of Cynefin, which was my version of it, and which maybe no longer exists) to represent different perspectives, which could be called ontologies, or views of what-is. That paper represented most of the reason I moved Confluence away from Cynefin, because visualizing clouds drifting across a landscape requires a landscape, not a bunch of boxes. (Maybe the dimensional form of Cynefin was a thing, but it was not a thing people seemed to pay much attention to - and that's fine, but it didn't suit my needs.)
~ ~ ~  

Now, as to whether Confluence is "more useful for describing the environment," I would say that Confluence is more useful for describing the environment when a person is in the right state of mind to use it to do that. When they are not in the right state of mind, it is not more useful; it is less so.

What do I mean about being in the right state of mind? It's like when people learn about biology. First you learn about kingdoms and phyla and classes and orders and genera and species, and you learn about evolutionary eras, and you learn about stages of embryonic development, and you learn lots of other names for things. You get the most you can from all those boxes, and then they tell you (or you realize) that the boxes are made up, that the boundaries between cells and organs and bodies, between colonies and organisms, between species, between historical periods, between stages of development, are lines we drew, not lines we found.

After you cross that threshold, you begin to explore biology at a completely different level. You see intermingling and interaction you could never have imagined before. You see things that might or might not be alive. You see living things that might be organisms and might be parts of organisms. You see organisms that are both plants and animals (or maybe neither). You see trees that might be forests and forests that might be trees. You even begin to see yourself differently - as both a being and an assemblage of beings. The things you think about become more confusing, and a lot more fascinating.

~ ~ ~

You cannot reach the second level of understanding biology without going through the first level. If I tried to explain biology to you without using any artificial constructions, I couldn't do it. For example, every time I said "species" I would have to qualify the word with a dozen statements about how the species concept is flawed and controversial and challenged by many examples in nature, and how we really shouldn't be putting a lot of weight onto it, but that it has some benefit as a mental construct in spite of its limitations. Two quick quotes:
No term is more difficult to define than "species," and on no point are zoologists more divided than as to what should be understood by this word. -- H.A. Nicholson, 1872.

We show that although discrete phenotypic clusters exist in most [plant] genera (greater than 80%), the correspondence of taxonomic species to these clusters is poor (less than 60%). -- Rieseberg et al., 2006
I would have to explain all of these layers of nuance for every term I used, and we would make very slow progress.

This is exactly why people create categories - useful yet artificial constructions - even though they know that the utility of those categories will decline later on. Categories are both necessary and insufficient. You can find them in every sphere of knowledge, from board games to rocket science. The beginners know the spaces, and the experts know the space.

This is why I like to think of boundaries that divide up the indivisible space of what-is as stepping stones to greater understanding.

What bothers me is when people step onto a stepping stone and become convinced that they have reached the other side. The best stepping stones wobble, that is, communicate their weaknesses to those who are in the right place to notice them. When a framework or model does not cause anyone to question it or want to reach beyond it, it stops people from making progress beyond it when it is time for them to do so.

~ ~ ~

The great majority of people start thinking about organization and self-organization by using one of the things-in-boxes frameworks, of which there are many. This is appropriate, because at that stage of thinking, you need boxes to be able to think about organization and self-organization. Attempting to understand those things on a continuous, blended landscape is too confusing to be useful - at that time.

Can you imagine explaining to someone who is trying to learn the various families of birds that the very species (and indeed families) whose names they are struggling to remember have shifted over time? What would they say if you told them that the categories they are learning have been influenced by accidents and personalities and disagreements and mistakes - and fashions and fads, of all things? What would they say if you told them that thirty years from now some of the names will have changed? No teacher would bring any of that messy stuff into the picture when their students were prepping for a test on bird identification. You don't learn about the messy stuff until years later, when you start exploring what's behind things like Linnaean taxonomy and the periodic table and other constructs we take for granted. Sometimes there are juicy stories about why we frame things the way we do, and how we almost framed them differently, but somebody said something to somebody at some party, and everything changed. Things are never as simple as they appear when you are first learning the names of things.

When you have got as much as you can out of thinking about organization and self-organization by using the things-in-boxes frameworks - and you can tell when you have reached that point, because you start asking more questions about the boxes than about the things inside them - it is time to move on to a more nuanced way of thinking. After you cross that threshold, you begin to explore organization and self-organization at a completely different level. You see patterns that are both complex and complicated - and usually, not in the way you expected them to come together. You see how these forces help and hinder and seed and destroy each other.

from flickr, by Alfred Grupstra
As you begin to explore the nuances of intermingling and interaction, you realize just how far down these things go - to the extent that simple words like "system" and "adaptive" start to seem ridiculous to you. The categories and boundaries and zones you once saw as canonical morph from certainties into contingencies. You see many new things: influxes, backflows, pockets, flourishes, flashes, disasters, revolutions, realignments, black holes, bright lights, peace, joy.

~ ~ ~

When you reach that second level of thinking about organization and self-organization, it's time to use the Confluence framework, or another of the continuous frameworks, such as ... such as ... I don't think there are any other continuous frameworks. Dee Hock's chaordic concept and Manuel deLanda's ideas on meshwork and hierarchy talk about blending, but they don't have sensemaking frameworks, in the sense of diagrams and exercises people can use to think with them. I know that Ralph Stacey talks about blending (somewhere, I'm not sure where), but his Agreement-Certainty matrix describes a priori bounded spaces, and seems to be used in that way.

I wonder why I have never seen another continuous sensemaking framework for organization and self-organization. (I wonder why I have never noticed this before.) Either there is nothing like the Confluence framework, or something like it exists and I am unaware of it. If anybody knows of a sensemaking framework that helps people describe the intermingling and interaction of organization and self-organization in a space that has no boundaries marked out in advance, please tell me.

Because this is a problem. It is either a weakness in my awareness about what is available or a weakness in our collective support of sensemaking. There should be more stepping stones at this level than just one. The people who build sensemaking frameworks and the people who use them are like authors and readers of novels: every framework has its thinkers and every thinker has their frameworks. There should be a variety of frameworks available at the continuous level, just as there are at the discrete level. If you know of another such framework, please tell me. If you don't, somebody build one! We need more tools of this type.

To be clear, however, I do not believe that everyone - that anyone - needs to use the Confluence framework to make sense of the ways in which organization and self-organization intermingle and interact, either in general or with respect to any situation or topic. Nobody needs a sensemaking framework to make sense of things; they can just make sense of things. Frameworks are playthings, not gatekeepers. They are unnecessary and insufficient. Never let anybody tell you that you can't think without the thing they made. You don't need my framework; you don't need anybody's framework. You've already got what you need: that squishy thing in your head.

~ ~ ~

What do I have say about the term "multi-ontology"? I'd say it is useful, but again, I don't think it's sufficient. If you think about ontologies that are commonly contrasted, such as between Western and Eastern, or between the global North and South, or between indigenous and non-indigenous people, seeing these worldviews as simply "different" is a simplistic way of looking at them. In reality, different ways of seeing the world are not so easily separated. They intermingle and interact. The term "multi-ontology" is itself a putting-things-in-boxes term, and as such, I think it is best used as a stepping stone to deeper exploration.

I spent a few minutes today looking up "the epistemology of ontology" and found some people talking about it. I particularly liked what Seth Miller said in a blog post: "the ontology of epistemology is the epistemology of ontology." Meaning, when you reach into the roots of ontology you find epistemology, and vice versa. The two phenomena intermingle and interact.

~ ~ ~

from flickr, by davidgsteadman
One of the reasons I like the metaphor of clouds, and the exercise of visualizing them on the space defined by the Confluence framework, is that clouds are useful metaphors for ontologies.

This is true for several reasons. A cloud is both an object and a process. Where its boundaries lie depends on how you look at them. Its behavior is partially observable and partially predictable, but never sure or certain. It can sometimes be placed into a category, but only temporarily and provisionally. It has a history: it is born, grows, and dies. A cloud might be two clouds for a while, and then one cloud again, but in a different way.

Another useful thing about clouds is that they are used to perceive and understand other forces - air masses - that lie unseen in and around them. In the same way, ontologies, or ways of seeing the world, can be used to perceive and understand people and the lives they lead. If I tell you some aspect of my understanding about the world, some element of my personal ontology, you have learned about me in the same way that a meteorologist looking at the clouds in the sky has learned about the masses of air moving overhead. 

When people represent perspectives as clouds on Confluence space, they can talk about how those clouds intermingle and interact. They can ask questions like:
  • How are these clouds distinguished? Where do they overlap? Where are they far apart? Where do they bleed into each other? Where is it hard to tell which is which? Where is it easy to tell? 
  • How do these clouds influence each other? Are there places where they cooperate? Compete? Do both at once? Do they have indirect influences? Through what? Where and when?
  • How have these clouds changed over time? What has their history been like, considered individually and together? How might they change in the future? What would that mean?
I'm sure I could write more such cloud-exploring questions given more time, but I would like to post this today. Maybe I'll come back to it later.

~ ~ ~

Now I'll ask the question I ask myself about everything I say and think and write: What if I'm wrong? What if nobody "needs" to move to "a higher level" in their thinking about organization and self-organization, with or without the aid of a framework? Nobody could possibly deny that these things intermingle and interact, but I can see someone saying that there is no point in exploring such nuances, that I'm asking people to waste their time counting angels on the heads of pins. Sure, I can see that perspective. It's not my perspective - my view is that there is so much in here, if you just open the door and come in. But I can definitely see the point of view that the things I like to think about - the intermingling and the interaction - are not that important, that the broad expanses of - No. There are no broad expanses. In human life there is no non-intermingled organization or self-organization. There just isn't. Whenever you find one thing you find the other, and either you pay attention to that or you don't. And you should. Eventually, when you are ready, you should.

Probably the best thing is to go by the 80/20 rule. You can probably achieve about eighty percent of what you would like to achieve in thinking about complexity by thinking about things in boxes. If you're already there and you want more, open the door and walk into the space where things are not so easily put into boxes, into the space of nebular ontology.

At my house right now, the air is warm in the daytime, but there's still quite a bit of snow on the ground, so every day we get these beautiful, serene ground fogs that thin out from bottom to top. I look for these ground fogs every year. I watch them as they wander through the trees, and I think about intermingling and interaction.

~ ~ ~

Now a few questions I think you might be asking. First, does my little come-to-the-clouds sermon mean that I think people should use things-in-boxes frameworks for a while, then abandon them? Absolutely not. I didn't stop calling red-winged blackbirds red-winged blackbirds after I explored the species concept more deeply. But I did see species differently than I did before. I stopped caring as much about whether I got every identification right, and I stopped thinking it mattered if other people did. The more I learned about all of the various categories in biology, the more they stopped being biology to me. They became things people built to study biology, and biology itself became something bigger, weirder, and harder to explain - but a lot more exciting. I didn't stop using the categories I had learned before, but I stopped them using them blindly, in every situation, without thinking about what they meant in context.

I've seen a lot of people go through changes like that as they have learned more about a subject (including stories, and including complexity). Everyone seems to start out by repeating terms from the dominant categories, memorizing them, holding onto them like totems, applying them to everything reflexively, without knowing why. Then, at some point, if they keep learning, they start asking more questions about the categories than about the things in them, and they start looking for the next stepping stone. Later, when they come back to the same categories, they treat them as resources, not templates. They use them like master chefs use cookbooks - to dip into for ideas, to step into and out of, to mix and match. But never again do they see those categories as fully capable of representing the subject they have learned about.

So, if you like the stepping stone you're standing on right now, and you don't want to leave it, I can give you this bit of encouragement: When you go back to a stepping stone after you've gone beyond it, it's an even better tool, because now you can use it in ways you never could before. Now you know why you are using it, and that changes everything. Now you use it when it's useful, and you don't use it when it's not useful, and you know which is which. You might even know how (and when) to use parts or aspects of it blended with parts or aspects of other tools, in a sort of cloud-like assemblage of ideas that intermingle and interact.

~ ~ ~

Another question you might have is: Does the Confluence framework wobble? It does for me. I keep thinking that the process needs more support at the point where you discover features in your populated landscape. Some feature categories might be useful there. Also, I think the model might want to grow a bud, with more sub-frameworks in the area where organization and self-organization come together most strongly (that is, in addition to the mixed sub-framework).

Does the Confluence framework wobble for other people? I have no idea. That's because it isn't done growing yet. It's half a framework and half an idea for a framework. It needs more testing, and I need more feedback on it. This is partly my fault. I'm like one of those rock bands that refuse to play their old songs. If I don't have something new to say about something, you are not likely to hear from me about it. And I've always got lots of new ideas I don't have time to turn into real things.

Still, I have thought about introducing Confluence more formally, in a paper published in a peer-reviewed journal. I haven't done that yet because the framework is not ready for it, not without more testing. I'm not going to describe a thing and how to use it until I can truthfully say that people have used it and found it useful. Sure, lots of people have used the simplest form of the framework, because it was part of another framework for a while (though it's a fair question how many people actually ever knew about or used it in that form). But I want to see if Confluence works now, the way I want it to work now. And if it doesn't work, I want to make it better. [Edit: I did.]

Here's an idea. If you can find at least six people who want to spend a few hours in a room using the Confluence framework to think together about some situation or problem, I'll coach you as you plan and prepare for the session (over Skype), as long as you promise to tell me what happened and allow me to tell the story of what you did in a paper. (I'll show you what I plan to write and give you the chance to change it.) You do have to use the whole framework, not just the first, simplest bit. Let's say that the first five people who send me a note about this (and then actually do it, like within the next year) will get my help doing this for free. You'll help me and I'll help you. Plus, it'll be fun.

[Edit: This offer of free consulting is no longer open. Nobody took me up on it, so instead I spent most of the next two years working on a book, during which time I talked to dozens of people and read hundreds of books and papers. So I got the feedback I needed in other ways.]







Monday, January 14, 2019

More Better NarraFirma

NarraFirma has been getting some attention lately. I received development commissions from several NarraFirma users, so I've been working on the software since early November, and I've bundled together all the things people wanted into a new version, 1.2.0.

The blog post on the NarraFirma web site goes into great detail about all the new features and changes. I'll just give you a quick overview here.

Easier importing

You should now find it easier to work with NarraFirma alongside such survey systems as SurveyMonkey, LimeSurvey, Google Forms, and the Open Data Kit.

To begin with, you can import a wider variety of question formats, and there are many new import options. You can also specify all of your import options in NarraFirma itself, without writing a CSV specification file.


And there is a new pre-import check, so you can see exactly what is being read correctly and incorrectly.


I also wrote a new 18-page Guide to importing data, which explains everything I know about it.

More better graphs

One of NarraFirma's strengths is its automatic combination of questions. Every possible combination of this-versus-that is generated for you. So when a NarraFirma user suggested a combination I had never thought of before, I just had to build it. It's a choice by choice by scale graph, a contingency-histogram chart, which looks like this:



The little pseudo-histograms in the boxes help you compare the distribution of a scale question (like: Did the people in this story get what they needed?) among subsets formed by the conjunction of two choice questions (like: the people in the story needed respect, and the storyteller felt disappointed about the story). 

There's also a new "Story length" question, so you can see if people more disposed to verbosity had anything different to say.

Also, you can now write out the graphs in your catalysis reports using SVG format. SVG graphs come out sharper than the old PNG option (which is still available) because they are vector-based. You can also style SVG graphs using CSS classes (even after you have saved the HTML report -- that is, outside NarraFirma). 

Also, you can now print an observations-only catalysis report. I added this specifically to support people who want to do their catalysis in a group workshop.

Better ways to clean house

NarraFirma uses what is called a journaling datastore. We chose this option because we wanted to make sure your data would be stable and difficult to corrupt or destroy. However, a journaling datastore is a little cluttery, in the sense that it's not that easy to delete things when you want to. However, with a bit of arm-twisting, I got my husband (who wrote the lower-level part of NarraFirma) to help me write some code that will help you clean house. You can read about the new housecleaning options in the blog post.

Usability improvements

In general, while I worked on this update to NarraFirma, I remembered all the things people said to me about it over the past year or so. I remembered their confusions and gripes, and I tried to address them all. So you will find many little improvements to explanations and placements of things. 

Here's one example: "quick links" on the Home page to the major things you created in your project (questions, forms, reports). 


I've also reduced the irritation factor in several places. To give just a few examples:
  • You can set a story count lower limit for drawing graphs, so you don't have to wade through graphs that have barely any stories in them.
  • You can now sort questions on your story cards by the original survey order as well as alphabetically.
  • Buttons for import and export have been moved to better places (I noticed that people forgot where they were).
  • In the node.js version, site administrators now have a link to the site-admin page right on the Home page of every project.
And so on. There's lots more stuff like that. 

Finally, I would like to thank everybody who paid me to improve NarraFirma. We have all benefited from your generosity. Here's to the future development and growth of NarraFirma!