Weather and Uncertainty: Warn or Wait?

Crossing the Rubicon?
Crossing the Rubicon?

One of the striking things that I learned about Americans when I began doctoral studies in the field of Communication is that there is a positive identity function to talking about the weather. If you’ve got to interact with a stranger one thing we all experience is the weather. Rather than being superficial, talking about the weather is a small ritual of interpersonal communication done in similar ways by so many different kinds of people that it aggregates into the significantly large effect of contributing to a common sense of shared national citizenship.

Source: http://www.ncdc.noaa.gov/oa/reports/billionz.html (retrieved 4 September 2011)
Source: NOAA (retrieved 4 September 2011)

The economic cost of big weather events is increasing. What about the social costs? What I mean by “social” are the quality and types of relationships among individuals in the United States, between these individuals and the publics we make up and belong to, the scientists and businesspeople involved in the weather enterprise, and among all these groups and the government. Whether or not the degree of public awareness and engagement with the weather is indicative of climate change or is merely a statistical blip that will wash out over time, media hype and active debate suggest a ripe opportunity for intervention in improving the emergency infrastructure so that everyone can better prepare and respond more resiliently to severe weather events (and other disasters).

Problem Definition: A Matter of History

Anyone who’s done research knows that how you ask the question has a lot to do with the results. The challenge of intervention is to ask the right questions; to ask these questions at the right times and in the right places, among the right people, and about the right thing/s; and then to see the discussion through. In Thinking in Time: Uses of History for Decision-Makers, Neustadt and May explain and illustrate the necessity for conceptualizing in terms of timestreams.

Thinking of time in such a way appears from our examples to have three components. One is recognition that the future has no place to come from but the past, hence the past has predictive value. Another element is recognition that what matters for the future in the present is departures from the past, alterations, changes, which prospectively or actually divert familiar flows from accustomed channels, thus affecting that predictive value and much else besides. A third component is continuous comparison, an almost constant oscillation from present to future to past and back, heedful of prospective change, concerned to expedite the limit: guide, counter or accept it, as the fruits of such comparison suggest. (1986:251)

I summarize Neustadt and May’s mini-methods, and argue for their application as a strategy for design, in this blog entry: Implementing an Organizational Vision through Thinking in Time.

Problematic Moments Signal the Potential for Powerful Change

WAS*IS problem definition exercise, 10 August 2011, at UCAR in Boulder
WAS*IS problem definition exercise, 10 August 2011, at UCAR in Boulder

The most lively discussion held at the 2011 WAS*IS Weather and Society Summer Workshop involved how to improve tornado warnings. Even though I was not formally in my action researcher role, I had been authorized to do ‘live blogging,’ so I was taking copious notes and paying close attention to the discourse and dynamics as they were unfolding. After this activity on August 10th (the 6th day of the workshop), Bob described it as “a barnstormer of a discussion” and Justin noticed a shift: “It’s all been happy-go-lucky so far, [and now] we’re going to have some disagreements – but the good thing is we all respect each other.”

watch your headFrom my vantage point, we had just gone through a group-level problematic moment. I’ve lived through several of them, and so have you – any time a whole group suddenly falls silent, or most members of the group spontaneously burst into talk – some underlying issue that effects everyone has somehow been tapped. James Cumming suggests these are “the same kind of moment that Bergson calls “durée:” a moment of silence that prepares the way for discourse, possibly new discourse, and with that the possibility of change.” [Noise is another kind of silencing: din obliterates sensible sound.]

Usually problematic moments are passed over, politely or awkwardly ignored. In our case, the discussion had already gone longer than scheduled and intruded into the break time. The moment of simultaneous talk became the excuse to end the activity and move into break and on with the rest of the program. Exceptionally, however, the vast majority of WAS*IS workshop participants kept talking with each other: over the next few minutes I counted at least thirteen animated interactions, from pairs to trios and one group of four, whose conversations continued as if there had been no interruption. If only we could have captured each of those unique conversations!

tornado shelterBarnburning: The “Warn on Forecast” Concept for Tornadoes

I typed as quickly as I could. This section is mainly the description provided by Workshop Leaders for a heuristic activity regarding problem definition. We were not supposed to try and solve a particular problem, rather, we were charged with the task of applying our collective intelligence to as many components of the issue as we could imagine to question.

ACTIVITY OVERVIEW:
a) The current system accounts for detecting a tornado threat (and issuing a warning) 0~45 minutes before it hits.
b) 10+ year goal is to increase this to ~2+ hours . . .
c) How????? Through better models, etc…. [although] “we have models that are coming out of our ears”

DISCUSSION ENSUES:

  • Kenny clarifies: a specific storm that does not exist yet, a particular threat in a given location…
  • Dan N: an area, maybe Boulder …
  • The default WarnGen shape – “I’m not crazy about that shape, but that’s sortof what they look like…”
  • Current system:
    • Assumes the public is homogeneous
    • ‘one size fits all’ – “you’re either in the warning or not, at risk or not
    • Purely meteorological polygon:
      • Purely bimodal
      • YES you are in or NO you are not
      • Orange dot could be (a popular state park, boy scout camp, county fair, mobile home park, school, hospital)
      • “ugly tornado, or we think it might be”
      • This is about the communication of uncertainty
      • Jay: “you might want to show where the expected tornado location is”
      • Bob: “I have a problem with that orange dot, apparently I have less value (having a bbq with my family) than the Boy Scout Camp.”
      • Mark: “How many dollars it costs for every hour somebody is under a tornado warning” ~ there is “an economic cost involved.” “It’s not a no-brainer.”
      • Jamie: “I’m being undervalued because I’m not in the polygon, but putting you in the polygon means increasing the area of uncertainty” …. five times you’re in the polygon, nothing happens, the 6th time you decide to ignore the warning…
      • Assign percentages?
        • Blue Dot: State Park
        • Green Triangle: Mobile Home Park
        • Red Star: State Fair Grounds
        • Green Square: College Football Game
        • “What is your probability threshold?” “How sure are you that locations inside the box will receive severe weather?” At receding distances beyond the boundaries of the polygon…
        • Greg: specific only to this one storm, not taking into account future/other storms
        • Dan N: only one/current to keep things simple
        • Talia: “How fast might a storm like this move,” influencing when you would move the boundaries…
        • Dan N: 20-30 mph (roughly)
        • Kenny: maybe there’s an on-the-ground report
        • Robert: that measure of speed is an average, some can move up to 70 mph
        • Dan N: just a simplified model for the purposes of the exercise: “Storms can do all kinds of crazy things.”
        • Ben: “Is that more of a fear that meteorologists will miss an event that impacts a lot of people, or is it to make the overall system more dynamic?”
        • Dan N: “Hold that question!”
        • Dan N: add  intensity of tight game – Buffaloes vs Huskies
        • Matt: are they playing in Lincoln?

Probabilistic Hazard Information (PHI)

At this point, it seems most of us have grasped the instructions and the scope of the example.  Now the types of question begin to shift, becoming more diagnostic: the group begins to address the task and interrogate the scenario.

  • Uncertainties/probabilities addedtornado outbreak
  • Longer lead times
    • Warnings for lesser certainty can be issued
    • 1-by-1 pixels

Holly: What happens if the storm regenerates? Could be misleading to the people in the blue?

Dan N: It’s extremely more complicated than this; storms could be popping up all over the place….”maybe in your discussions, you could kick all these ideas around” . . . presented this way “for simplicity’s sake” . . . aware “maybe that’s the problem”

Spinney: “Is each a different product?”

Eve: We had an advanced WAS*IS  about this in 2008.  “I thought, people just need to know each other, as soon as this gets explained, social scientists, emergency managers, hospital administers… total flashback, and a little of post-traumatic stress….. an anthropologist and others asked, “Why didn’t you ask us what we might need?” “why did you assume that this is what we wanted?” . . . “No one has really recovered from this yet….” Our assumption that …. How much trust in the NWS …. It would be magic.  “This is really hard; if we could make progress on this, it could really change the way these things go: we’ve got private sector people in the room, more social scientists in the room, much more sophisticated understanding… the dream was, we’ll explain it to you and you’ll make it easier for us….”  Look at webpage for 2008, _______ (?)…. She doesn’t do this anymore, she went back to lightning. This was too hard.”

Susanna: one of the potential mistakes here, going from warm colors to cool colors, these things we’re used to from Weather.com – intensity indicated by hot colors…. If blue I’m thinking the weather is going to be mild – the color scheme of a storm

Dan N: “light rain….moderate…. etc:

Robert: the color scheme is resonant of what the Dept of Homeland Security used… we were always on green or blue?

Kenny: rules of cartography were not considered when these codes were put together – they spend a lot of time learning about design: hues, color, intensity, perceived meanings… “a mismatch between the product and what’s intended”

Dan N: “be careful quoting me on representing a weather forecast” because that’s not the point here, which is to get us into an exercise

  • 1 km by 1 km grid boxes,
  • this product will give me a percentage, e.g., 17% of being hit by a tornado,
  • two primary issues –
    • 1.  much longer lead time of 2-3 hours, but then we have
    • 2.  increased the probabilistic warning information

Bob: I’ll start looking for secondary info, the longer lead time will reduce the urgency of threat . . . weaken the intent of warning

TASK PROMPT: Do we Warn on Forecast?

ZillianalienThe group is immersed in imagining variables, conditions, assumptions, the breadth of brainstorming is phenomenal. Some members of the group begin to question why, if, when, how, and who should be warned – or not.

  1. What problems were defined?
  2. Does this concept appear to solve the defined problems?

Susanna: This is focused on problem at the local level; but what are the factors at the higher level? Do we have to stay in this paradigm, what do to make it better? How introduce new factors at the higher level – because this isn’t even the paradigm we want to be working within?

  • Who needs more lead time? Football stadium… it takes a long time to evacuate…
  • Gaby: clarification ~ issues addressed in the powerpoint or the exercise?
  • Rebecca: both
  • Bev: we’re assuming lead time will make a difference? Will lead time move into actions that people can take to protect themselves?
  • Gaby: who is this for? May benefit some individuals more than others ~ meteorologists, or comm between emergency managers, but not the public
  • Dan N: generic answer is for everyone – currently tornado warnings are for everyone.
  • Rebecca: audience matters
  • Gaby: if we’re thinking public there are things to consider, if emergency mgmt. personnel there are others to consider
  • Chad: a convective outlook (meteorological) – the meat of the argument…. But jargon-laced to the public. More detail oriented for those people and more general for the public, some nerds who will love looking at the convective outlooks ~ maybe that’s an analogy?
  • Dix: a lot of those products were not made for the public, but they are out there…. We still need them, do you use them/convert them for use by emergency managers….
  • Chad: Interpret and spin into message for their audiences?
  • Dan N: from a physical science perspective, this all makes great sense
  • Greg: will the idea of uncertainty and probability be understood? In the location? In the timing? Where is the uncertainty? Will it appear?
  • Susanna: would it be helpful to communicate the numerical probability?
  • Rebecca: meteorologists assume this will happen, if communicate the numbers it will work
  • Susanna: doesn’t work
  • Greg: that’s the assumption
  • Jay: “whenever we do something like this in real time, we disregard the bigger threat on the north side of that storm which is the hail…. Telling people to get indoors but we should tell ‘em to get their cars indoors to protect from the baseball sized hail”
  • Talia posed something
  • Dan N: why issue tornado warnings in the first place
  • Jamie: give people the info and let people make the decision
  • Rebecca: empower decision-making as the goal or “I know what the right thing is, you do it?”
  • SJK: should be recording this conversation
  • Susanna: bureaucracy
  • Brittany: “will see what I can do”
  • Ben: who decides? Authorities or whose responsible for their family – we’re experts, analogy with a doctor, you need this surgery or here’s the options with their probabilities, people want advice that’s what a tornado warning does…
  • Dan N:
  • Susanna: where we’re miscommunicating is audience ~ doctor, patient, FEMA, hospital administrator, we keep forgetting which public when giving examples, the assumption keeps assuming there is one audience, how do we adapt products to multiple audiences rather than just one
  • Dan N: if we go this route, what Eve was talking about… if we go this route, how (and what effects)
  • Susanna: You’ve shot yourself in the foot, “this is for everyone in the US”
  • Dan N: defends
  • Susanna: must have different entry points in web design, cannot start
  • Greg: people have scales too, just like weather does
  • Justin: what is a problem, what do we do with: bimodal you get hit or not by a tornado same with rain… must have confidence levels – problems with the current model; nothing else in weather is done bimodally
  • Robert: Check out the Warn on Forecast webpage
  • http://www.nssl.noaa.gov/projects/wof/
  • Dan N: let’s not get stuck there
  • Rebecca: intriguing to analyze their assumptions
  • Brittany: assuming that the public knows what their context is – people know if they’re in a football stadium
  • http://docs.google.com/viewer?a=v&q=cache:cMf0I_UPv38J:www.nssl.noaa.gov/news/backgrounders/WoF%25202011.pdf+warn+on+forecast+backgrounders&hl=en&gl=us&pid=bl&srcid=ADGEEShDkbinbS22ye8afFsm-GdWupuCoCj6L-z6TS-WjzfZO8D0Nh3acfGnIOwmSAmh6VaR2Ti6tlJznFdlFxoUMxzVISxVpmM6bvI1KDWtQfXJFNmP_GHZxjsoNYnZmq3CYePIKbxh&sig=AHIEtbRGaQTjRVWyF1ErssSC7xr9TGNzCg
  • Dan N: you’re bringing it to another level, how could it be done better (isn’t this what we’re supposed to be doing?)
  • Bob: are they just doing this for themselves to show people how smart they are? They’re missing it now, more of the same will help? Not.
  • http://www.nssl.noaa.gov/news/backgrounders/WoF%202011.pdf
  • Jamie: both, ½ the org has one motivation and ½ another (to help the public or show off)
  • Alan: if people aren’t doing something at 100% why are we thinking they’ll do something at 17%?
  • Bev: belief that more/better will be sufficient. What do we know that will empower people? A personal relationship? Something else?
  • Dan N: “We can definitely kick around a lot of problems, but does it solve it?”
  • NO.
  • Kenny: “bring out some of the inherent tensions: In science, if someone asks me to define a problem, what are some…. E.g., “Do people have enough lead time?” there are assumptions in there…. Are these even valid questions to be asking?”  What the weather service would like, and what problem needs to be solved?  “Is this even a problem?” We need a statistical technique to show this before we go to the second stage…. Academics are asking, ‘what the hell are we doing?’ Need to apply a null hypothesis test to, before we even proceed.
  • Rebecca: often we have the data that shows how people do/don’t respond… we have contra-evidence
  • Kenny: assumption that the public needs tornado warnings – I’m not sure this has even been demonstrated. Juicy meta-problems… as we’re attempting to uncover the comments
  • Rebecca back to Jay: what are the threats we’re supposed to be addressing?
  • Dan N: the physical scientists bring up the natural phenomena (hail, etc), social scientists – do people even understand these things?
  • Dan N: “Well, it’s 10:00.”
  • Rebecca: validity – some scientific accuracy or truth, the probabilities are not well-calculated…. Not scientifically-sensible.  “You can’t assume the science is perfect, you make a lot of assumptions with science.”
  • Dan N: current RQ ~“How to draw this box?”
  • Matt: “I’m sitting here as a broadcaster feeling job security. You cannot make a policy that is going to fit every situation. There is always going to be humans involved, no matter how good computers and modeling gets – you can’t cover every situation that comes up…”
  • Rebecca: what is the larger problem definition?
  • Dan N: “This is a real one. This is real life.”
  • Jamie: “When it comes to high profile warnings, that really is the identity of the agency. A lot of people are going to come to the table with tremendous emotional attachment or baggage, to them, this is who they are. The tornado or hurricane warning is how they define themselves.”
  • Dan N; “Meteorologists are almost born thinking, ‘It would be great if we can give 5 hours lead time…’”
  • Greg: how organic systems interact: atmosphere, oceans, cryosphere, land – a classic graphic. The human aspect needs to be put into that five-part connection; the problem is that we’re dealing in two different systems that are always evolving, we always have to make assumptions to simplify…
  • Jelmer: “everybody needs coffee, I know” “ We’re discussing the things, but they are obviously not isolated to this room, there are people doing research about it, already, I hope… a list of references? Otherwise we are talking about it now and maybe forget it.”
  • Bob: question, insane, bear with me: “Does  a reduction in lead time increase probability increase risk…. Short-term decision-making… expose people to greater risk…. Short-term lose more lives & property but over time…. “ a recency effect….
  • Steph H: larger venue decision-making for large institutions: schools etc ~ some sort of probabilities info could help, they may make better decisions than the usual public
  • Susanna disagrees: school principals? Probably not
  • Steph H: longer lead time, seems less life-threatening, not beneficial, more adverse effects (their speculations)
  • Bev: school officials may have more anxiety
  • Susanna: making a joke, can’t assume people with more education automatically understand probability, but some audiences do need more lead time. Bob – for some people it won’t make a difference but for some it will, the hospital administrator needs it
  • Brittany: for emergency mgmt a very useful tool, not the general public
  • Chris: I can see giving the info to an elite group, then feeds rumors – we’re preparing for such and so, but nobody knows, “It’s gonna get out whether we put it out or not”
  • Amy: ethical, legal
  • Chris:

PROBLEMATIC MOMENT!

road sky thunderstorm

Several talking same time: Justin-Chris, Amy-Robert, Steph H (at least)
Rebecca H – the importance of the proactive step
(Mark-Dix, Chris, Dan N, Kenny…Rebecca….)

SLOWAn ethical dilemma – withhold info?

Tell who? (and who not?)

Ownership of taxpayer to the info, responsibility to provide ~ Chris on the shock of imagining the possibility of not telling – proving Jamie’s point about identity.

2 thoughts on “Weather and Uncertainty: Warn or Wait?”

  1. Notes on a WAS*IS problematic moment

    The problematic moment is indicative of the tensions underlying the discourses taking place at this conference. These discourses are formed by and taking place within a context marked by these features:

    • A grassroots movement advocating for change in its profession
    • A long (8-day), interdisciplinary, scientific learning event
    • An effort to integrate social science into meteorological research and practice
    • Recognition by the sponsoring organization (Weather and Society*Integrated Studies) that it is part of a larger, social, economic and political framework which it calls “the weather enterprise.”
    • Members seek to address the societal impacts of “the weather enterprise” in real and sustained ways.

    My assumption is that there are a number of discourses being talked and accessed by the participants. For example, as a grassroots movement a discourse of equality and social action will be present that uneasily sits alongside the objective and rational scientific discourse of meritocracy. The emotional need to mitigate the harmful negative impact of climate change challenges the subduing emotional need for rationality and coherent argument. In addition, the various disciplines within meteorology and the social sciences all have their own discourses and terminology making communication and interaction a complex and uncertain process. Furthermore, there are issues of power, resources, influence and prestige that heavily influence the agendas of those communities of practice that are present.

    Judging from the extract of dialogue that Steph presents, the discourse of the conference seems to be all over the place with many competing genres. As agreement is unlikely in the absence of any process set up to achieve that goal, it is not surprising that the conversation should gradually reach a point where the group can no longer go on and everybody is talking loudly at the same time (the problematic moment). At such points, we can usually infer a number of competing issues at work: experts making decisions for the people’s own good vs. an approach that democratically involves the whole system of stakeholders in make decisions and disseminating information; a bias towards the potential economic damage vs. the potential loss of human life; government agencies in the role of tyrants vs. a benevolent role; matters of faith vs. scientific fact; expressing certainty vs. probabilities; simplistic descriptions of the situation vs. a portrayal of its complexity; not acknowledging the importance of issues of race, gender and class vs. attempting to deal with multiple perspectives, identities and privilege; and so on.

    The weather enterprise sets the boundaries of their competence and authority in forecasting the weather, and it is difficult to challenge them in these terms. However, the real question is not the probability of a devastating weather event, but what risk of bodily harm is acceptable to the population living in the target area. They may be willing to tolerate a larger risk of a small storm that causes few casualties, than the smaller risk of a huge storm that causes many casualties. We can draw a graph of this range of probabilities and acceptance. It is called a “risk function.”

    Contrary to what many experts believe, people do have the competence to assess what is an acceptable risk to their health and livelihoods. Many people play the lottery or bet on the outcomes of sport and do not find the task of assessing risk to be particularly problematic. For the weather enterprise technical, political and business interests are intertwined. The media usually has an agenda. As a result, we need to do more to democratize our decision making process about disaster preparedness in all its complexity by ensuring wide participation in decisions about what actions to take when there is a risk (no matter how small) of devastating harm from the weather.

    I understand the problematic moment occurred when a session had gone over time so it was natural to call a break when everyone started talking at once. This meant that the energy generated by the discussion was taken out of the group as a whole and into interpersonal interactions. An opportunity was lost to deal in the moment with the ethical issues raised that may have helped bridge some of the competing issues. However, the way the event was handled suggests to me that boundaries were not kept particularly well and that the group was attuned to handling process issues. This conference is probably typical of many academic conferences where the interplay of content and process issues is not well understood. Learning events need to be designed taking group dynamics into account because as Marshall Mcluhan used to say, “the medium is the message.” Discourse analysis is one tool that could be used to assess more deeply the interaction of discourses at work in the conference. From this understanding, I believe awareness would come of new ways of how the conference could be designed and facilitated next year in order to achieve its goals.

    James Cumming
    9/7/2011

  2. Hey James,

    Fascinating to read your diagnosis from a distance!

    I have a couple of responses, one is a bit defensive of how well the WAS*IS workshop was designed, particularly how generally unlike a regular academic conference: strong pedagogy, great scaffolding, and several NTL-like group development, closure and re-entry features. So, while the “interplay of content and process issues” wasn’t foregrounded, the structure was embedded with some sophistication about group dynamics.

    Where WAS*IS turned out to be like other academic conferences is in the absence of expectation for an outcome, a tangible product created or produced by participants. Now, the class of 2011 is actually trying to buck that trend by writing a White Paper and also constructing a collaborative website designed to serve some identified practical purposes . . . but these are both initiatives generated by participants spontaneously. Organizers allowed these developments but weren’t (as near as I could tell) prepared for them – the design did not call for the group to do anything in particular with the knowledge we gained over those eight incredible days. That is very academic – to stay in the realm of theory and research at the atomized individual level.

    The break down, then, of losing that moment to deal with the ethical issues of warning, is a feature of most academic settings where activity is geared more to talking about things rather than producing things. Future workshops may find it advantageous (or even necessary) to move toward establishing some kind of project or goal that requires the participants to act as members of a team. Another possibility is that other, different events, piggyback on work initiated here and move further along the path of problem definition.

    The other response I have is to your prescription for more democratization of risk assessment regarding weather. Wow – I can imagine a ton of weather enterprise products to meet this need! I think what you’re raising here is another, different angle that came up only tangentially during all of our discussions. Specifically, what if weather forecasting could be personally localized? The CASA project is testing radar technology that would improve prediction and reduce uncertainty considerably. But does everyone want that much information? Need it? Who will pay for it? Is trying to get more control the right answer? The authors of Resilience Thinking present compelling cases against too much control, but what, then, is the alternative?

    These questions need sustained engagement by a range of stakeholders. The mission needs to be defined unambiguously and facilitated in order to sort out the moral and political arguments. Then agreements about what can and can’t be done, within what parameters, can be forged together and transparently within an overall framework that has foundational support of a broad consensus.

Leave a Reply

Your email address will not be published. Required fields are marked *