COMM1110LectureSlidesWeek10_T3_2022.pdf

COMM1110 Evidence-Based Problem Solving

Week 10: Evaluation and communication

Lecturer: Kevin Liu

General housekeeping:

• Please switch your microphone to mute to avoid disruption to the class

• Use the chat channel to ask questions or make a comment, or raise your 'virtual' hand

• If you have poor internet, turn off your video

• Wait for your lecturer to start

1 2 3 4 5 6 7 8 9 10

LEC

TUR

EST

UT

OR

IALS

10Problem

articulation and

disaggregation

Frameworks for ethical decision-making

Understanding problems –

fact Gathering

Patterns, biases,

hypothesising effects

Flexibility Week

Analysing the issues –

Identifying the causes

Analysing the issues – With

limited evidence

Problem solving & making

decisions

Evaluation and communicate

Week 1Toolkit

application

Week 2 Toolkit

application

Week 3 Toolkit

application

Week 4 Toolkit

application

Week 5 Toolkit

application

Week 7 Toolkit

application

Week 8 Toolkit

application

Week 9 Toolkit

application

11

Problems & evidence-

based problem solving

11

ASS

ESSM

ENTS

COMM1110 EVIDENCE-BASED PROBLEM SOLVING

Assessments

Week 10 Toolkit

application

Briefing Pack: 25%

Associate level: 10%

Expert level: 10%

Business Report:

40%

Please see COMM1110 Assessment Guide for details including exact deadlines

Excel Training Program: 10% + 10%

Case: 25% + 40%

Online Discussion Questions: 15%

This week:

Bullet Proof Problem Solving

Framework

Problem Solving Tools

Case studies and examples

• Evaluating Evidence

• Boundaries of Problem-

solving

• Synthesis and

Communication

• Furniture store

This week will focus on:

Problem-solving step

1. Scoping (define the problem,

disaggregate)

2. Analyse (prioritize, workplan,

analyse)

3. Decision (synthesize,

communicate)

This week’s to-do list:

• Attend: Synchronous (live) lecture online. You can access the zoom link underneath the Lecture and Tutorial Links Moodle section.

• Watch: Recording of lecture, if you missed the lecture or need to do revision.

• Attend: Synchronous (live) tutorial. You can access the zoom link underneath the Lecture and Tutorial Links Moodle section.

• Provide feedback: Tell us how Week 10 went for you.

• Assessments: continue completing the Case Report (Assessment 2b)

• Reliability and validity

Evaluating Evidence

Reliability and validity

Reliability – Consistency Validity – Accuracy

Reliability ≠ Validity (i.e. getting consistent results doesn’t mean that your results are correct!)

Reliability

Reliability refers to how consistently a method measures something, there are 3 types of consistency:

Type of reliability Measures the consistency of… Examples

Test-Retest Over time

A measure of stability

Multiple blood tests for

diabetes show same results

Internal Consistency Across items

A measure of how consistently

each item measures the same

underlying construct

Customer satisfaction

survey: multiple questions

on the same issue are

answered with same

responses by the

respondent

Inter-Rater Across different observers

A measure of agreement

Multiple judges in a sporting

competition that arrives at

same or very similar score

Validity: internal vs. external validity

Validity: relates to how well the results of a study accurately reflect the “truth”

Internal validity: relates to how well a study is conducted – are we sure that we know what caused the results of the study?

• e.g. Are we sure that the low customer satisfaction is caused by long waiting time?

External validity: relates to how applicable/generalisable the findings are in the real world

• e.g. This study shows that the low customer satisfaction is caused by long waiting time in our Kensington store. Is this finding also applicable to our Town Hall store?

Internal Validity

The extent to which a piece of evidence supports a claim about a cause and effect, within the context of a particular study

It can also be determined by how well alternative explanations for the findings can be ruled out

Cause Effect

Training staff on how to deliver

company specific customer service

procedures e.g. greeting, follow up,

after sales service

High standardisation of customer

service delivery across different

customer service representatives

Offering a bonus on the sale of

specific furniture items

Increase in the sale of specific

furniture items that are tied to the

bonus

Brewer, M.2000. Research Design and Issues of Validity. In Reis, H. and Judd, C. (eds.) Handbook of Research Methods in Social and Personality Psychology. Cambridge: Cambridge University Press.

Internal Validity

• Have just stressed the role of sampling error

− This is uncertainty that is unavoidable because we are dealing with a sample and not the entire population

− This by itself does not invalidate the inferences we make

− Other types of error exist and may be more problematic

• Recall our discussion of confoundment

− Is the evidence that is being considered free of this (and other) problems?

− This is the issue of internal validity

• A statistical analysis is said to have internal validity if inferences are valid for the population being studied

− Confoundment leads to biased estimates of the population parameters of interest

Internal Validity

Sample selection as a threat to internal validity – e.g. survivorship bias (non-representative sample)

Internal Validity

Sample selection as a threat to internal validity – e.g. “self-selection” (non-representative sample)

• Another common sampling problem occurs where the sampling units are “self-selected”

• Examples occur where the sampling unit “chooses” to be part of the sample

• In such cases the reason for self-selecting may be related to the outcome being measured and so bias may again occur

Internal Validity

Internal Validity

Internal Validity

External Validity

The extent to which the findings can be generalised to other situations, people, stimuli and times.

Australian Housing Market Example

Sydney to Brisbane property prices Low external validity since Sydney and

Brisbane contrast greatly with population

size and demand

Aronson, E., Wilson, T. D., Akert, R. M., & Fehr, B. 2007. Social psychology. (4 ed.). Toronto, ON: Pearson Education

Sydney eastern suburbs to western

suburbs

Some external validity but still large

differences in demand and income

Sydney eastern suburbs (e.g. Bondi

Junction) to eastern suburbs (e.g.

Maroubra)

High external validity since suburb

demographics, demand and income very

similar

External Validity

• A statistical analysis is said to have external validity if inferences and conclusions can be generalized from the population studied to other populations and settings

• Evaluating external validity will often be very subjective but will typically require detailed knowledge of the context and subject matter

• Inferences from well-designed and conducted experiments provide evidence with good internal validity

External Validity

• It makes sense to analyse house prices separately for capital cities

• The differences in Sydney and Brisbane prices are large and likely reflect population differences in price distributions

• It would be inappropriate to infer much about Brisbane prices on the basis of a sample of Sydney prices

External Validity

• If similar findings emerge from studies on different but related populations then this supports external validity

• Recall our discussion of accumulating evidence

• A formal comparison of related studies in order to detect commonalities is called meta-analysis

• An important topic but beyond the scope of the course

• Cognitive biases continued

• (Some) solutions to biases involve gathering feedback• Individual feedback to build self-awareness

• Team reviews to improve collaborative processes and tasks

Boundaries of problem-solving

Recall bounded rationality (from Week 1)

• Our ability to be rational problem solvers is limited by numerous constraints

• Common constraints are:

ComplexityTime & money

constraints

Cognitive capacity,

values, skills habits and

unconscious reflexes

Imperfect information

Information overload

Different priorities

Problem is complex and hard to understand

Not enough time and/or money to gather information

People have varying degrees of abilities, biases and heuristics

Available information is fragmented

Too much information to process

Some data perceived to be more important, so certain facts ignored

Cognitive biases: Extending Week 1

• See asynchronous video on Moodle for more details and examples:

Sunk costs bias Escalation of commitment

Over-confidence biases

Hindsight bias Groupthink

The tendency of decision makers to be influenced by

the way a situation or problem

is presented to them.

Where decision makers increase

their commitment to a project despite

negative information about

it.

The bias in which people’s subjective confidence in their decision making is greater than their

objective accuracy.

The tendency of people to view events as being

more predictable than they really

are.

When group members strive to

agree for the sake of unanimity

and thus avoid accurately

assessing the decision situation.

Overcome biases

First step: gather feedback to build self-awareness

• Feedback is a powerful tool to facilitate thinking and changes in behaviour that produce improved outcomes.

• Johari window highlights different ways self-awareness can grow

• In COMM1110, a peer feedback tool was used in week 9 tutorial

The Johari Window

• Synthesis of findings

• Telling compelling stories – communication• Using logic tree structure pyramid to organise a compelling

story

• Your one-day answer structure of situation-observation-resolution is the starting point for the governing thought of your narrative

• Try several storyline structures to see which are most clear and compelling

• Influence tactics – the power of persuasion

Synthesis and Communication

Synthesis and Communication

Final stage of good problem-solving

• Synthesising your findings (in a way that highlights your insights) and telling a compelling story (i.e. answer your decision maker’s question, “What should I do?” in compelling way that motivates action)

• Done right, your conclusions are an engaging story, supported with facts, analyses, and arguments that convince your audience of the merits of your recommended path

Synthesis of findings

• The quality of your synthesis will determine the quality of your decision making

• First step toward compelling story telling: synthesise the findings from the relevant literature, your data gathering, interviews, analysis, and modelling (i.e. all the evidence you collected)

• Synthesis brings together all the separate pieces of evidence (e.g. literature or your analytic work) and often yields new insights you didn't notice when you were in the weeds of analysis

Synthesis of findings

Start at the end

In consulting firms, consultants typically:

• Start writing your final presentation before you even meet the client!

• Have early-stage hypotheses ready (even though some of them might be wrong), so you know what questions to ask the client when you meet them

• Imagining a set of hypotheses allowed the team to begin focused data gathering from the first meeting

Before moving to final synthesis

• Iterative problem-solving process: interaction of the hypotheses with the analyses

• Alternative hypotheses should already have been tested, and embraced or rejected (and dead-end analyses pruned off)

• Link analysis and emerging findings in a convincing way

Synthesis of findings

Final synthesis

• Draw together the individual findings of the work on each branch and twig of your logic tree into an overall picture

• Represent each of your findings in the form of graphics that highlight the insights that emerged from your work

• Where possible, the most powerful visualisation is to show each graphic as branches on your revised tree structure

– This kind of visualisation of findings illuminates the overall problem.

– It allows for insights to emerge that cut across and combine the different branches of analysis → the overall story typically begin to emerge from individual branches

Conn, C. & McLean, R. 2019. Bulletproof Problem Solving The One Skill that Changes Everything, John Wiley & Sons Inc (US)

Synthesis of findings

Final synthesis

The synthesis in pictures (previous slide) begins to tell a visual narrative (which helps you to draft your final report)

Draw graphs from the analysis that synthesise your findings also allow you to pressure test your solutions with your evidence and analyses

• Do these graphs support what you’ve concluded? If not, ditch them.

• Is there a really interesting point you made about a factor, but you don’t have a chart on that? Get that in the pack to support your findings.

• Go through this pressure test yourself and/or bring the whole team (if you solve a problem in a group) together for synthesis sessions to synthesise findings until you have a more complete and supported argument (from the simple situation-observation-resolution story summary)

Telling a compelling story

Once findings have been synthesised, structuring a compelling story is key in communicating to your audience

Move from simply stating the implications of analyses (early stages) toward how those results can motivate a plan for change (final stage) – i.e. What should we do and how should we do it?

Conn, C. & McLean, R. 2019. Bulletproof Problem Solving The One Skill that Changes Everything, John Wiley & Sons Inc (US)

Telling a compelling story (e.g. written report)

From one-day answers to pyramid structure

• Once you have synthesised your findings → move to the final step of

structuring a compelling story for your audience

Start from a story summary:

• Start from a simple situation-observation-resolution story summary

• Situation: summarising the situation allows us to update our best

understanding of the problem

• Observation (or complication): provide the tension in the problem,

what isn't working, and our best insights into ways to unpick it;

• Resolution is our best understanding of the solution path that moves

us toward the answer

Telling a compelling story (e.g. written report)

e.g. situation-observation-resolution story summary

Conn, C. & McLean, R. 2019. Bulletproof Problem Solving The One Skill that Changes Everything, John Wiley & Sons Inc (US)

Telling a compelling story (e.g. written report)

Structures for organising the storyline

Conn, C. & McLean, R. 2019. Bulletproof Problem Solving The One Skill that

Changes Everything, John Wiley & Sons Inc (US)

• The standard visual structure we use to represent our story structure is a pyramid structure (which helps to structure arguments and support into a powerful story)

• The basic pyramid structure:

– is just the logic tree on its side

– enables an organised structure that links arguments, data, analyses to overarching argument at the top

– helps show clearly how each element of our argument is supported by evidence

Telling a compelling story (e.g. written report)

Structuring your arguments

• Choose grouping structure or argument structure

• There is no general preference for these structures (your decision)

• Try several storyline structures to see which are most clear and compelling

Conn, C. & McLean, R. 2019. Bulletproof Problem Solving The One Skill that Changes Everything, John Wiley & Sons Inc (US)

or

Telling a compelling story (e.g. written report)

Structuring your arguments – e.g.

Conn, C. & McLean, R. 2019. Bulletproof Problem Solving The One Skill that Changes Everything, John Wiley & Sons Inc (US)

Recommendation: Hechinger needs to change its business model quickly to address the competitive threat of Home Depot.

Telling a compelling story (e.g. written report)

Structuring your arguments – e.g.

Recommendation: Oilco (the refinery business) needs to cut costs substantially and become a modest growth, niche operator

Conn, C. & McLean, R. 2019. Bulletproof Problem Solving The One Skill that Changes Everything, John Wiley & Sons Inc (US)

Persuasion in written communication

Logic

• Persuasive report writing requires the presentation of ideas through reason and logic, in order to influence the audience

• Persuasion may use an argument to convey an idea or to convince the audience to perform a certain action

• It is an art of using logic, demonstration of credibility and invoking emotions to effectively communicate the argument

Storytelling

• Once you get your logic right, you also need to work on telling a compelling story – i.e. answer your decision maker’s question, “What should I do?” in compelling way that motivates action

• Humans are storytelling creatures, not logical robots (remember this as you craft your messages!)

Problem-solving step

(bullet-proof problem solving in

brackets)

Critical thinking tools

to analyse different forms of evidence

Information tools Ethics tools Statistical tools

1. Scoping (define the problem,

disaggregate)

• 5Ws

• Cognition & problem-solving styles

• Library skills to review literature

• Assessing the situation with moral

imagination (step 1)

• Assumptions and worldviews (step

2)

• Descriptive statistics

• Graphs: frequency distributions,

bar charts, pie charts & histograms

2. Analyse (prioritize, workplan,

analyse)

• Logic trees – reiterative process

• Heuristics

• Pattern recognition:

− Novice vs expert

− Inductive, deductive, abductive

• Thematic analysis

• Fishbone tool

• Causality with quantitative data:

research design (including

experiments, confoundment &

reverse causality)

• Principles, duties and care needs

(step 3)

• Process, outcomes and

consequences (step 4)

• Character factors (step 5)

• Comprehensive assessment (step

6)

• Scatterplots and correlations

• Regression

• Probability

• Normal distribution

• Sampling distribution – Central

Limit Theorem

• Inferential statistics

• T-test, regression

3. Decision (synthesise,

communicate)

• Collaborative decision-making

techniques

• Evaluating evidence: reliability and

validity

• Boundaries of decisions: cognitive

biases

• Using logic tree structure pyramid

to organise a compelling story

• Justify your decision (step 7) • Hypothesis testing

• Confidence intervals, statistical

significance, effect sizes

• Boundaries of decisions: statistical

biases

COMM1110 Summary

Thank you

If you have any questions about the

course, please email:

comm1110@unsw.edu.au

The lecture recording will be made

available in your Moodle course site.

The Survey is now open!

Tell us about your experience and shape the future of education at UNSW.

Log-in on your laptop, tablet or phone:• Link on your Moodle course page; or• URL http://myexperience.unsw.edu.au use (zID@ad.unsw.edu.au).

The myExperience survey is anonymous and brief. It will only take about 2 minutes to complete.

Each session feedback is sought from students about the course and continual improvements aremade based on your feedback. We take your feedback very seriously and respond to your commentsand suggestions in future course design. Your feedback is very important to us. We look forward tohearing your feedback.

Calculate your order
Pages (275 words)
Standard price: $0.00
Client Reviews
4.9
Sitejabber
4.6
Trustpilot
4.8
Our Guarantees
100% Confidentiality
Information about customers is confidential and never disclosed to third parties.
Original Writing
We complete all papers from scratch. You can get a plagiarism report.
Timely Delivery
No missed deadlines – 97% of assignments are completed in time.
Money Back
If you're confident that a writer didn't follow your order details, ask for a refund.

Calculate the price of your order

You will get a personal manager and a discount.
We'll send you the first draft for approval by at
Total price:
$0.00
Power up Your Academic Success with the
Team of Professionals. We’ve Got Your Back.
Power up Your Study Success with Experts We’ve Got Your Back.
Open chat
1
Hello. Can we help you?